A method for generating a saliency map, a method for detecting abnormal objects, and a device

Abstract

The embodiment of the present application relates to the field of artificial intelligence, and discloses a method for generating a saliency map. The method includes: performing a perturbation process on a first object to obtain multiple objects. Screening the multiple objects to obtain multiple updated objects. The updated multiple objects satisfy the target data distribution, and the training samples of the target model also satisfy the target data distribution. The updated multiple objects are input into the target model to output a first prediction result. A saliency map of the first object is generated according to the first prediction result and the updated multiple objects to improve the accuracy of the saliency map.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method for generating a saliency map, a method and a device for detecting abnormal objects. Background Art

[0002] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0003] The most well-known drawback of AI is its "black box" nature, which means that users do not know how and why the model comes up with a certain output. For example, when a user inputs an image of a cat into a classification model and the classification model predicts that it is a car, it is difficult to understand why it was caused to make this prediction. In the past few years, AI has made remarkable achievements in many applications in different fields. However, as people rely more and more on machine learning models, and the decisions from these models affect human life, how can we ensure that the decisions made by AI are trustworthy? This is the biggest problem with current artificial intelligence based on big data and deep learning: unexplainability. In other words, the lack of transparency and explainability of the model will seriously affect the user's trust in it, which will in turn limit the application and development of the model in real-world tasks (especially risk-sensitive tasks such as driverless driving, healthcare, financial tasks, etc.).

[0004] Because black box models do not have inherent interpretability, we need to use saliency maps to obtain which features have the greatest impact on the model output. Therefore, how to make the saliency map accurately reflect the impact of features on the model output needs to be solved urgently. Summary of the invention

[0005] The present application provides a method for generating a saliency map, a method and a device for detecting abnormal objects, which can improve the accuracy of the saliency map.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a method for generating a saliency map, the method comprising: obtaining multiple objects, wherein the multiple objects are obtained after perturbation processing of a first object. Among them, perturbation processing of the first object may include multiple ways, such as noise processing of the first object; or blur processing of the first object (such as changing the values ​​of multiple pixels within a preset range); or occlusion processing of the first object (such as randomly sampling multiple pixels for occlusion), etc. The first object may be image data, text data or voice data. The multiple objects are screened according to a first condition to obtain multiple updated objects, and the multiple updated objects meet the target data distribution, and the target data distribution is obtained according to the training sample. In other words, the data distribution satisfied by the multiple updated objects is consistent with the data distribution satisfied by the training sample. Each object in the updated multiple objects (or the object feature after feature extraction of each object) can be represented by a coordinate in a spatial coordinate system. In the spatial coordinate system, the space including the coordinates of all objects can be regarded as the data distribution satisfied by the multiple updated objects (hereinafter referred to as the first data distribution). Each training sample (or the training sample feature after feature extraction for each training) can also be represented by a coordinate in the spatial coordinate system. In the spatial coordinate system, the space including the coordinates of all training samples can be regarded as the data distribution satisfied by the training sample. When the deviation between the first data distribution and the second data distribution is within a preset range, it can be considered that the first data distribution and the second data distribution are consistent. In the scheme provided in the first aspect of the present application, the first data distribution and the second data distribution are both target data distributions. For better understanding, a specific example is used here to illustrate: assuming that the training samples are all animal images, these animal images satisfy the target data distribution, and the target data distribution can be used to reflect the characteristics of these animal images (or image features). The input of the trained model should also satisfy the target data distribution. For example, if the input is also an animal image (the model has learned the characteristics of the input object), the trained model can make accurate predictions for the input. However, if the input is a person image, it is obvious that the person image and the animal image have different characteristics, and the distribution of the coordinates in the spatial coordinate system should also be different. Then, the accuracy of the prediction result of the input person image by the model trained by the animal image will be greatly reduced.

[0008] The solution provided in the present application finds that the first object after disturbance may not satisfy the target data distribution, so the first object after disturbance is screened and processed to obtain the first object after disturbance that satisfies the target data distribution. The updated multiple objects are input into the target model to output the first prediction result. A saliency map of the first object is generated based on the first prediction result and the updated multiple objects. The solution provided in the first aspect is an improvement on the existing perturbation-based interpretable method. The saliency map generated by the perturbation-based interpretable method can be used to obtain which features of the first object have the greatest impact on the model output, and then the black box model can be explained.

[0009] The solution provided in the present application can improve the interpretability of models whose input is image data, such as improving the interpretability of image classification models. The saliency map obtained through the first aspect can better understand which part of the features of the image input to the image classification model has a greater impact on the output of the image classification model. The solution provided in the present application can improve the interpretability of models whose input is voice data, such as improving the interpretability of a multi-round dialogue model. The saliency map obtained through the first aspect can better explain which part of the features of the input voice the multi-round dialogue model is based on to interact with the user. The solution provided in the present application can improve the interpretability of models whose input is text data, such as improving the interpretability of machine translation. The saliency map obtained through the first aspect can better understand which part of the features of the input text has a greater impact on the output of the translation model.

[0010] In a possible implementation, the target model is obtained by updating the preset model by a first loss value, the first loss value is determined according to the deviation between the feature of the training sample and the weight vector of the preset model, the feature of the training sample is obtained after the preset model extracts the feature of the training sample, and the multiple objects are screened and processed according to the first condition to obtain the updated multiple objects, including: if the multiple objects include the target object, the target object in the multiple objects is deleted to obtain the updated multiple objects, the distance between the feature of the target object and the weight vector of the target model exceeds the preset threshold, and the feature of the target object is obtained after the target model extracts the feature of the target object. In this implementation, as the preset model is trained, the first loss value will become smaller and smaller, and the weight vector of the preset model will become closer and closer to the feature of the training sample. The preset model after training, that is, the weight vector of the target model can already reflect the data distribution characteristics of the training sample. In this implementation, the classification model only needs to be trained with normal samples to enable the classification model to distinguish between normal samples and abnormal samples, which reduces the training difficulty and training cost of the classification model, and can also improve the performance of the classification model.

[0011] In a possible implementation, the features of the target object are specifically extracted through a first feature extraction layer, which is any one of multiple feature extraction layers in the target model, and the distance between the features of the target object and the weight vector of the target model is specifically the distance between the features of the target object and the weight vector of a second feature extraction layer, which is any one of multiple feature extraction layers. In this implementation, the distance between the features of the target object extracted through any feature extraction layer and the weight vector of any feature extraction layer can be expressed as the distance between the features of the target object and the weight vector of the target model.

[0012] In a possible implementation, the first feature extraction layer and the second feature extraction layer can be set to be the same feature extraction layer according to the needs of actual applications, or the first feature extraction layer and the second feature extraction layer can be set to be different feature extraction layers according to the needs of actual applications.

[0013] In a possible implementation, the first feature extraction layer is specifically the penultimate feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is specifically the last feature extraction layer among the multiple feature extractions. In this implementation, a preferred first feature extraction layer and a preferred second feature extraction layer are provided, which increases the diversity of the solutions.

[0014] In a possible implementation, if the weight vector of the second feature extraction layer includes multiple, the distance between the feature of the target object and the target weight vector of the second feature extraction layer exceeds a preset threshold, and the distance between the feature of the target object and the target weight vector of the second feature extraction layer is the maximum distance among multiple distances, and the multiple distances include the distance between the feature of the target object and each weight vector of the second feature extraction layer. In this implementation, in order to better identify abnormal samples, in this implementation, it is determined whether the target object is an abnormal sample based on the maximum distance among multiple distances.

[0015] In a possible implementation, the target model is specifically obtained by updating the preset model with the first loss value and the second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the second prediction result and the preset function, the second prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the second prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function. In this implementation, a preset function is introduced in the training process of the model, the input and output of the preset function are negatively correlated, and different outputs are selected according to different inputs of the preset function. There is no need to manually adjust parameters, and different outputs can be automatically obtained according to different inputs, thereby avoiding the model from falling into a local optimum, slowing down the speed of model convergence, and making the performance of the trained model better.

[0016] In a possible implementation, the method further includes: setting the weights of the updated multiple objects to the first weight. Setting the weights of the remaining multiple objects to the second weight, the remaining multiple objects are the objects in the multiple objects other than the updated multiple objects, and the first weight is greater than the second weight. Inputting the first result and the second result into the target model to output a first prediction result, the first result is determined based on the first weight and the updated multiple objects, and the second result is determined based on the second weight and the remaining multiple objects. In this implementation, a larger weight is set for normal samples and a smaller weight is set for abnormal samples. In the process of generating a saliency map, normal samples are mainly used and abnormal samples are supplemented, thereby weakening the influence of abnormal samples in the process of generating a saliency map. The generated saliency map can better reflect the influence of features on the output of the model.

[0017] In a second aspect, the present application provides a method for detecting abnormal objects, including: obtaining multiple objects. Extracting features from a target object through a feature extraction model to obtain features of the target object, the target object is any one of the multiple objects, the feature extraction model is obtained after updating a preset model through a first loss value, the first loss value is determined according to the deviation between the features of the training sample and the weight vector of the preset model, and the features of the training sample are obtained after the preset model extracts features from the training sample. Obtain the distance between the features of the target object and the weight vector of the feature extraction model. If the distance exceeds a preset threshold, the target object is determined to be an abnormal object. In this embodiment, the preset model can be updated through the first loss value, and then the first loss value is updated based on the updated preset model, so as to update the preset model again. As the number of updates increases, the weight vector of the preset model will become closer and closer to the features of the training sample. The trained preset model, that is, the weight vector of the target model can already reflect the data distribution characteristics of the training sample. In this embodiment, the classification model only needs to be trained through normal samples to enable the classification model to distinguish between normal samples and abnormal samples, which reduces the training difficulty and training cost of the classification model, and can also improve the performance of the classification model.

[0018] In a possible implementation, feature extraction is performed on a target object through a feature extraction model to obtain features of the target object, including: feature extraction is performed on the target object through a first feature extraction layer to obtain features of the target object, the first feature extraction layer is any one of multiple feature extraction layers in the feature extraction model. Obtaining the distance between the features of the target object and the weight vector of the feature extraction model includes: obtaining the distance between the features of the target object and the weight vector of the second feature extraction layer, the second feature extraction layer is any one of multiple feature extraction layers. In this implementation, the distance between the features of the target object extracted through any feature extraction layer and the weight vector of any feature extraction layer can be expressed as the distance between the features of the target object and the weight vector of the target model.

[0019] In a possible implementation, the first feature extraction layer and the second feature extraction layer are the same feature extraction layer or different feature extraction layers.

[0020] In a possible implementation, the first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions. In this implementation, a preferred first feature extraction layer and a preferred second feature extraction layer are provided, which increases the diversity of the solutions.

[0021] In a possible implementation, obtaining the distance between the feature of the target object and the weight vector of the second feature extraction layer includes: if the weight vector of the second feature extraction layer includes multiple, obtaining the distance between the feature of the target object and each weight vector of the second feature extraction layer. If the distance exceeds a preset threshold, determining that the target object is an abnormal object includes: if the maximum distance among multiple distances exceeds a preset threshold, determining that the target object is an abnormal sample. In this implementation, in order to better identify abnormal samples, in this implementation, whether the target object is an abnormal sample is determined based on the maximum distance among multiple distances.

[0022] In a possible implementation, the feature extraction model is specifically obtained by updating the preset model with the first loss value and the second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the first prediction result and the preset function, the first prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the first prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function. In this implementation, a preset function is introduced in the training process of the model, the input and output of the preset function are negatively correlated, and different outputs are selected according to different inputs of the preset function. There is no need to manually adjust parameters, and different outputs can be automatically obtained according to different inputs, thereby avoiding the model from falling into a local optimum, slowing down the speed of model convergence, and making the performance of the trained model better.

[0023] In a possible implementation, the multiple objects are obtained after perturbing the same first image (the multiple objects can be obtained by performing different perturbations on the same image), and the method further includes: if the distance exceeds a preset threshold, deleting the target object from the multiple objects to obtain updated multiple objects, and the updated multiple objects are used to obtain the saliency map of the first image. In this implementation, the multiple objects are screened, the multiple objects after perturbation that meet the target data distribution are retained, and the saliency map of the first image is obtained using the updated multiple objects, thereby improving the accuracy of the saliency map.

[0024] In a possible implementation, multiple objects are obtained after perturbation processing is performed on the same first image (multiple objects can be obtained by performing different perturbations on the same image), and the method further includes: if the distance does not exceed a preset threshold, the target object is determined to be a normal object. If the target object is an abnormal object, the weight of the target object is set to the first weight. If the target object is a normal object, the weight of the target object is set to the second weight, and the second weight is greater than the first weight. According to the first weight or the second weight, the features of the target object are processed to obtain the processed target object, and the processed target object is used to obtain the saliency map of the first image. In this implementation, a larger weight is set for normal samples and a smaller weight is set for abnormal samples. In the process of generating the saliency map, normal samples are mainly used and abnormal samples are supplemented, so as to weaken the influence of abnormal samples in the process of generating the saliency map. The generated saliency map can better reflect the influence of the features on the output of the model.

[0025] In a third aspect, the present application provides a device for generating a saliency map, comprising: an acquisition module for acquiring multiple objects, the multiple objects being acquired after perturbation processing of a first object. A screening module for screening the multiple objects acquired by the acquisition module according to a first condition to acquire multiple updated objects, the updated multiple objects satisfying a target data distribution, the target data distribution being acquired based on training samples, the training samples being used to train a preset model to obtain a target model. A prediction module for inputting the updated multiple objects acquired by the screening module into the target model to output a first prediction result. A generation module for generating a saliency map of the first object based on the first prediction result acquired by the prediction module and the updated multiple objects.

[0026] In a possible implementation, the target model is obtained by updating a preset model by using a first loss value, the first loss value is determined based on a deviation between features of the training sample and a weight vector of the preset model, the features of the training sample are obtained after the preset model performs feature extraction on the training sample, and the screening module is specifically used to: if a target object is included in multiple objects, delete the target object from the multiple objects to obtain multiple updated objects, the distance between the features of the target object and the weight vector of the target model exceeds a preset threshold, and the features of the target object are obtained after the target model performs feature extraction on the target object.

[0027] In a possible implementation, the features of the target object are specifically extracted through a first feature extraction layer, which is any one of multiple feature extraction layers in the target model. The distance between the features of the target object and the weight vector of the target model is specifically the distance between the features of the target object and the weight vector of the second feature extraction layer, which is any one of multiple feature extraction layers.

[0028] In a possible implementation, the first feature extraction layer and the second feature extraction layer can be set to be the same feature extraction layer according to the needs of actual applications, or the first feature extraction layer and the second feature extraction layer can be set to be different feature extraction layers according to the needs of actual applications.

[0029] In a possible implementation, the first feature extraction layer is specifically the penultimate feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is specifically the last feature extraction layer among the multiple feature extractions.

[0030] In one possible implementation, if the weight vector of the second feature extraction layer includes multiple, the distance between the feature of the target object and the target weight vector of the second feature extraction layer exceeds a preset threshold, the distance between the feature of the target object and the target weight vector of the second feature extraction layer is the maximum distance among multiple distances, and the multiple distances include the distance between the feature of the target object and each weight vector of the second feature extraction layer.

[0031] In a possible implementation, the target model is specifically obtained by updating a preset model by using a first loss value and a second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the second prediction result and a preset function, the second prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the second prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

[0032] In a possible implementation, a weight module is further included, the weight module is used to: set the weights of the updated multiple objects to the first weight. Set the weights of the remaining multiple objects to the second weight, the remaining multiple objects are objects in the multiple objects other than the updated multiple objects, and the first weight is greater than the second weight. The prediction module is specifically used to input the first result and the second result into the target model to output a first prediction result, the first result is determined based on the first weight and the updated multiple objects, and the second result is determined based on the second weight and the remaining multiple objects.

[0033] For the specific implementation steps of the third aspect of the present application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the first aspect, and will not be repeated here one by one.

[0034] In a fourth aspect, the present application provides a device for detecting abnormal objects, comprising: a first acquisition module, for acquiring multiple objects. A feature extraction module, for performing feature extraction on a target object through a feature extraction model to obtain features of the target object, wherein the target object is any one of multiple objects, and the feature extraction model is obtained after updating a preset model through a first loss value, and the first loss value is determined based on the deviation between the features of the training sample and the weight vector of the preset model, and the features of the training sample are obtained after the preset model performs feature extraction on the training sample. A second acquisition module, for acquiring the distance between the features of the target object and the weight vector of the feature extraction model. An abnormality detection module, for determining that the target object is an abnormal object if the distance exceeds a preset threshold.

[0035] In a possible implementation, the feature extraction module is specifically used to: extract features of the target object through a first feature extraction layer to obtain features of the target object, where the first feature extraction layer is any one of multiple feature extraction layers in the feature extraction model. Obtaining the distance between the features of the target object and the weight vector of the feature extraction model includes: obtaining the distance between the features of the target object and the weight vector of the second feature extraction layer, where the second feature extraction layer is any one of the multiple feature extraction layers.

[0036] In a possible implementation, the first feature extraction layer and the second feature extraction layer can be set to be the same feature extraction layer according to the needs of actual applications, or the first feature extraction layer and the second feature extraction layer can be set to be different feature extraction layers according to the needs of actual applications.

[0037] In a possible implementation, the first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in a feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions.

[0038] In a possible implementation, the second acquisition module is specifically configured to: if the weight vector of the second feature extraction layer includes multiple weight vectors, then acquire the distance between the feature of the target object and each weight vector of the second feature extraction layer. The anomaly detection module is specifically configured to determine that the target object is an abnormal sample if the maximum distance among the multiple distances exceeds a preset threshold.

[0039] In a possible implementation, the feature extraction model is specifically obtained by updating a preset model by using a first loss value and a second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the first prediction result and a preset function, the first prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the first prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

[0040] In a possible implementation, multiple objects are obtained after perturbation processing is performed on the same first image (multiple objects can be obtained by performing different perturbations on the same image), and the anomaly detection module is specifically used to: if the distance exceeds a preset threshold, delete the target object from the multiple perturbed first images to obtain updated multiple perturbed first images, and the updated multiple perturbed first images are used to obtain a saliency map of the first image.

[0041] In a possible implementation, multiple objects are obtained after perturbation processing is performed on the same first image (multiple objects can be obtained by performing different perturbations on the same image), and the device also includes a weight module, an abnormality detection module, and is also used to: if the distance does not exceed a preset threshold, determine that the target object is a normal object. The weight module is used to: if the target object is an abnormal object, set the weight of the target object to a first weight. If the target object is a normal object, set the weight of the target object to a second weight, and the second weight is greater than the first weight. According to the first weight or the second weight, the characteristics of the target object are processed to obtain a processed target object, and the processed target object is used to obtain a saliency map of the first image.

[0042] For the specific implementation steps of the fourth aspect of this application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the second aspect, and will not be repeated here one by one.

[0043] In a fifth aspect, the present application provides an apparatus for generating a saliency map, comprising a processor, wherein the processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the method in the first aspect or any possible implementation of the first aspect is implemented.

[0044] For the specific implementation steps of the fifth aspect of this application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the first aspect, and will not be repeated here one by one.

[0045] In a sixth aspect, the present application provides a device for detecting abnormal objects, comprising a processor, wherein the processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the method in the second aspect or any possible implementation of the second aspect is implemented.

[0046] For the specific implementation steps of the sixth aspect of the present application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the second aspect, and will not be repeated here one by one.

[0047] In the seventh aspect, the present application provides a computer-readable storage medium, including a program, which, when executed on a computer, enables the computer to execute a method in the first aspect or any possible implementation of the first aspect, or enables the computer to execute a method in the second aspect or any possible implementation of the second aspect.

[0048] In an eighth aspect, the present application provides a circuit system, comprising a processing circuit, wherein the processing circuit is configured to execute a method as in the first aspect or any possible implementation of the first aspect, or is configured to execute a method as in the second aspect or any possible implementation of the second aspect.

[0049] In a ninth aspect, the present application provides a computer program product, which includes instructions. When the instructions are loaded and executed by an electronic device, the electronic device executes the method in the first aspect or any possible implementation of the first aspect, or the electronic device executes the method in the second aspect or any possible implementation of the second aspect.

[0050] In a tenth aspect, the present application provides a chip, which is coupled to a memory and is used to execute a program stored in the memory to execute a method such as the first aspect or any possible implementation of the first aspect, or to execute a method such as the second aspect or any possible implementation of the second aspect.

[0051] For the specific implementation steps of the seventh to tenth aspects of the present application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the first or second aspect, and will not be repeated here one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A structural diagram of the main framework of artificial intelligence;

[0053] Figure 2 A flowchart of a perturbation-based interpretable method;

[0054] Figure 3A schematic diagram of a flow chart of a method for generating a saliency map provided in an embodiment of the present application;

[0055] Figure 4 Another schematic diagram of a flow chart of a method for generating a saliency map provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of a flow chart of a model training method provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of an application scenario of the model training method provided in an embodiment of the present application;

[0058] Figure 7 Another schematic diagram of a flow chart of a method for model training provided in an embodiment of the present application;

[0059] Figure 8 A schematic diagram of a process for generating a saliency map provided in an embodiment of the present application;

[0060] Fig. 9 A schematic diagram of a process for generating a saliency map provided in an embodiment of the present application;

[0061] Fig.10 A schematic diagram of an application scenario of the solution provided in an embodiment of the present application;

[0062] Fig.11 A schematic diagram of an application scenario of the solution provided in an embodiment of the present application;

[0063] Fig.12 The experimental effect diagram of the solution provided in the embodiment of the present application;

[0064] Fig.13 A schematic diagram of the architecture of a system provided in an embodiment of the present application;

[0065] Fig.14 A schematic diagram of the architecture of an execution device provided in an embodiment of the present application;

[0066] Fig.15 A schematic diagram of a structure of a device for generating a saliency map provided in an embodiment of the present application;

[0067] Fig.16 A schematic diagram of the structure of an abnormal object detection device provided in an embodiment of the present application;

[0068] Fig.17 A schematic diagram of the structure of a training device provided in an embodiment of the present application;

[0069] Fig.18 A schematic diagram of a structure of an execution device provided in an embodiment of the present application;

[0070] Fig.19 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The embodiment of the present application provides a method for generating a saliency map, wherein a first object after disturbance is screened, and after the screening process, the updated first object after disturbance satisfies the target data distribution, and the target data distribution is obtained according to the data distribution of the training sample. The saliency map is generated by using the first image after disturbance that satisfies the target data distribution, so that the saliency map can accurately reflect the influence of the feature on the output of the model, and the explanation of the model is more convincing, which is conducive to improving the user's trust in the model.

[0072] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0073] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1 The figure shows a structural diagram of the main framework of artificial intelligence. The following is an explanation of the main framework of artificial intelligence from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of artificial intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.

[0074] (1) Infrastructure

[0075] The infrastructure provides computing power support for the artificial intelligence system, enables communication with the outside world, and is supported by the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips, as an example, the smart chip includes hardware acceleration chips such as central processing unit (CPU), neural-network processing unit (NPU), graphics processing unit (GPU), application specific integrated circuit (ASIC), field programmable gate array (FPGA); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0076] (2) Data

[0077] The data on the upper layer of the infrastructure indicates the data source in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0078] (3) Data processing

[0079] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0080] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0081] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0082] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0083] (4) General ability

[0084] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, image classification, personalized image management, personalized battery charging management, text analysis, computer vision processing, speech recognition, etc.

[0085] (5) Smart products and industry applications

[0086] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart manufacturing, smart transportation, smart homes, smart medical care, smart security, autonomous driving, smart cities, etc.

[0087] The embodiments of the present application can be applied to the above-mentioned various fields in the scene where a saliency map needs to be obtained. A saliency map is an image that shows the uniqueness of each pixel, and the goal is to simplify the representation of a general image or change it into a style that is easier to analyze. For example, if a pixel has a higher contribution value in the label of an image, it will be displayed in a more obvious way in the saliency map.

[0088] In order to better understand this application scenario, the model's interpretable algorithm is introduced below. There are two main types of interpretable algorithms at present: one is the gradient-based interpretable method, and the other is the perturbation-based interpretable method. Among them, the perturbation-based interpretable method is more versatile because the interpretation module is decoupled from the model and is more widely used.

[0089] like Figure 2 As shown in Figure 1, it is a flowchart of a perturbation-based interpretable method. Figure 2As shown, taking the first object pair image as an example, the perturbation-based interpretable method may include the following steps: first, the image to be processed (image) is perturbed, and a plurality of pixels are randomly sampled from the image to be processed for masking (masking) to obtain the first image after perturbation. Among them, there are many ways to perturb the image to be processed, such as blurring the first image, making the first image transparent, or adding noise to the first image, etc. Then, the first image after perturbation is input into the model, so that the model predicts the first image to obtain the prediction result. Among them, the model can be a model for performing any task, such as the model can be a model for performing image classification tasks, or a model for performing image segmentation tasks, etc., which is not limited in the embodiment of the present application. Subsequently, the features of the first image after perturbation are linearly weighted to obtain a saliency map of the first image, wherein the corresponding weights are determined according to the prediction results. Take the model used to perform classification tasks as an example. Assume that the image features corresponding to the disturbed area include M1 and M2, where the perturbed image corresponding to M1 is I1, and the perturbed image corresponding to M2 is I2. Assume that the probability that the classification model predicts correctly for I1 is P1, and the probability that the classification model predicts correctly for I2 is P2, then the prediction result and the perturbed image features are linearly weighted, that is, I1*P1+I2*P2, and the result is used to represent the saliency map of the first image.

[0090] Through the saliency map, it is easy to observe which features have the greatest impact on the output of the model. To make it easier to understand, the following is a specific example to illustrate how to explain the model through the saliency map. Assuming that the first image is an image including a cat, the first image is perturbed, such as by perturbing the cat's eye area (M1), obtaining the perturbed image I1; by perturbing the cat's mouth area (M2), obtaining the perturbed image I2; by perturbing the cat's tail area (M3), obtaining the perturbed image I3. Assume that the classification model predicts that the probability of I1 being a cat is 0.3 for I1, the probability of I2 being a cat is 0.6 for I2, and the probability of I3 being a cat is 0.1 for I3. It can be seen that when the cat's tail area M3 is perturbed, the model predicts that the probability of the first object being a cat is the lowest, indicating that the features of the cat's tail area M3 have the greatest impact on the model's output. The result represented by 0.3*I1+0.6*I2+0.1*I3 is used to obtain the saliency map.

[0091] However, the applicant has found that the current perturbation-based interpretable methods have a big flaw, that is, after the first object is perturbed, the multiple perturbed first images obtained may be inconsistent with the data distribution of the training samples used in the model training, which will affect the accuracy and reliability of the model interpretation. It should be noted that researchers in the field of explainable artificial intelligence (XAI) are not easily aware of this problem (this problem belongs to the knowledge of probability domain), resulting in all current perturbation-based interpretable methods having this flaw. The applicant has discovered this problem and proposed a specific improvement method. In order to better demonstrate this defect, it is explained below in conjunction with formula 1-1.

[0092]

[0093] Wherein, y represents the prediction result. Taking the classification model as an example, y represents a certain category. d represents the training sample that satisfies the target data distribution (the data distribution satisfied by the training sample is called the target data distribution). x represents the first image after disturbance (this application is also referred to as the disturbed sample or the image after disturbance, they have the same meaning). p(y|d, x) represents the probability that x satisfies the target data distribution and x belongs to y; the calculation process of p(y|d, x) can be represented by formula 1-1. Wherein p(y, d|x) represents the probability that x satisfies the target data distribution and the data satisfying the target data distribution is predicted to be y, and p(d|x) represents the probability that x belongs to the target data distribution. When x belongs to an abnormal sample, that is, when x does not satisfy the target data distribution, the probability that x belongs to y should be very small. This is because the classification model is trained by training samples, and the trained model can only make predictions on the input based on the knowledge that has been learned, that is, the input of the trained model should be consistent with the data distribution of the training sample. Therefore, when the input x does not meet the target data distribution (hereinafter, x that does not meet the target data distribution is called an abnormal sample x), the abnormal sample x may not belong to any of the preset categories (including y), that is, the probability that the abnormal sample x belongs to y is very small. However, the actual situation is that for the abnormal sample x, the model predicts it as y with a high probability. Combined with the above formula 1-1, when x is an abnormal sample, the value of p(y, d|x) will be very small, assuming it is 0.09, and the value of p(d|x) will also be very small, assuming it is 0.1. The ratio of the two is 0.9, that is, the value of p(y|d, x) is 0.9, indicating that the model predicts that the probability that the abnormal sample belongs to y is very high. The above analysis shows that when the input x does not meet the target data distribution, the model has a very high probability of misclassification. Since the determination of the saliency map is closely related to the classification result, the error in the classification result will cause the saliency map to fail to accurately reflect the impact of the feature on the output of the model, which will seriously affect the user's trust in the model.

[0094] In response to the above problems, the present application provides a method for generating a saliency map, in which the first image after disturbance is screened, and the data distribution of the screened first image after disturbance is consistent with the data distribution of the training sample. The saliency map is generated by using the screened first image after disturbance, so that the saliency map can accurately reflect the impact of the features on the output of the model, making the explanation of the model more convincing, and helping to improve the user's trust in the model. The solution provided by the present application is introduced in conjunction with specific embodiments below.

[0095] See also Figure 3 , Figure 3 A schematic flow chart of a method for generating a saliency map provided in an embodiment of the present application, the method may include:

[0096] 301. Get multiple objects.

[0097] The multiple objects are obtained after performing disturbance processing on the first object.

[0098] The first object may be disturbed in a variety of ways. For example, the first object may be subjected to noise addition, or may be subjected to blurring (such as changing the values ​​of multiple pixels within a preset range), or may be subjected to occlusion (such as randomly sampling multiple pixels for occlusion), etc. The embodiment of the present application does not limit the manner of disturbance. The purpose of disturbance is to make a difference between a partial area of ​​the first object after disturbance and a partial area of ​​the first object before disturbance.

[0099] Multiple objects can be image data, voice data, or text data, etc., and the embodiments of the present application do not limit this. When multiple objects are image data, the solution provided by the present application can improve the interpretability of the model whose input is image data, such as improving the interpretability of the image classification model, and better understanding which part of the features of the image input to the image classification model has a great impact on the output of the image classification model. When multiple objects are voice data, the solution provided by the present application can improve the interpretability of the model whose input is voice data, such as improving the interpretability of the multi-round dialogue model, and better explaining which part of the features of the input voice the multi-round dialogue model is based on to interact with the user. When multiple objects are text data, the solution provided by the present application can improve the interpretability of the model whose input is text data, such as improving the interpretability of machine translation, and better understanding which part of the features of the input text has a great impact on the output of the translation model.

[0100] In a preferred embodiment, the multiple objects are obtained by perturbing medical images (such as CT images, B-ultrasound images, MRI images, etc.). In another preferred embodiment, the multiple objects are obtained by perturbing traffic images (such as images obtained by a vehicle-mounted camera).

[0101] 302. Screen multiple objects according to the first condition to obtain multiple updated objects, where data distribution of the multiple updated objects meets target data distribution.

[0102] The target data distribution is obtained based on the data distribution of the training samples, and the training samples are used to train the preset model to obtain the target model.

[0103] The solution provided by the present application finds that the multiple objects obtained after the first object is disturbed may not meet the target data distribution, which will affect the accuracy and reliability of the model interpretation. Assume that the training samples include image 1, image 2, image 3 and image 4, and obtain the target data distribution according to these training samples. The input of the trained model should also meet the target data distribution, otherwise the accuracy of the prediction results made by the trained model for the input will be greatly reduced. Here is an example to explain, assuming that the training samples are all animal images, and these animal images meet the target data distribution, and the input of the trained model should also meet the target data distribution, such as the input is also an animal image (the model has learned the characteristics of the input), then the trained model can make accurate predictions for the input. However, if the input is a person image or an animal image that the model has not learned, the accuracy of the prediction results of the trained model for the input will be greatly reduced. The solution provided by the present application finds that the first object after disturbance may not meet the target data distribution, so the first object after disturbance is screened and processed, and the first object after disturbance that meets the target data distribution is retained. The perturbed first objects that satisfy the target data distribution are used as the input of the model to improve the accuracy of model prediction.

[0104] The solution provided in this application can screen multiple objects in a variety of ways, and several specific screening methods are given below.

[0105] In a possible implementation, the multiple objects can be used as inputs of a classification model, which is obtained by training an initial classification model with training samples, and the trained classification model can be used to identify abnormal samples. The output of the classification model is used as input of a first preset function. If the value of the output of the first preset function is greater than a first preset threshold, the object is considered to be a normal sample. If the value of the output of the first preset function is not greater than the first preset threshold, the object is considered to be an abnormal sample. The output of the first preset function is positively correlated with the input of the first preset function. The set of normal samples is the updated multiple objects.

[0106] In a possible implementation, the multiple objects can be used as input of a reconstruction model (auto encoder), and the multiple objects are reconstructed by the reconstruction model so that the data distribution of the reconstructed multiple objects meets the target data distribution. The deviation between each object before and after reconstruction is obtained, and the objects whose deviations do not exceed the preset threshold are screened out, and the objects whose deviations do not exceed the preset threshold constitute the updated multiple objects. It should be noted that in the implementation of the present application, multiple preset thresholds can be set. For example, the preset threshold in this implementation may be different from the preset threshold in the implementation described in the previous paragraph, and the explanation will not be repeated below.

[0107] In a preferred embodiment, a target object among multiple objects is deleted to obtain multiple updated objects. The distance between the feature of the target object and the weight vector of the target model exceeds a preset threshold, and the feature of the target object is obtained after the target model performs feature extraction on the target object. The target model is obtained by updating the preset model with a first loss value, and the first loss value is determined based on the deviation between the feature of the training sample and the weight vector of the preset model. The feature of the training sample is obtained after the preset model performs feature extraction on the training sample. In this embodiment, during the iterative training process, the target model updates the preset model with the first loss value so that the weight vector of the preset model can be closer to the feature of the training sample. The trained preset model, that is, the weight vector of the target model, can reflect the distribution of the training samples. Therefore, by comparing the distance between the object input to the target model and the weight vector of the target model, it can be determined whether the object input to the target model meets the target data distribution. This embodiment of deleting the target object will be described below. Figure 5 The corresponding embodiments are described in detail.

[0108] This application refers to the other objects in the multiple objects except the updated multiple objects as abnormal objects or abnormal samples. For example, in the several implementations mentioned in the above step 302, the object corresponding to the maximum probability not exceeding the preset threshold is called an abnormal sample; the object corresponding to the deviation exceeding the preset threshold is called an abnormal sample; and the target object is called an abnormal sample. This application also refers to each object in the updated multiple objects as a normal sample (or normal object).

[0109] 303. Generate a saliency map of the first object according to the first prediction result.

[0110] In a possible implementation, the updated multiple objects may be input into the target model to output a first prediction result, and a saliency map of the first object may be generated according to the first prediction result. For example, assuming that the first object is a first image 1, after executing step 301, multiple disturbed first images may be obtained for the first image 1, such as multiple disturbed first images including disturbed image 1, disturbed image 2, disturbed image 3, disturbed image 4, and disturbed image 5. After executing step 302, multiple updated objects are obtained, such as multiple updated objects including disturbed image 1, disturbed image 2, and disturbed image 3. That is, through step 302, disturbed image 1, disturbed image 2, and disturbed image 3 are obtained as normal samples, and disturbed image 4 and disturbed image 5 are abnormal samples. In this implementation, disturbed image 1, disturbed image 2, and disturbed image 3 may be used as inputs of the target model to output a first prediction result, and disturbed image 4 and disturbed image 5 are removed from multiple objects and are no longer used as inputs of the target model. The perturbed image 1, the perturbed image 2 and the perturbed image 3 are weighted to obtain a saliency map of the first image, wherein the weights of the perturbed image 1, the perturbed image 2 and the perturbed image 3 are determined according to the first prediction result, specifically, the weight of the perturbed image 1 is determined according to the prediction result of the target model for the perturbed image 1, specifically, the weight of the perturbed image 2 is determined according to the prediction result of the target model for the perturbed image 2, and specifically, the weight of the perturbed image 3 is determined according to the prediction result of the target model for the perturbed image 3. In this embodiment, only normal samples are used as inputs of the target model, and the saliency map is obtained only according to the normal samples, so as to improve the performance of the saliency map and enable the generated saliency map to accurately reflect the influence of the features on the output of the model.

[0111] In one possible implementation, refer to Figure 4, it may also include step 3031, setting the weight of the normal object to the first weight, setting the weight of the abnormal object to the second weight, and the first weight is greater than the second weight. In this embodiment, after the normal objects and the abnormal objects are screened out, different weights are set for the normal objects and the abnormal objects, and the weight of the normal objects is greater than the weight of the abnormal objects. It should be noted that the weights of all normal objects can be the same or different, and the weights of all abnormal objects can be the same or different. At the same time, the weight of any normal object is greater than the weight of any abnormal object. In this embodiment, the normal object is processed by the first weight to obtain the processed normal object, and the abnormal object is processed by the second weight to obtain the processed abnormal object. The processed normal object and the processed abnormal object are both used as inputs of the target model to obtain a first prediction result, and the processed normal object and the processed abnormal object are weighted according to the first prediction result to obtain a saliency map of the first object. For example, assuming that the first object is the first image 1, after executing step 301, multiple disturbed first images can be obtained for the first image 1, such as the multiple disturbed first images include disturbed image 1, disturbed image 2, disturbed image 3, disturbed image 4 and disturbed image 5. After executing step 302, multiple updated objects are obtained, such as the multiple updated objects include disturbed image 1, disturbed image 2 and disturbed image 3. That is, through step 302, disturbed image 1, disturbed image 2 and disturbed image 3 are obtained as normal samples, and disturbed image 4 and disturbed image 5 are abnormal samples. In this embodiment, a weight is set for each disturbed image. Assuming that the weight of a normal sample is set to 0.9 and the weight of an abnormal sample is set to 0.1, that is, the weights corresponding to disturbed image 1, disturbed image 2 and disturbed image 3 are 0.9, and the weights corresponding to disturbed image 4 and disturbed image 5 are 0.1, then the disturbed images are processed by the weights, and the processed disturbed image 1 can be regarded as 0.9* disturbed image 1, the processed disturbed image 2 can be regarded as 0.9* disturbed image 2, and the processed disturbed image 3 can be regarded as 0.9* disturbed image 3. The processed disturbed image 4 can be regarded as 0.1* disturbed image 4, and the processed disturbed image 5 can be regarded as 0.1* disturbed image 5. All the processed disturbed images are used as inputs of the target model, that is, 0.9* disturbed image 1, 0.9* disturbed image 2, 0.9* disturbed image 3, 0.1* disturbed image 4 and 0.1* disturbed image 5 are all used as inputs of the target model to output a first prediction result.The processed disturbed image 1, the processed disturbed image 2 and the processed disturbed image 3 are weighted to obtain a saliency map of the first image, wherein the weights of the processed disturbed image 1, the processed disturbed image 2 and the processed disturbed image 3 are determined according to the first prediction result, specifically, the weight of the processed disturbed image 1 is determined according to the prediction result of the target model for the processed disturbed image 1, specifically, the weight of the processed disturbed image 2 is determined according to the prediction result of the target model for the processed disturbed image 2, and specifically, the weight of the processed disturbed image 3 is determined according to the prediction result of the target model for the processed disturbed image 3. In this embodiment, a larger weight is set for normal samples and a smaller weight is set for abnormal samples. In the process of generating the saliency map, normal samples are mainly used and abnormal samples are supplemented, so as to weaken the influence of abnormal samples in the process of generating the saliency map. The generated saliency map can better reflect the influence of the features on the output of the model.

[0112] Through the above Figure 3 From the introduction of the corresponding embodiment, it can be seen that the first perturbed image is screened, and the data distribution of the screened first perturbed image is consistent with the data distribution of the training sample. The saliency map is generated using the screened first perturbed image, so that the saliency map can accurately reflect the impact of the feature on the output of the model, making the explanation of the model more convincing, which is conducive to improving the user's trust in the model.

[0113] In the introduction Figure 3 In the corresponding embodiment, a method for screening abnormal samples is mentioned in step 302. Figure 5 This method is described in detail. It should be noted that Figure 5 The corresponding implementation method can be Figure 3 The corresponding implementation methods are combined. Figure 5 The corresponding implementation method can also be used as an independent implementation method. Figure 5 When the corresponding implementation is taken as an independent implementation, Figure 5 The corresponding implementation method can be applied to application scenarios where abnormal sample detection is required. Figure 5The possible application scenarios of the corresponding implementation methods are introduced: as an example, users often receive financial text messages, some of which come from banks. These text messages are useful to users and may be used to notify users that their account balances have changed; some financial text messages are fraudulent text messages. If users mistakenly believe these text messages, such as clicking on the links carried in these fraudulent text messages, or contacting fraudsters, they may cause monetary losses to users. In this scenario, there is a problem of how to identify fraudulent text messages. The solution provided by this application can effectively screen out financial fraud text messages, and the screened financial fraud text messages are not displayed to users, avoiding losses caused by users mistakenly believing in fraudulent text messages. As another example, when diagnosing medical images (such as CT films, B-ultrasound films, MRI films, etc.) through AI, it is necessary to identify which are normal medical images and which are abnormal medical images from a large number of medical images. Abnormal medical images indicate that the patient corresponding to the medical image may suffer from a certain disease. In this scenario, the solution provided by this application can effectively improve the efficiency of the model in diagnosing diseases. As another example, industrial manufacturing requires fault detection, such as the need to collect mine temperature, and issue an alarm when the mine temperature is abnormal. In this scenario, it is necessary to identify whether the mine temperature is abnormal based on the collected mine temperature. The solution provided by this application can effectively identify whether the mine temperature is abnormal, and issue an alarm in time to avoid danger. As another example, it is hoped to find abnormal transaction operations from a large number of transaction operation samples, so as to prevent fraudulent transactions in advance. As another example, it is hoped to detect abnormal access from network access samples, so as to find unsafe access, such as hacker attacks. As another example, it is hoped to find abnormal accounts from user accounts that perform various operations, so as to lock accounts suspected of high-risk operations (fraudulent transactions, false transactions such as brushing orders, network attacks), etc.

[0114] The particularity of the abnormal sample detection problem poses a great challenge to the classification model. The purpose of abnormal sample detection is to distinguish normal samples from abnormal samples. However, unlike traditional classification models, the frequency of abnormal samples is low, which makes it difficult to collect enough abnormal samples to train the classification model. Through the solution provided by the present application, it is only necessary to train the classification model with normal samples to enable the classification model to distinguish normal samples from abnormal samples, which reduces the training difficulty and training cost of the classification model, and can also improve the performance of the classification model. The following is an introduction to a model training method provided in an embodiment of the present application, and the use of the trained model for abnormal object detection.

[0115] See also Figure 5 , Figure 5 A schematic diagram of a flow chart of a model training method provided in an embodiment of the present application, the method may include:

[0116] 501. Perform feature extraction on the training samples to obtain features of the training samples.

[0117] The present application does not limit the data type of the training sample, which can be image data, voice data, or text data, etc. When the training sample is image data, the trained model can be used to predict the image data. Specifically, the trained model can identify abnormal images from multiple images to be detected, such as the detection of medical images; when the training sample is voice data, the trained model can be used to predict voice data. Specifically, the trained model can identify abnormal voice data from multiple detected voice data; for another example, when the training sample is text data, the trained model can be used to predict text data to identify abnormal text data from multiple text data to be detected, such as the identification of financial fraud text messages.

[0118] The present application extracts features from training samples through a preset model. Specifically, the feature extraction model (or feature extraction module) of the preset model extracts features from training samples. The preset model can predict the training samples based on the features of the training samples extracted by the feature extraction model, and update the preset model based on the deviation between the prediction results and the actual results of the training samples, so that the prediction results of the updated preset model for the training samples can be closer to the actual results. The preset model can perform a variety of predictions based on the features of the training samples extracted by the feature extraction model, such as predicting the category of the training samples, predicting the area where the target object in the training samples is located, and so on.

[0119] The feature extraction model consists of multiple feature extraction layers connected end to end. The output of the previous feature extraction layer is used as the input of the next feature extraction layer. The input of the first feature extraction layer is the training sample. The work of each layer of the feature extraction model can be expressed mathematically as To describe: From a physical level, the work of each layer in a deep neural network can be understood as completing the transformation from input space to output space (i.e., from row space to column space of a matrix) through five operations on the input space (a set of input vectors). These five operations include: 1. Dimension increase / reduction; 2. Enlargement / reduction; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are represented by Completed, operation 4 is completed by +b, and operation 5 is realized by . The reason why the word "space" is used here is because the classified object is not a single thing, but a class of things, and space refers to the collection of all individuals of this class of things. Among them, W is a weight vector, and each value in the vector represents the weight value of a neuron in this layer of the neural network. This vector determines the spatial transformation from the input space to the output space described above, that is, the weight of each layer controls how to transform the space. The purpose of training the preset model is to finally obtain the weight matrix of all layers of the trained preset model (the weight matrix formed by many layers of vectors). Therefore, the training process of the preset model is essentially to learn how to control spatial transformation, and more specifically to learn the weight matrix.

[0120] Because we want the output of the preset model to be as close as possible to the value we really want to predict. The value we really want to predict is related to the training target of the preset model or the task that the preset model needs to complete. For example, if the preset model is used for image classification tasks, the output of the preset model is as close as possible to the actual image classification results. In order to make the output of the preset model as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict lower, and keep adjusting until the neural network can predict the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the preset model becomes a process of minimizing this loss as much as possible.

[0121] 502. Obtain a first loss value according to a deviation between a feature of the training sample and a weight vector of a preset model.

[0122] The training process introduced in step 501 is to minimize the loss as much as possible. The loss of the solution provided in this application includes a first loss value. As the preset model is trained, the first loss value will become smaller and smaller, and the weight vector of the preset model will become closer and closer to the characteristics of the training sample. The weight vector of the preset model after training, that is, the target model, can already reflect the data distribution characteristics of the training sample. The solution provided in this application can obtain the first loss value in a variety of ways, and possible implementation methods are described below.

[0123] In a possible implementation, features extracted by any feature extraction layer of the preset model can be used as features of the training samples in step 502, and weight vectors of any feature extraction layer of the preset model can be used as weight vectors of the preset model in step 502. For example, feature extraction is performed on the training sample through the first feature extraction layer (hereinafter, the feature obtained after feature extraction of the training sample through the first feature extraction layer is referred to as the first feature), wherein the first feature extraction layer can be any feature extraction layer in the multi-layer feature extraction layer. The weight vector of the preset model can be the weight vector of the second feature extraction layer (hereinafter, the weight vector of the second feature extraction layer is referred to as the first weight vector), wherein the second feature extraction layer can be any feature extraction layer in the multi-layer feature extraction layer. The first loss value can be obtained by the deviation between the first feature and the first weight vector.

[0124] In a possible implementation manner, the first feature extraction layer and the second feature extraction layer are the same feature extraction layer.

[0125] In one possible implementation, the first feature extraction layer and the second feature extraction layer are different feature extraction layers. In a preferred implementation, the first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in a feature extraction model, the multiple feature extraction layers are connected end to end, the training sample is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions.

[0126] In one possible implementation, the weight vector of the preset model may be a result obtained by weighting the weight vectors of multiple feature extraction layers. For example, the weight vectors of all feature extraction layers are weighted, and the result after the weighting process is considered to be the weight vector of the preset model. For another example, the weight vector of the first feature extraction layer and the weight vector of the second feature extraction layer may be weighted, and the result may be considered to be the weight vector of the preset model.

[0127] In one possible implementation, each feature extraction layer may include multiple weight vectors, and after performing feature extraction on a training sample through each privilege extraction layer, multiple features of the training sample may be obtained. Multiple first loss values ​​can be obtained based on the deviation between each feature and each weight vector, and the preset model can be updated through the multiple first loss values ​​so that each weight vector can be closer to each feature. For example, assume that the first feature extraction layer includes weight vector 1, weight vector 2, and weight vector 3, wherein feature extraction is performed on the input training sample according to weight vector 1 to obtain feature 1; feature extraction is performed on the input training sample according to weight vector 2 to obtain feature 2; feature extraction is performed on the input training sample according to weight vector 3 to obtain feature 3. Loss value 1 can be obtained based on the deviation between weight vector 1 and feature 1, loss value 2 can be obtained based on the deviation between weight vector 2 and feature 2, and loss value 3 can be obtained based on the deviation between weight vector 3 and feature 3. The first loss value includes loss value 1, loss value 2 and loss value 3. The process of updating the preset model through the first loss value is also the process of continuously reducing loss 1, loss 2 and loss 3, so that weight vector 1 is closer to feature 1, weight vector 2 is closer to feature 2, and weight vector 3 is closer to feature 3. When the target model is a classification model, continue to give an example, assuming that the preset categories include dogs, cats and rabbits. Feature 1 extracted by weight vector 1 pays more attention to the features belonging to dogs in the input training sample, feature 2 extracted by weight vector 2 pays more attention to the features belonging to cats in the input training sample, and feature 3 extracted by weight vector 3 pays more attention to the features belonging to rabbits in the input training sample. In the process of training the preset model, as the loss value 1 continues to decrease, weight vector 1 will be more able to reflect the features of dogs. The weight vector 1 of the target model can be regarded as the category center of the dog category, where the category center can be used to represent the typical features of the dog category. The weight vector 2 of the target model can be regarded as the category center of the cat category, where the category center can be used to represent the typical features of the cat category. The weight vector 3 of the target model can be regarded as the category center of the rabbit category, where the category center can be used to represent the typical characteristics of the rabbit category.

[0128] 503. Update the preset model according to the first loss value.

[0129] The preset model is updated by the first loss value obtained in step 502, and steps 501 to 503 are repeated until a preset stop condition is met. The preset stop condition can be understood as the preset model has converged, or the number of iterative training has reached a preset number.

[0130] The preset model is trained through step 501 to step 503 to obtain the trained preset model, that is, the target model. Normal samples and abnormal samples can be identified through the target model.

[0131] Depend on Figure 5 It can be seen from the corresponding embodiments that the preset model is trained through the solution provided by the present application. During the training process, only normal samples need to be used as training data, and there is no need to use abnormal samples. Compared with the features that are difficult to obtain from abnormal samples, normal samples are easier to obtain, which can greatly reduce the difficulty of training the model and reduce the cost of model training. During the training process of the model, the preset model is updated by the first loss value. As the preset model is trained, the first loss value will become smaller and smaller, and the weight vector of the preset model will become closer and closer to the features of the training samples. The preset model after training, that is, the weight vector of the target model, can already reflect the data distribution characteristics of the training samples. In the inference stage, the object is input into the target model. By comparing the distance between the weight vector of the target model and the features of the input object, it can be determined whether the input object is an abnormal sample.

[0132] In a possible implementation, multiple objects are obtained; and feature extraction is performed on the target object through a target model to obtain features of the target object, where the target object is any one of the multiple objects. The target model here is Figure 5 The target model described in the corresponding embodiment. Obtain the distance between the feature of the target object and the weight vector of the target model. If the distance exceeds a preset threshold, the target object is determined to be an abnormal object.

[0133] In a possible implementation, feature extraction is performed on the target object through a first feature extraction layer to obtain features of the target object, where the first feature extraction layer is any one of multiple feature extraction layers in the target model. The distance between the features of the target object and the weight vector of the second feature extraction layer is obtained, where the second feature extraction layer is any one of the multiple feature extraction layers.

[0134] In a possible implementation, the first feature extraction layer and the second feature extraction layer are the same feature extraction layer or different feature extraction layers.

[0135] In a possible implementation, the first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in the target model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions.

[0136] In a possible implementation, if the weight vector of the second feature extraction layer includes multiple weight vectors, the distance between the feature of the target object and each weight vector of the second feature extraction layer is obtained; if the maximum distance among the multiple distances exceeds a preset threshold, the target object is determined to be an abnormal sample. It has been introduced above that a weight vector can be regarded as the category center of a category. Figure 6 , assuming that there are 10 weight vectors, namely W1 to W10. According to the weight vectors W1 to W10, the target data distribution space can be obtained, so that the target data distribution space includes the weight vectors W1 to W10. Specifically, the size of the target data distribution space range obtained according to the weight vectors W1 to W10 can be determined according to actual conditions, and the embodiments of the present application do not limit this. Figure 6 The box in represents the target data distribution space. In theory, each training sample is included in the target data distribution space. When screening abnormal samples, the distance between the features of the target object and each weight vector of the second feature extraction layer can be obtained, and the maximum distance among multiple distances can be used to determine whether the target object is an abnormal sample. Figure 6 , assuming that the feature of the target object is f(x), exceeds the preset threshold, and obtains the distance between f(x) and each weight vector in W1 to W10, from Figure 6 It can be seen that the distance between f(x) and W1 is the farthest. Based on the distance between f(x) and W1, the target object is determined to be an abnormal sample. The reason why the maximum distance among multiple distances is used to determine whether the target object is an abnormal sample is to achieve a better recognition effect. For example, continue to refer to Figure 6 , if the distance between f(x) and W10 is used to determine whether the target object is an abnormal sample, since the distance between f(x) and W10 is very close (assuming that the distance between f(x) and w10 is the minimum distance between f(x) and each weight vector from W1 to W10), this distance may not exceed the preset threshold, and f(x) is considered not to be an abnormal sample. However, if f(x) is not in the target data distribution space (such as Figure 6 As shown, f(x) is not inside the box), which should be an abnormal sample. Therefore, in order to better identify abnormal samples, in this implementation, it is determined whether the target object is an abnormal sample based on the maximum distance among multiple distances.

[0137] In a preferred embodiment, the training process can be further optimized to make the performance of the preset model after training, that is, the target model, better. This will be described below in conjunction with a specific embodiment.

[0138] See also Figure 7 , Figure 7 Another flowchart of the method for model training provided in the embodiment of the present application may include:

[0139] 701. Perform feature extraction on the training samples to obtain features of the training samples.

[0140] 702. Obtain a first loss value according to a deviation between a feature of the training sample and a weight vector of a preset model.

[0141] Step 701 and step 702 can refer to Figure 5 The corresponding embodiment is generally understood by referring to step 501 and step 502, which will not be repeated here.

[0142] 703. Obtain a second loss value according to a deviation between the target result and the actual result of the training sample.

[0143] The target result is determined based on the first prediction result and the preset function. The first prediction result is the prediction result of the preset model for the training sample. The input of the preset function is the first prediction result. The output of the preset function is the target result. The output of the preset function is negatively correlated with the input of the preset function.

[0144] exist Figure 3 In the corresponding implementation, the training process of the model is introduced. The training process is a process of continuously reducing the loss value, so that the prediction results of the updated preset model for the training samples can be closer to the actual results. Assuming that after a few rounds of training, the prediction results of the prediction model for the training samples are close to the actual results, the training stops. In theory, the more rounds of model training, the more conducive to improving the performance of the trained model. The model continues to learn and improve the accuracy of prediction in each round of training. Therefore, in order to slow down the convergence of the model and enable the model to be trained for more rounds, the prediction results obtained from each training can also be processed to narrow the distance between the prediction results with the highest probability and the actual results. Compared with not processing the prediction results obtained from each training, after processing, the loss value obtained from each training will increase. For example, suppose that in one training, the prediction result of the preset model for training sample 1 is that the probability that training sample 1 belongs to a cat is 0.9, and the probability that training sample 1 belongs to a tiger is 0.1. The prediction results of this training are used as the input of the preset function to obtain target results 1 and target results 2. Since the output of the preset function is negatively correlated with the input of the preset function, the target result 1 may be that the probability that training sample 1 belongs to a cat is 0.6, and the probability that training sample 1 belongs to a tiger is 0.4. Compared with the prediction results before processing (0.9, 0.1), the probabilities in the processed prediction results (0.6, 0.4) are more balanced, which is conducive to slowing down the convergence of the model. The second loss value includes the loss value obtained based on the deviation between the target result 1 and the true category of training sample 1, and the loss value obtained based on the deviation between the target result 2 and the true category of training sample 1.

[0145] 704. Update the preset model according to the first loss value and the second loss value.

[0146] The preset model is updated by the first loss value and the second loss value obtained in step 702 and step 703, and steps 701 to 704 are repeatedly performed until a preset stop condition is met, wherein the preset stop condition can be understood as the preset model has converged.

[0147] Depend on Figure 7 The corresponding embodiment shows that Figure 5 On the basis of the corresponding implementation, a preset function is introduced in the training process of the model. The input and output of the preset function are negatively correlated. Different outputs are selected according to different inputs of the preset function. There is no need to manually adjust parameters. Different outputs can be automatically obtained according to different inputs (also called obtaining different temperatures), so as to avoid the model falling into local optimality, slow down the speed of model convergence, and make the performance of the trained model better.

[0148] In a preferred embodiment, Figure 5 The target model pair obtained by the method provided in the corresponding embodiment Figure 3 In the corresponding embodiment, multiple disturbed samples are screened to identify abnormal samples among the multiple disturbed samples. Figure 7 The target model pair obtained by the method provided in the corresponding embodiment Figure 3 The multiple disturbed samples in the corresponding embodiment are screened to identify abnormal samples in the multiple disturbed samples. The abnormal samples are deleted from the multiple disturbed samples, and the saliency map of the first object is obtained by deleting the multiple disturbed samples from which the abnormal samples are deleted. In order to better demonstrate this scheme, two typical flowcharts are used to illustrate this. It should be understood that the following two typical flowcharts only provide two possible ways of combining the embodiments. In addition, there may be other ways of combining. For example, the following Figure 8 In the corresponding embodiment, the first object is described as a first image, but the first object can also be other types of data, for details, refer to Figure 3 The corresponding embodiments can be understood and will not be repeated here.

[0149] See also Figure 8 , Figure 8 A schematic diagram of a process for generating a saliency map provided in an embodiment of the present application. A first image is acquired, and a disturbance process is performed on the first image to acquire a disturbed first image. Figure 8In the process shown, multiple groups of pixels are randomly selected from the first image for occlusion, and a perturbed first image can be obtained based on the occluded pixel points selected in each group. The abnormal samples in the obtained perturbed first image are screened by the abnormal sample screening module, and the abnormal samples in the perturbed first image are screened out and deleted, and the perturbed first image with the abnormal samples deleted is used as the input of the classification model. The perturbed first image with the abnormal samples deleted is weighted according to the output of the classification model to obtain a weighted result, and the obtained weighted result is input into the XAI interpreter to obtain a saliency map of the first image. Please refer to Fig. 9 , further illustrating the process of screening abnormal samples. The first image after disturbance is input into the target model, and the target model is used to extract features of each first image after disturbance, and the features extracted by the penultimate feature extraction layer of multiple feature extraction layers in the target model and the distance between the last feature extraction layers are obtained. If multiple distances are included, it is determined whether the maximum distance among the multiple distances exceeds the preset threshold. If it exceeds the preset threshold, the first image after disturbance is an abnormal sample and is deleted. If it does not exceed the preset threshold, the first image after disturbance is a normal sample. The normal sample is used as the input of the classification model. The normal sample is weighted according to the output of the classification model to obtain a weighted result, and the obtained weighted result is input into the XAI interpreter to obtain a saliency map of the first image. In addition, in the process of training the preset model to obtain the target model, the first loss value and the second loss value are used, and a preset function is introduced to slow down the convergence speed of the model to improve the performance of the target model.

[0150] The solution provided in the embodiments of the present application can improve the accuracy of model interpretation. The solution provided in the present application is introduced below by taking the application of the solution provided in the present application to several typical scenarios as an example.

[0151] See also Fig.10, the solution provided in this application can be applied to the application scenario of medical image detection. For example, multiple medical images (such as lung CT films) are obtained for the same body area. The training samples of the target model are normal medical images. Normal medical images refer to the source objects (people) of the medical images are healthy and not suffering from diseases. By inputting the multiple medical images into the target model, abnormal samples can be screened out. The abnormal samples here are abnormal medical images. The abnormal medical images refer to the source objects (people) of the medical images that may be suffering from diseases. In another possible implementation, each medical image is perturbed, and the medical images after perturbation are screened to delete the abnormal samples in the medical images after perturbation. The training samples of the target model are abnormal medical images. By inputting the medical images after perturbation into the target model, abnormal samples can be screened out. The abnormal samples here are samples that do not meet the data distribution of the training samples. The normal samples in the medical images after perturbation are input into the target model, and the saliency map of each medical image is obtained according to the prediction results of the target model. The saliency map of the medical image can be better explained.

[0152] See also Fig.11 , the solution provided by the present application can be applied to the application scenarios of intelligent driving. For example, traffic images (such as images obtained by a vehicle-mounted camera) are obtained. Each traffic image is subjected to disturbance processing, and the traffic images after disturbance processing are screened to delete abnormal samples in the traffic images after disturbance processing. Normal samples in the traffic images after disturbance processing are input into the target model, and a saliency map of each traffic image is obtained according to the prediction results of the target model. The saliency map of the traffic image can better show which part of the features of the traffic image has a greater impact on the output of the target model, thereby improving the interpretability of the model.

[0153] In order to more intuitively understand the beneficial effects brought about by this solution, the beneficial effects brought about by the embodiments of the present application are explained below with reference to data.

[0154] In a test experiment, the test data set is the ImageNet data set. An image classification test experiment is performed on the ImageNet data set based on the ResNet50 model, and the classification results are explained, specifically by using a saliency map. Fig.12 For display. Fig.12As shown, by classifying three typical original images, one original image includes multiple targets (multiple fish, multiple flowers), another original image is a long-distance scene (street scene), and another original image is a single target, and the target occupies a relatively large area in the original image. Indicators for measuring the accuracy of interpretable methods include localization. The interpretation based on the saliency map should significantly highlight the pixels related to the label in the image. The localization indicator uses the target detection data set. For the same image and the same label, the positioning ability of the saliency map is measured by the overlap between the highlighted part of the saliency map and the ground truth. The higher the positioning ability, the more accurate the interpretative method. The solution provided in this application screens the disturbed image, and the saliency map obtained through normal samples can significantly improve the localization indicator compared to existing solutions. Specifically, in multi-target scenes, this application can more accurately highlight multiple targets, while existing solutions can only highlight a single target, and other bright parts cannot accurately capture additional targets; in street scenes, this application highlights two traffic lights, while existing solutions only highlight one; in single large target scenes, the highlighted part of this application is closer to the target outline. The solution provided by this application has significantly improved the localization index - 63.1% in multi-target scenes and 58.3% in street scenes.

[0155] In another test experiment, DenseNet-BC and ResNet50 were selected as target models respectively. Refer to Table 1: When DenseNet-BC is selected as the target model, the Cifar10 dataset is used as a normal sample, and the TinyImageNet dataset is used as an abnormal sample for reasoning. Compared with the existing scheme, the area under the receiver operating characteristic curve (AUROC) index of the solution provided by this application is improved by 2.9%, and the true negative rate (true negative rate at 95% TruePositive rate, TNR@TPR95) index corresponding to the true positive rate of 95% is improved by 26.2%. When ResNet50 is selected as the target model, the ImageNet dataset is used as a normal sample, and the Gaussian Noise dataset is used as an abnormal sample for reasoning, the AUROC index is improved by 9.2%, and the TNR@TPR95 index is improved by 98.3%. The higher the AUROC index and the TNR@TPR95 index, the higher the accuracy of detecting abnormal samples.

[0156]

[0157] Table 1

[0158] In order to better understand this solution, first combine Fig.13 For an introduction to a system provided in an embodiment of the present application, please refer to Fig.13 , Fig.13 A system architecture diagram is provided for an embodiment of the present application. Fig.13 In the embodiment, the system 200 includes an execution device 210, a training device 220, a database 230 and a data storage system 240.

[0159] During the training phase, a training data set is stored in the database 230, and the database 230 can be specifically expressed as a storage medium in any form, and is not limited to a database in the traditional sense. There can be multiple training samples in the training data set. The present application does not limit the data type of the training sample, for example, the training sample can be image data, or the training sample can be voice data, or the training sample can be text data. It should be noted that, under normal circumstances, the data types of the training samples included in the training data set are the same. The training device 220 generates a preset model, and iteratively trains the preset model using the training data set in the database to obtain a mature preset model (i.e., the target model). The corresponding embodiments above have already described in detail how to train the preset model to obtain the target model, which will not be repeated here.

[0160] In the inference stage, the execution device 210 can call the data, code, etc. in the data storage system 240, or store the data, instructions, etc. in the data storage system 240. The data storage system 240 can be configured in the execution device 210, or it can be a memory outside the execution device 210. The execution device 210 can call a mature preset model to extract the features of the first object, and perform a classification task according to the extracted features of the first object, obtain a saliency map according to the prediction result (refer to the above related embodiments for obtaining a saliency map for understanding), or detect abnormal samples according to the extracted features of the first object (refer to the above related embodiments for abnormal sample detection for understanding).

[0161] In some embodiments of the present application, for example Figure 3In the example, the "user" can interact directly with the execution device 210, that is, the execution device 210 and the client device are integrated into the same device. As an example, in some application scenarios, the execution device 210 can be a terminal device, such as a mobile phone, a camera, a smart home, etc. In the reasoning stage, the user can input the first object through the execution device 210, such as the user takes a photo through a camera, and the image obtained by the camera is used as the input of a mature preset model. In other application scenarios, the execution device 210 can be specifically expressed as an execution device equipped with a display screen. In the reasoning stage, after completing a task (or multiple tasks), the execution device 210 can display the output result of the preset model to the user. For example, after the execution device 210 performs an image classification task, it displays the result of image classification to the user. The execution device 210 can also be expressed in other forms, which are not listed here one by one, but Figure 3 It is only a schematic diagram of the architecture provided by an embodiment of the present invention, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation.

[0162] In other embodiments of the present application, the execution device 210 and the client device may be separate devices. The execution device 210 is configured with an input / output interface to interact with the client device for data. The "user" may input at least one task to the execution device 210 through the input / output interface of the client device, and the execution device 210 may return the processing result to the client device through the input / output interface.

[0163] The above introduces a method for obtaining a salient map, a model training method, and a method for detecting abnormal objects provided in an embodiment of the present application. The solution provided in the present application can improve the performance of the salient map, reduce the difficulty of training the abnormal detection model, and improve the accuracy of abnormal object detection.

[0164] It is understandable that, in order to achieve the above functions, related devices for implementing the above scheme are also provided below. These related devices include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0165] See also Fig.14 , is a schematic diagram of the architecture of an execution device provided in an embodiment of the present application. Fig.14As shown, the execution device includes hardware structures and software modules corresponding to the execution of each function. Among them, the execution device is used to receive input data, and the interpretable toolkit outputs a saliency map according to the input data. In the process of the interpretable toolkit outputting a saliency map according to the input data, it is necessary to call the AI ​​framework, wherein the target model described in the above embodiment is deployed in the AI ​​framework. The calculation process requires the support of the hardware structure, and the execution device also includes a processor for running the code in the software. Among them, the processor can be a central processing unit (CPU), a network processor (neural-network processing unit, NPU), a graphics processing unit (graphics processing unit, GPU), a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC) or a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general processor can be a microprocessor or any conventional processor, etc.

[0166] Specific reference Fig.15 , Fig.15 A schematic diagram of a structure of a device for generating a saliency map provided in an embodiment of the present application. The device for generating a saliency map may include an acquisition module 1501 , a screening module 1502 , a prediction module 1503 , and a generation module 1504 .

[0167] In a possible implementation, the device for generating a saliency map includes: an acquisition module 1501, used to acquire multiple objects, the multiple objects are acquired after perturbation processing of a first object. A screening module 1502, used to screen the multiple objects acquired by the acquisition module 1501 according to a first condition to acquire multiple updated objects, the updated multiple objects satisfying a target data distribution, the target data distribution is acquired according to training samples, and the training samples are used to train a preset model to obtain a target model. A prediction module 1503, used to input the updated multiple objects acquired by the screening module 1502 into the target model to output a first prediction result. A generation module 1504, used to generate a saliency map of the first object according to the first prediction result acquired by the prediction module 1503 and the updated multiple objects.

[0168] In one possible implementation, the target model is obtained by updating a preset model by using a first loss value, the first loss value is determined based on a deviation between features of the training sample and a weight vector of the preset model, the features of the training sample are obtained after the preset model performs feature extraction on the training sample, and the screening module 1502 is specifically used to: if a target object is included in multiple objects, delete the target object from the multiple objects to obtain multiple updated objects, the distance between the features of the target object and the weight vector of the target model exceeds a preset threshold, and the features of the target object are obtained after the target model performs feature extraction on the target object.

[0169] In a possible implementation, the features of the target object are specifically extracted through a first feature extraction layer, which is any one of multiple feature extraction layers in the target model. The distance between the features of the target object and the weight vector of the target model is specifically the distance between the features of the target object and the weight vector of the second feature extraction layer, which is any one of multiple feature extraction layers.

[0170] In a possible implementation, the first feature extraction layer and the second feature extraction layer are the same feature extraction layer or different feature extraction layers.

[0171] In a possible implementation, the first feature extraction layer is specifically the penultimate feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is specifically the last feature extraction layer among the multiple feature extractions.

[0172] In one possible implementation, if the weight vector of the second feature extraction layer includes multiple, the distance between the feature of the target object and the target weight vector of the second feature extraction layer exceeds a preset threshold, the distance between the feature of the target object and the target weight vector of the second feature extraction layer is the maximum distance among multiple distances, and the multiple distances include the distance between the feature of the target object and each weight vector of the second feature extraction layer.

[0173] In a possible implementation, the target model is specifically obtained by updating a preset model by using a first loss value and a second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the second prediction result and a preset function, the second prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the second prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

[0174] In a possible implementation, the apparatus for generating a saliency map further includes a weight module 1505, the weight module 1505 being used to: set the weights of the updated multiple objects to the first weight. Set the weights of the remaining multiple objects to the second weight, the remaining multiple objects being objects other than the updated multiple objects in the multiple objects, and the first weight is greater than the second weight. The prediction module 1503 is specifically used to input the first result and the second result into the target model to output a first prediction result, the first result being determined based on the first weight and the updated multiple objects, and the second result being determined based on the second weight and the remaining multiple objects.

[0175] It should be noted that Fig.15 The information interaction, execution process, etc. between the modules in the device for generating a saliency map shown in the present application are the same as those in the present application. Figures 3 to 9 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0176] Specific reference Fig.16 , Fig.16 A schematic diagram of a structure of an abnormal object detection device provided in an embodiment of the present application. The abnormal object detection device may include a first acquisition module 1601 , a feature extraction module 1602 , a second acquisition module 1603 , and an abnormality detection module 1604 .

[0177] In one possible implementation, the present application provides a device for detecting abnormal objects, including: a first acquisition module 1601, used to acquire multiple objects. A feature extraction module 1602, used to extract features of a target object through a feature extraction model to acquire features of the target object, where the target object is any one of multiple objects, and the feature extraction model is acquired after updating a preset model through a first loss value, and the first loss value is determined based on the deviation between the features of a training sample and a weight vector of a preset model, and the features of the training sample are acquired after the preset model performs feature extraction on the training sample. A second acquisition module 1603, used to acquire the distance between the features of the target object and the weight vector of the feature extraction model. Anomaly detection module 1604, used to determine that the target object is an abnormal object if the distance exceeds a preset threshold.

[0178] In a possible implementation, the feature extraction module 1602 is specifically used to: extract features of the target object through a first feature extraction layer to obtain features of the target object, where the first feature extraction layer is any one of multiple feature extraction layers in the feature extraction model. Obtaining the distance between the features of the target object and the weight vector of the feature extraction model includes: obtaining the distance between the features of the target object and the weight vector of the second feature extraction layer, where the second feature extraction layer is any one of the multiple feature extraction layers.

[0179] In a possible implementation, the first feature extraction layer and the second feature extraction layer are the same feature extraction layer or different feature extraction layers.

[0180] In a possible implementation, the first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in a feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions.

[0181] In a possible implementation, the second acquisition module 1603 is specifically configured to: if the weight vector of the second feature extraction layer includes multiple weight vectors, then obtain the distance between the feature of the target object and each weight vector of the second feature extraction layer. The abnormality detection module 1604 is specifically configured to determine that the target object is an abnormal sample if the maximum distance among the multiple distances exceeds a preset threshold.

[0182] In a possible implementation, the feature extraction model is specifically obtained by updating a preset model by using a first loss value and a second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the first prediction result and a preset function, the first prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the first prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

[0183] In a possible implementation, the multiple objects are multiple disturbed first images, which are obtained after the first image is disturbed. The abnormality detection module 1604 is specifically used to: if the distance exceeds a preset threshold, delete the target object from the multiple disturbed first images to obtain updated multiple disturbed first images, and the updated multiple disturbed first images are used to obtain a saliency map of the first image.

[0184] In a possible implementation, the multiple objects are multiple disturbed first images, and the multiple disturbed first images are obtained after the first image is disturbed. The device also includes a weight module 1605 and an abnormality detection module 1604, which is also used to: if the distance does not exceed a preset threshold, determine that the target object is a normal object. The weight module 1605 is used to: if the target object is an abnormal object, set the weight of the target object to a first weight. If the target object is a normal object, set the weight of the target object to a second weight, and the second weight is greater than the first weight. According to the first weight or the second weight, the features of the target object are processed to obtain the processed target object, and the processed target object is used to obtain a saliency map of the first image.

[0185] It should be noted that Fig.16 The information interaction, execution process, etc. between the modules in the abnormal object detection device shown in the embodiment are the same as those in the present application. Figures 3 to 9 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0186] The present application also provides a training device. Fig.17 , Fig.17 A schematic diagram of a structure of a training device provided in an embodiment of the present application. The training device 1700 may be deployed with Figure 5 , Figure 6 Specifically, the training device 1700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1722 (for example, one or more processors) and memory 1732, and one or more storage media 1730 (for example, one or more mass storage devices) storing application programs 1742 or data 1744. Among them, the memory 1732 and the storage medium 1730 may be temporary storage or permanent storage. In one embodiment, the memory 1732 is a random access memory (RAM), which can directly exchange data with the central processor 1722, and is used to load data 1744 and application programs 1742 and / or operating system 1741 for direct operation and use by the central processor 1722, usually as a temporary data storage medium for the operating system or other running programs. The program stored in the storage medium 1730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the training device. Furthermore, the central processor 1722 can be configured to communicate with the storage medium 1730 and execute a series of instruction operations in the storage medium 1730 on the training device 1700.

[0187] The training device 1700 may also include one or more power supplies 1726, one or more wired or wireless network interfaces 1750, one or more input and output interfaces 1758, and / or, one or more operating systems 1741, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0188] It should be noted that the CPU 1722 is also used to execute Figure 5 , Figure 6 The other steps of the preset model execution are performed by the central processor 1722 Figure 5 , Figure 6 The specific implementation methods and beneficial effects of the steps executed by the preset model in the corresponding embodiment can be referred to Figure 5 , Figure 6 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0189] The present application also provides an execution device. Fig.18 , Fig.18 A schematic diagram of the structure of an execution device provided in an embodiment of the present application. The execution device 1800 may be deployed with Figures 3 to 9 The target model described in , used to execute Figures 3 to 9 Specifically, the execution device 1800 includes: a receiver 1801, a transmitter 1802, a processor 1803 and a memory 1804 (wherein the number of the processor 1803 in the execution device 1800 may be one or more, Fig.18 In the example of FIG. 1803 , the processor 1803 may include an application processor 18031 and a communication processor 18032. In some embodiments of the present application, the receiver 1801, the transmitter 1802, the processor 1803 and the memory 1804 may be connected via a bus or other means.

[0190] The memory 1804 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1803. A portion of the memory 1804 may also include a non-volatile random access memory (NVRAM). The memory 1804 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0191] The processor 1803 controls the operation of the execution device. In a specific application, the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0192] The method disclosed in the above embodiment of the present application can be applied to the processor 1803, or implemented by the processor 1803. The processor 1803 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1803. The above processor 1803 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1803 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined and executed. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1804, and the processor 1803 reads the information in the memory 1804 and completes the steps of the above method in combination with its hardware.

[0193] The receiver 1801 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1802 can be used to output digital or character information through the interface; the transmitter 1802 can also be used to send instructions to the disk group through the above interface to modify the data in the disk group; the transmitter 1802 can also include a display device such as a display screen.

[0194] In one embodiment of the present application, the application processor 18031 is used to execute Figures 3 to 9 The method for executing the target model described in the corresponding embodiment.

[0195] For application processor 18031 execution Figures 3 to 9The specific implementation methods and beneficial effects of the functions of the target model in the corresponding embodiments can be referred to in Figures 3 to 9 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0196] It should be understood that the above is only one example provided in the embodiment of the present application, and the vehicle may have more or fewer components than those shown, may combine two or more components, or may have different configurations of components.

[0197] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0198] The execution device and training device provided in the embodiment of the present application may specifically be a chip, so that the chip executes the above Figures 3 to 9 Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0199] For details, please refer to Fig.19 , Fig.19 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 190, NPU 190 is mounted on the host CPU (Host CPU) as a coprocessor, and the Host CPU assigns tasks. The core part of the NPU is the operation circuit 1903, which is controlled by the controller 1904 to extract matrix data from the memory and perform multiplication operations.

[0200] In some implementations, the operation circuit 1903 includes multiple processing units (Process Engine, PE) inside. In some implementations, the operation circuit 1903 is a two-dimensional systolic array. The operation circuit 1903 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1903 is a general-purpose matrix processor.

[0201] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1902 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1901 and performs matrix operations with matrix B. The partial results or final results of the obtained matrix are stored in the accumulator 1908.

[0202] The unified memory 1906 is used to store input data and output data. The weight data is directly transferred to the weight memory 1902 through the direct memory access controller (DMAC) 1905. The input data is also transferred to the unified memory 1906 through the DMAC.

[0203] The bus interface unit 1910 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1909 to obtain instructions from the external memory, and is also used for the storage unit access controller 1905 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0204] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1906 or to transfer weight data to the weight memory 1902 or to transfer input data to the input memory 1901.

[0205] The vector calculation unit 1907 includes multiple operation processing units, and further processes the output of the operation circuit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of feature planes, etc.

[0206] In some implementations, the vector calculation unit 1907 can store the processed output vector to the unified memory 1906. For example, the vector calculation unit 1907 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1903, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 1907 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1903, for example, for use in a subsequent layer in a neural network.

[0207] The controller 1904 is connected to an instruction fetch buffer 1909, which is used to store instructions used by the controller 1904; the unified memory 1906, the input memory 1901, the weight memory 1902 and the instruction fetch buffer 1909 are all on-chip memories. The external memory is private to the NPU hardware architecture.

[0208] Among them, the operations of each layer in the recurrent neural network can be performed by the operation circuit 1903 or the vector calculation unit 1907.

[0209] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0210] The present application also provides a chip, which includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit to enable the chip to execute the above Figures 3 to 9 The method described in . Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc. Specifically, the aforementioned processing unit or processor may be a central processing unit (CPU), a network processor (neural-network processing unit, NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0211] In an embodiment of the present application, a computer-readable storage medium is also provided, in which a program for training a model is stored, and when the program is run on a computer, the computer executes the above Figures 3 to 9 The method described in .

[0212] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the above Figures 3 to 9 The steps in the method described in the illustrated embodiment. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0213] The present application also provides a circuit system, which includes a processing circuit, wherein the processing circuit is configured to perform the above Figures 3 to 9 The illustrated embodiments describe steps in a method.

[0214] Through the description of the above implementation methods, the technical personnel in the relevant field can clearly understand that the present application can be implemented by pure software or software plus necessary general hardware, and of course it can also be implemented by dedicated hardware including dedicated integrated circuits, dedicated CLUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or dedicated circuits. However, for this application, software program implementation is a better implementation method in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc., including several instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application. In addition, the computer software product can also be embodied in the form of a control, a driver, an independent or downloadable software object, etc.

[0215] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in a sequence other than the content illustrated or described here. The term "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three kinds of relationships, for example, A and / or B can be represented: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are an "or" relationship. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some ports, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the present application solution.

Claims

1. A method for generating a saliency map, characterized in that: include: Acquire multiple objects, where the multiple objects are acquired after performing disturbance processing on the first object; The multiple objects are screened according to a first condition to obtain the updated multiple objects, the updated multiple objects satisfying a target data distribution, the target data distribution being obtained according to a training sample, the training sample being used to train a preset model to obtain a target model; wherein the first condition is to delete a target object from the multiple objects, the distance between a feature of the target object and a weight vector of the target model exceeds a preset threshold, and the feature of the target object is obtained by extracting a feature of the target object through the target model; obtaining an input of the target model based on the updated plurality of objects; A saliency map of the first object is generated according to the first prediction result output by the target model and the updated multiple objects.

2. The method according to claim 1, characterized in that The features of the target object are specifically extracted through a first feature extraction layer, which is any one of multiple feature extraction layers in the target model. The distance between the features of the target object and the weight vector of the target model is specifically the distance between the features of the target object and the weight vector of the second feature extraction layer, and the second feature extraction layer is any one of the multiple feature extraction layers.

3. The method according to claim 2, characterized in that The first feature extraction layer and the second feature extraction layer are different feature extraction layers.

4. The method according to claim 2 or 3, characterized in that: The first feature extraction layer is specifically the second to last feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is specifically the last feature extraction layer among the multiple feature extractions.

5. The method according to claim 2 or 3, characterized in that: The distance between the feature of the target object and the target weight vector of the second feature extraction layer exceeds the preset threshold, and the distance between the feature of the target object and the target weight vector of the second feature extraction layer is the maximum distance among multiple distances, and the multiple distances include the distance between the feature of the target object and each of the weight vectors of the second feature extraction layer.

6. The method according to claim 2 or 3, characterized in that: The target model is obtained by updating the preset model by a first loss value, wherein the first loss value is determined based on the deviation between the characteristics of the training sample and the weight vector of the preset model, and the characteristics of the training sample are obtained after the preset model extracts features from the training sample.

7. The method according to claim 6, characterized in that The target model is specifically obtained by updating the preset model with the first loss value and the second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the second prediction result and a preset function, the second prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the second prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

8. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Setting the updated weights of the plurality of objects to a first weight; Setting weights of the remaining multiple objects to a second weight, the remaining multiple objects being objects in the multiple objects excluding the updated multiple objects, the first weight being greater than the second weight; The obtaining the input of the target model based on the updated multiple objects includes: A first result is obtained according to the first weight and the updated multiple objects, the first result being an input of the target model, the input of the target model also including a second result, and the second result is obtained according to the second weight and the remaining multiple objects.

9. A method for detecting abnormal objects, characterized in that: include: Get multiple objects; Extracting features from a target object using a feature extraction model to obtain features of the target object, wherein the target object is any one of the multiple objects, and the feature extraction model is obtained by updating a preset model using a first loss value, wherein the first loss value is determined based on a deviation between features of a training sample and a weight vector of the preset model, and the features of the training sample are obtained after the preset model extracts features from the training sample; The distance between the feature of the target object and the weight vector of the feature extraction model is obtained, wherein when the distance exceeds a preset threshold, the target object is an abnormal object.

10. The method according to claim 9, characterized in that The step of extracting features from the target object by using a feature extraction model to obtain features of the target object includes: Performing feature extraction on the target object through a first feature extraction layer to obtain features of the target object, wherein the first feature extraction layer is any one of a plurality of feature extraction layers in the feature extraction model; The obtaining of the distance between the feature of the target object and the weight vector of the feature extraction model includes: The distance between the feature of the target object and the weight vector of a second feature extraction layer is obtained, where the second feature extraction layer is any one feature extraction layer among the multiple feature extractions.

11. The method according to claim 10, characterized in that The first feature extraction layer and the second feature extraction layer are different feature extraction layers.

12. The method according to claim 10 or 11, characterized in that: The first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions.

13. The method according to claim 10 or 11, characterized in that: The weight vectors of the second feature extraction layer are multiple, and the step of obtaining the distance between the feature of the target object and the weight vector of the second feature extraction layer includes: The distance between the feature of the target object and each of the weight vectors of the second feature extraction layer is obtained, wherein when a maximum distance among the multiple distances exceeds the preset threshold, the target object is an abnormal sample.

14. The method according to any one of claims 9 to 11, characterized in that: The feature extraction model is specifically obtained by updating the preset model by the first loss value and the second loss value, wherein the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the first prediction result and a preset function, the first prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the first prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

15. The method according to any one of claims 9 to 11, characterized in that: The multiple objects are obtained after the first image is disturbed, and the method further includes: The target object is deleted from the plurality of objects to obtain the updated plurality of objects, where the updated plurality of objects are used to obtain a saliency map of the first image.

16. The method according to any one of claims 9 to 11, characterized in that: The multiple objects are obtained after the first image is disturbed, and the method further includes: If the distance does not exceed the preset threshold, determining that the target object is a normal object; If the target object is an abnormal object, setting the weight of the target object to a first weight; If the target object is a normal object, setting the weight of the target object to a second weight, wherein the second weight is greater than the first weight; The features of the target object are processed according to the first weight or the second weight to obtain the processed target object, and the processed target object is used to obtain a saliency map of the first image.

17. A device for generating a saliency map, characterized in that: include: An acquisition module, used for acquiring a plurality of objects, wherein the plurality of objects are acquired after perturbation processing is performed on the first object; A screening module, used for screening the multiple objects acquired by the acquisition module according to a first condition to acquire the updated multiple objects, wherein the updated multiple objects satisfy a target data distribution, wherein the target data distribution is acquired according to a training sample, and the training sample is used to train a preset model to acquire a target model; wherein the first condition is to delete a target object from the multiple objects, wherein the distance between the feature of the target object and the weight vector of the target model exceeds a preset threshold, and the feature of the target object is acquired after extracting the feature of the target object by the target model; A generating module is used to generate a saliency map of the first object according to a first prediction result output by the target model and the updated multiple objects, wherein the input of the target model is obtained based on the updated multiple objects.

18. The device according to claim 17, characterized in that The features of the target object are specifically extracted through a first feature extraction layer, which is any one of multiple feature extraction layers in the target model. The distance between the features of the target object and the weight vector of the target model is specifically the distance between the features of the target object and the weight vector of the second feature extraction layer, and the second feature extraction layer is any one of the multiple feature extraction layers.

19. The device according to claim 18, characterized in that The first feature extraction layer and the second feature extraction layer are different feature extraction layers.

20. The device according to claim 18 or 19, characterized in that The first feature extraction layer is specifically the second to last feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is specifically the last feature extraction layer among the multiple feature extractions.

21. The device according to claim 18 or 19, characterized in that The distance between the feature of the target object and the target weight vector of the second feature extraction layer exceeds the preset threshold, and the distance between the feature of the target object and the target weight vector of the second feature extraction layer is the maximum distance among multiple distances, and the multiple distances include the distance between the feature of the target object and each of the weight vectors of the second feature extraction layer.

22. The device according to any one of claims 17 to 19, characterized in that The target model is obtained by updating the preset model by a first loss value, wherein the first loss value is determined based on the deviation between the characteristics of the training sample and the weight vector of the preset model, and the characteristics of the training sample are obtained after the preset model extracts features from the training sample.

23. The device according to claim 22, characterized in that The target model is specifically obtained by updating the preset model with the first loss value and the second loss value, the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the second prediction result and a preset function, the second prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the second prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

24. The device according to any one of claims 17 to 19, characterized in that It also includes weight modules, The weight module is used to: Setting the updated weights of the plurality of objects to a first weight; Setting weights of the remaining multiple objects to a second weight, the remaining multiple objects being objects in the multiple objects excluding the updated multiple objects, the first weight being greater than the second weight; One input of the target model is specifically obtained based on the updated multiple objects and the first weights, and another input of the target model is obtained based on the remaining multiple objects and the second weights.

25. A device for detecting abnormal objects, characterized in that: include: A first acquisition module, used to acquire multiple objects; a feature extraction module, configured to extract features of a target object through a feature extraction model to obtain features of the target object, wherein the target object is any one of the multiple objects, and the feature extraction model is obtained by updating a preset model through a first loss value, wherein the first loss value is determined according to a deviation between features of a training sample and a weight vector of the preset model, and the features of the training sample are obtained after the preset model extracts features of the training sample; The second acquisition module is used to acquire the distance between the feature of the target object and the weight vector of the feature extraction model, wherein when the distance exceeds a preset threshold, the target object is an abnormal object.

26. The device according to claim 25, characterized in that The feature extraction module is specifically used for: Performing feature extraction on the target object through a first feature extraction layer to obtain features of the target object, wherein the first feature extraction layer is any one of a plurality of feature extraction layers in the feature extraction model; The obtaining of the distance between the feature of the target object and the weight vector of the feature extraction model includes: The distance between the feature of the target object and the weight vector of a second feature extraction layer is obtained, where the second feature extraction layer is any one feature extraction layer among the multiple feature extractions.

27. The device according to claim 26, characterized in that The first feature extraction layer and the second feature extraction layer are different feature extraction layers.

28. The device according to claim 26 or 27, characterized in that The first feature extraction layer is the penultimate feature extraction layer among multiple feature extraction layers in the feature extraction model, the multiple feature extraction layers are connected end to end, the target object is the input of the first feature extraction layer among the multiple feature extraction layers, and the second feature extraction layer is the last feature extraction layer among the multiple feature extractions.

29. The device according to claim 26 or 27, characterized in that The second acquisition module is specifically used for: The distance between the feature of the target object and each of the weight vectors of the second feature extraction layer is obtained, wherein when a maximum distance among the multiple distances exceeds the preset threshold, the target object is an abnormal sample.

30. The device according to any one of claims 25 to 27, characterized in that The feature extraction model is specifically obtained by updating the preset model by the first loss value and the second loss value, wherein the second loss value is determined based on the deviation between the target result and the actual result of the training sample, the target result is determined based on the first prediction result and a preset function, the first prediction result is the prediction result of the preset model for the training sample, the input of the preset function is the first prediction result, the output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

31. The device according to any one of claims 25 to 27, characterized in that The multiple objects are obtained after the first image is disturbed, and the device further includes a generating module, the generating module being used to: The target object is deleted from the plurality of disturbed first images to obtain the plurality of updated objects, and the plurality of updated objects are used to obtain a saliency map of the first image.

32. The device according to any one of claims 25 to 27, characterized in that The multiple objects are obtained after the first image is disturbed, and when the distance does not exceed a preset threshold, the target object is a normal object. The device also includes a weight module and a generation module. The weight module is used to: If the target object is an abnormal object, setting the weight of the target object to a first weight; If the target object is a normal object, setting the weight of the target object to a second weight, wherein the second weight is greater than the first weight; The generating module is used to process the features of the target object according to the first weight or the second weight to obtain the processed target object, and the processed target object is used to obtain the saliency map of the first image.

33. A device for generating a saliency map, characterized in that: The method comprises a processor, wherein the processor is coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

34. A device for detecting abnormal objects, characterized in that: The method comprises a processor, wherein the processor is coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 9 to 16 is implemented.

35. A computer-readable storage medium, characterized in that: The invention comprises a program which, when being executed on a computer, causes the computer to execute the method according to any one of claims 1 to 8, or causes the computer to execute the method according to any one of claims 9 to 16.

36. A circuit system, characterized in that: The circuit system comprises a processing circuit configured to perform the method of any one of claims 1 to 8 or configured to perform the method of any one of claims 9 to 16.

37. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are loaded and executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 8, or the electronic device executes the method according to any one of claims 9 to 16.

38. A chip, characterized in that: The chip is coupled to a memory and is used to execute a program stored in the memory to execute the method according to any one of claims 1 to 8, or to execute the method according to any one of claims 9 to 16.

Images