A system and method for interpretable image classification using essentially explainable neural networks

Abstract

The artificial intelligence-based image processing system comprises a processor that, by executing instructions stored in memory, classifies an input image using a prototype sub-neural network including a backbone subnetwork, a prototype subnetwork, and a readout subnetwork to generate an interpretable classification of the input image, which includes either or a combination of the classification result and / or interpretation of the classification result. The backbone subnetwork is trained using machine learning to generate feature embeddings that represent features extracted from pixels in different regions of the input image by processing the input image with an incomplete sequence of active convolutional layers. The prototype subnetwork is trained to compare the feature embeddings with the prototype feature embeddings to generate comparison results, and the readout subnetwork is configured to analyze the comparison results to generate an interpretable classification of the input image.

Description

[Technical Field]

[0001] This disclosure relates in general to image processing, and more specifically to the recognition of objects within images. [Background technology]

[0002] Neural networks are powerful models for learning arbitrary classification or regression functions. Often, along with the results of the computational process performed with the help of a neural network, it is desirable that the output of the neural network also provides the reasoning behind why the input was classified in a particular way. However, one of the main drawbacks of neural networks is that they are black boxes, which essentially means they do not provide an understanding of why the neural network classified the input in a particular way. A standard neural network provides no explanation of any kind for its output. As AI evolves more and more, humans are required to understand how algorithms arrive at their results and trace back the process. The entire computational process is turned into something that cannot be interpreted, commonly referred to as a "black box." These black-box models are created by neural networks directly from data, and even the engineers or data scientists who designed the algorithms cannot understand or explain exactly what is happening inside or how the AI ​​algorithm arrived at a particular result.

[0003] Explainable artificial intelligence (XAI) is a set of processes and methods that enable human users to understand and trust the results and outputs produced by machine learning algorithms. Explainable AI is used to describe AI models, their expected effects, and potential biases. This helps characterize the accuracy, fairness, transparency of the model, and the deliverables in AI-powered decision-making. For organizations, explainable AI is crucial for building trust and confidence when deploying AI models into production. AI explainability also helps organizations adopt an responsible approach to AI development.

[0004] Some advances in explainable AI have been achieved using prototype subset networks, sometimes called partial prototype networks. Prototype subset networks attempt to add some interpretability to a neural network by learning a set of prototype parts for each training class. The prototype parts are intended to represent characteristic local regions from an image of an object. However, all such means of learning prototype parts suffer from low explainability. For example, in the field of image processing, prototypes learned by available prototype subset networks actually have receptive fields that encompass the entire input image, rather than just local regions of the input image.

[0005] Therefore, there is a need for improved methods to realize explainable AI that can provide a robust explanation of the reasoning behind the classification of inputs. [Overview of the project]

[0006] The objective of some embodiments is to provide means for making AI-based decisions more understandable and interpretable. Examples of some embodiments are directed towards efficiently explaining deep neural networks. Examples of some embodiments are also directed towards improving the integration and adoption of AI-based models into technical fields such as image processing, anomaly detection, and data security.

[0007] Examples of several embodiments are based on the recognition that there are numerous benefits associated with understanding how an AI-enabled system was led to a particular output. For example, explainability can help ensure that the system is operating as expected, or that it may be necessary to meet regulatory standards, or that those affected by the decision may be able to question or modify its output. Examples of several embodiments also recognize that explainability in AI is important for the adoption and integration of AI systems with other devices and algorithms. Some embodiments also recognize that the performance of an AI model may drift or degrade because production data differs from training data. Therefore, it becomes important to continuously monitor and manage the model to promote the explainability of the AI ​​while evaluating the impact that the use of such algorithms will have on the business. Explainable AI also helps to promote end-user trust, model auditability, and the productive use of the AI. This also mitigates compliance, legal, security, and reputational risks for production AI.

[0008] Some examples of embodiments are based on the recognition that, in order to meet the above needs and criteria, the explainability of AI-based decision-making must be robust and clear. Some examples of embodiments also recognize that, in order to improve classification accuracy, it is desirable that the receptive field for training the prototype encompasses the entire input data.

[0009] Examples of several embodiments also recognize that, in the context of AI-based image processing, one way to provide explainability about data classification is to match learned prototype portions with portions of the input image, identify the best-matching prototypes and corresponding matching locations within the input image, and provide explainability for the output of the prototype subset network. For example, the explanation might be that a particular set of portions (i.e., regions) of the input image well matches a particular set of prototype portions of a particular object class. The explainability of such a prototype subset network would be limited to a morphological explanation, such as "these regions in the input image match these portions of a typical image of a certain object class, and therefore this object class was selected as the output." However, some applications require more meaningful explanations about classification decisions, such as explanations that can help tune the underlying AI model. Thus, examples of several embodiments are based on the recognition that, for classification, a learned prototype with a receptive field encompassing the entire input data may not be optimal for generating an explanation about the classification.

[0010] Examples of some embodiments provide systems and methods for improved explainability of underlying AI algorithms. Examples of some embodiments provide means and strategies for prototype subset networks that improve explainability by learning prototypes that precisely correspond to local regions of an image.

[0011] Some embodiments of the present invention reflect the recognition that, as AI approaches become more sophisticated, decision-making is increasingly being made by ML models whose design and the rationale behind their decision-making processes are opaque to the user. Similarly, some embodiments of the present invention reflect the recognition that the opacity of such ML models hinders AI explainability and, consequently, undermines user confidence in the resulting outcomes. Therefore, some embodiments of the present invention reflect the recognition that, ideally, AI explainability provides users with interpretable insights into how an ML model performed a particular action and why it arrived at a particular decision. At this point, it is essential to understand that while AI models may offer better decision-making than other means, a lack of robust explainability for such models can hinder their adoption for scaled-down operations. One important application area targeted by some embodiments is image processing using artificial intelligence-based models.

[0012] Examples of several embodiments point to improved network architectures for image classification. These improved network architectures include a mechanism for learning a set of prototypes for each training class associated with local regions of a training image. During classification, as part of the process, the learned prototypes are matched against local regions of the input image to classify the image (e.g., classify the types of objects contained in the image). The best-matching prototypes serve as an explanation for why the image was classified in a particular way. Prototypes may be used to say that a particular part of the image matched a particular part of the training image, providing evidence for the final classification of the image. In some embodiments, using prototype sub-subnetworks, the neural network may provide the best-matching prototypes and the locations where they matched most as evidence for a particular classification.

[0013] Some examples of embodiments also recognize that processing time for such classification is a critical parameter for employing explainable AI in many applications. Some examples of embodiments also recognize that processing time for image classification depends on whether the prototype corresponds to a local region of the training image or the entire training image. Therefore, an objective of some examples of embodiments is to reduce the processing time for classifying input images. In this regard, some examples of embodiments limit the receptive field of the network layer from which the prototype is extracted.

[0014] The objective of some embodiments is to restrict the embedded patches of the training images from which prototypes are extracted to correspond to local regions of the input image rather than the complete image. Thus, some embodiments modify the count of layers used to extract features from the input image to be less than the total count of layers available for feature extraction. In this way, some embodiments utilize an incomplete sequence of layers to generate feature embeddings representing features extracted from pixels in different regions of the input image. In this context, a complete sequence of extraction layers can be understood as ensuring that each feature embedding contains features derived from pixels across the entire input image. Thus, the number of layers used for feature extraction (i.e., active layers) may be a tunable parameter, allowing for fine-tuning of the explainability of AI-based decision-making during the classification of input images by adjusting the number of layers utilized for feature extraction. In some embodiments, the number of active layers used may be considered a parameter for image classification.

[0015] Examples of several embodiments yield local prototypes that precisely correspond to object portions within the input image, and therefore the neural networks disclosed herein have superior interpretability according to different interpretability metrics. In addition, the improved neural networks disclosed herein have provisions for utilizing fewer layers than the total number of available layers for feature extraction, thus significantly reducing the total time spent on image classification. Thus, along with the improved explainability of AI-based image classification, examples of several embodiments also result in faster image classification systems. For this reason, several application areas that would otherwise not incorporate AI-based decision-making can benefit from the seamless integration of the disclosed AI-based image classification methods and systems to perform a variety of tasks.

[0016] To achieve the above objectives and advancements, some embodiments provide artificial intelligence-based image processing systems, methods, and computer program products for interpretable classification of input images.

[0017] Examples of several embodiments provide an AI-based image processing system for interpretable classification of input images. The system comprises a processor and a memory storing instructions, which, when executed by the processor, cause the AI ​​image processing system to classify an input image using a prototype sub-neural network and output an interpretable classification of the input image. The prototype sub-neural network generates an interpretable classification of the input image, which includes either or a combination of the classification result and / or interpretation of the classification result. In this regard, the prototype sub-neural network includes a backbone subnetwork, a prototype subnetwork, and a readout subnetwork. The backbone subnetwork is trained using machine learning to generate feature embeddings that represent features extracted from pixels in different regions of the input image by processing the input image with an incomplete sequence of active convolutional layers. The prototype subnetwork is trained to compare the feature embeddings with the prototype feature embeddings to generate comparison results, and the readout subnetwork analyzes the comparison results to generate an interpretable classification of the input image.

[0018] In another embodiment, an image processing method for interpretable classification of an input image is provided. This method includes classifying an input image using a prototype sub-neural network comprising a backbone subnetwork, a prototype subnetwork, and a readout subnetwork to generate an interpretable classification of the input image, which includes either or a combination of the classification result and / or an interpretation of the classification result. The prototype sub-neural network comprises a backbone subnetwork, a prototype subnetwork, and a readout subnetwork. The backbone subnetwork is trained using machine learning to generate feature embeddings that represent features extracted from pixels in different regions of the input image by processing the input image with an incomplete sequence of active convolutional layers. The prototype subnetwork is trained to compare the feature embeddings with prototype feature embeddings to generate a comparison result, and the readout subnetwork analyzes the comparison result to generate an interpretable classification of the input image. This method further includes outputting an interpretable classification of the input image.

[0019] In some further examples of other embodiments, a non - transient computer - readable medium storing computer - executable instructions for performing a method of interpretable classification of an input image is provided. The method includes classifying an input image using a prototype partial neural network including a backbone subnet, a prototype subnet, and a readout subnet to generate an interpretable classification of the input image including one or a combination of a classification result of the input image and an interpretation of the classification result. The prototype partial neural network includes a backbone subnet, a prototype subnet, and a readout subnet. The backbone subnet is trained using machine learning to generate a feature embedding representing features extracted from pixels of different regions of the input image by processing the input image using an incomplete sequence of active convolutional layers. The prototype subnet is trained to compare the feature embedding with a prototype feature embedding to generate a result of the comparison, and the readout subnet analyzes the result of the comparison to generate an interpretable classification of the input image. The method further includes outputting an interpretable classification of the input image.

[0020] The embodiments disclosed herein will be further described with reference to the following drawings. The drawings shown are not necessarily to scale, but rather are generally focused on explaining the principles of the embodiments of the present disclosure.

Brief Description of the Drawings

[0021] [Figure 1A] A block diagram of a framework of an AI - based explainability engine according to some example embodiments is shown. [Figure 1B] A diagram showing a workflow of an AI - based explainability engine according to some example embodiments. [Figure 2A] A diagram showing a network architecture of a prototype partial neural network for interpretable classification of an input image for object recognition according to some example embodiments. [Figure 2B]A diagram showing a flowchart of steps executed by an AI-based image processing system for interpretable classification of input images according to examples of some embodiments. [Figure 3A] A diagram showing a feature map of a convolutional layer constituting a backbone sub-network of the prototype partial neural network of FIG. 2A according to examples of some embodiments. [Figure 3B] A diagram showing the total number of active convolutional layers constituting the backbone sub-network of FIG. 3A according to examples of some embodiments. [Figure 3C] A diagram showing a flowchart of steps executed by the backbone sub-network of FIG. 3A according to examples of some embodiments. [Figure 4A] A diagram showing a prototype sub-network of the prototype partial neural network of FIG. 2A according to examples of some embodiments. [Figure 4B] A diagram showing a flowchart of steps executed by the prototype sub-network of FIG. 4A according to examples of some embodiments. [Figure 5A] A diagram showing a readout sub-network of the prototype partial neural network of FIG. 2A according to examples of some embodiments. [Figure 5B] A diagram showing a flowchart of steps executed by the readout sub-network of FIG. 5A according to examples of some embodiments. [Figure 6A] A diagram showing a use case as a specific example for generating context-based audio for video according to examples of some embodiments. [Figure 6B] A diagram showing a use case as another specific example for detecting anomalies in a scene according to examples of some embodiments. [Figure 6C] A diagram showing a use case as another specific example for explaining anomalies detected in a factory setup according to examples of some embodiments. <​A block diagram of a system for implementing several components of an AI-based image processing system, relating to some examples of embodiments, is shown. [Modes for carrying out the invention]

[0022] The above drawings illustrate embodiments disclosed herein, but other embodiments are also intended as described in the specification. This disclosure presents embodiments that are illustrative and not limiting. Those skilled in the art can devise numerous other variations and embodiments that fall within the scope and spirit of the principles of the embodiments disclosed herein.

[0023] The following description provides only specific embodiments and is not intended to limit the scope, applicability, or configuration of the Disclosure. Rather, the following description of specific embodiments will provide a description that enables the realization of one or more specific embodiments for those skilled in the art. Various modifications are intended to be made to the function and configuration of the elements without departing from the spirit and scope of the disclosed subject matter as described in the appended claims.

[0024] Specific details are provided in the following description to ensure a full understanding of the embodiments. However, those skilled in the art will understand that the embodiments can be carried out even without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in the form of block diagrams to avoid obscuring the embodiments with unnecessary details. In other examples, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. Furthermore, similar reference numbers and names in different drawings refer to similar elements.

[0025] Furthermore, individual embodiments may be described as processes shown as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may terminate when its operations are complete, but it may have additional steps that are not discussed or included in the diagrams. Moreover, not all operations in any specifically described process can occur in all embodiments. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, the termination of the function may correspond to the function returning to a calling function or main function.

[0026] Furthermore, embodiments of the disclosed subject matter may be implemented manually or automatically, at least in part. Manual or automatic implementation may be performed, or at least assisted, through a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. If implemented in software, firmware, middleware, or microcode, program code or code segments for performing the required tasks may be stored in a machine-readable medium. A processor(s) may perform the required tasks.

[0027] Different types of neural networks are finding applications in many fields of application. There is a growing attempt to automate many tasks with the help of artificial intelligence to introduce speed and efficiency into the execution of such tasks. Neural networks are powerful models for learning arbitrary classification or regression functions. Given input, a trained neural network will classify at least some of the data into specific learned classes. However, standard neural networks do not provide the reasoning behind such classifications, leaving a gap in understanding how neural networks work and perform tasks. As AI evolves, humans are required to grasp how algorithms arrive at their results and trace back the process. This aspect of understanding how AI-based models implemented using neural networks classify data is called explainability of the output of such networks. Explainability of AI-based models is crucial in many applications and use cases. Ensuring the completeness and reliability of explainability in AI-based decision-making is another challenge in this field. The importance of explainability in AI decision-making processes can be understood from the fact that it is crucial for organizations to fully understand AI decision-making processes with AI model monitoring and accountability, and not to blindly trust them. Explainable AI can help humans understand and explain machine learning (ML) algorithms, deep learning, and neural networks.

[0028] Machine learning (ML) models are often considered black boxes that are impossible to interpret. Neural networks used in deep learning are among the most difficult for humans to understand. Biases, often based on race, gender, age, or region, have long been a risk in training AI models. Furthermore, the performance of AI models can drift or degrade because production data differs from training data. Therefore, it is crucial to continuously monitor and manage models to promote the explainability of AI while assessing the impact of using such algorithms on the business. Explainable AI also helps to promote end-user trust, model auditability, and the productive use of AI. This also mitigates compliance, legal, security, and reputational risks for production AI.

[0029] Explainable AI is one of the key requirements for implementing responsible AI, and in real organizations, it is a methodology for large-scale implementation of AI methods with fairness, model explainability, and accountability. To help organizations adopt AI responsibly, they need to incorporate ethical principles into their AI applications and processes by building AI systems based on trust and transparency. All of this requires a robust explanation of the reasoning behind how the model arrived at its decisions, something that is lacking in available models.

[0030] Engineers can use explainable AI to troubleshoot and improve model performance while helping stakeholders understand the behavior of AI models. Investigating model behavior through tracking model insights regarding deployment status, fairness, quality, and drift is essential for scaling AI. Continuous model evaluation allows us to compare model predictions, quantify model risk, and optimize model performance. Model evaluation is accelerated by displaying positive and negative values ​​in model behavior along with the data used to generate explanations.

[0031] Therefore, as used herein, interpretability or explainability means the degree to which an observer can understand the reasoning behind a decision. Thus, in the sense of machine learning, interpretability or explainability means the ability to interpret or explain how a machine learning model produced results, why the machine learning model produced those results, and / or otherwise. However, as machine learning models become increasingly complex, it can be difficult to directly interpret or explain the results of machine learning. However, known AI engines do not retain the origin of the deductive processes they apply, for example, they do not retain logs of the rules applied by the engine. Moreover, known techniques do not provide traceability (including complete traceability) and origin of interpretation, reasoning, deliberation, and judgment through various layers, including from raw evidence to final observation. Furthermore, known techniques do not provide flexibility in presenting generated recommendations and / or improvements, adaptability in light of evolving needs, or minimization and localization of necessary changes within the system. Furthermore, known techniques do not provide a robust and flexible framework for applying different classifications / vocabulary / logic to each of multiple layers, generating output data and explainability outputs in each of multiple layers, and allowing local modifications to any layer without modifying others.

[0032] Therefore, the objective of some examples of embodiments is to provide means to make AI-based decision-making more understandable and interpretable. Some examples of embodiments are directed toward efficiently explaining deep neural networks. Some examples of embodiments are also directed toward improving the integration and adoption of AI-based models into the technical fields of image processing, anomaly detection, and data security. In the domain of image processing, the processing time for image classification depends on whether the prototype corresponds to a local region of the training image or the entire training image. Therefore, the objective of some examples of embodiments is to reduce the processing time for classifying input images. In this regard, some examples of embodiments limit the receptive field of the network layer from which the prototype is extracted.

[0033] To achieve the above objectives and advancements, examples of several embodiments provide systems, methods, and computer program products for artificial intelligence-based image processing for interpretable classification of input images. A detailed description of the structure and operation of such embodiments is then provided.

[0034] Figure 1A shows a block diagram of an AI-based explainability engine framework 100A relating to some examples of embodiments. The explainability engine 132 may be implemented as a prototype partial neural network on an end device 130, such as a computing device having a processor 134 and memory 136. The end device 130 may be implemented as a standalone computing device or as a distributed computing device, such as one implemented on a cloud server. The processor 134 may perform functions according to instructions stored in memory 136. The explainability engine 132 may be embodied as one or more computer programs or software, or as a mixture of hardware and software. The end device 130 may interface with a server 110, an input image database 140, and a reference image database 150 through a communication network 120. In this way, the processor 134 may fetch or accept data from the server 110 and / or databases 140 and 150. The input image database 140 may store input images for processing by the end device 130. The input image database 140 may be compiled using images provided by appropriate imaging sources. The input images stored in the input image database 140 may include still images and videos of one or more scenes containing objects. The reference image database 150 may store training images that the trainer module 102 uses to train the explainability engine 132. In some embodiments, one or both of the databases 140 and 150 may divide the video into frames and store them in a suitable storage medium.

[0035] Server 110 may store a trainer module 102 which is embodied as a software program and implemented through appropriate computing equipment. As part of the training process, the explainability engine 132 may be trained to learn multiple prototypes or objects or parts thereof. The explainability engine 132 may be trained to learn a set of prototype parts corresponding to each training class. At runtime, a method for explaining the output of the explainability engine may be provided by matching these learned prototype parts with parts of one or more input images in the input image database 140 to determine the best-matching prototype and its corresponding matching location in the input image.

[0036] The architecture of the explainability engine 132 may include a cascaded combination of multiple subnetworks. Details of the structure and operation of the explainability engine 132 are then provided with reference to Figure 1B, which illustrates a workflow of the AI-based explainability engine 132 in some examples of embodiments. While these details are described in the context of image processing, it may be intended that the explainability engine may be equally applicable to any form of input data.

[0037] Referring to Figure 1B, the explainability engine 132, implemented as a prototype sub-neural network, comprises a backbone subnetwork 132A, a prototype subnetwork 132B, and a readout subnetwork 132C. The explainability engine 132 takes input data 142 as input to the backbone subnetwork 132A and outputs the classification result for the input data and the explainability of the classification as output data 180. In some embodiments, the input data may be images captured by a suitable imaging device. The backbone subnetwork 132A is trained to learn a set of prototypes for each training class associated with local regions of the training image. In some embodiments, the backbone subnetwork 132A is trained using machine learning to generate feature embeddings that represent features extracted from pixels in different regions of the input image by processing the input image with an incomplete sequence of active convolutional layers. As part of its workflow, the backbone subnetwork 132A performs a function called feature extraction 162 and a function called patch embedding 164 of the input image. In this regard, the backbone subnetwork 132A includes multiple embedding layers for generating feature embeddings. The embedding layers may be implemented using different neural network layers, for example, a series of convolutional layers using a Rectified Linear Unit (ReLU) or other nonlinear activation function. According to some embodiments, the complete sequence of active convolutional layers in the backbone subnetwork 132A causes each feature embedding to contain features derived from pixels of the entire input image. Thus, by utilizing an incomplete sequence of active convolutional layers (i.e., fewer layers than the complete sequence), embodiments of the present invention significantly reduce the processing time required to generate feature embeddings, thereby speeding up the entire image classification job and reducing the consumption of processing resources compared to standard AI models.

[0038] The prototype subnetwork 132B is trained to compare feature embeddings extracted from input images with prototype feature embeddings obtained from training images. As part of its workflow, the prototype subnetwork 132B performs a function called patch-prototype comparison 166 and a function called pooling of similarity scores obtained as an artifact of the prototype comparison 168. According to the example of some embodiments, the set of prototype embeddings may be defined for each training class and may be learned during training along with other network weights. During execution, the prototype subnetwork 132B generates a similarity map for each prototype by comparing each prototype with all patches (feature embeddings) of the input images according to a distance function. The prototype subnetwork 132B then pools the similarity maps using max pooling to generate a set of scalar similarity scores for each prototype.

[0039] The readout subnetwork 132C analyzes the results of comparisons performed by the prototype subnetwork 132B to generate an interpretable classification of the input image. As part of its workflow, the readout subnetwork 132C performs the functions of summing the similarity scores for each class 170 for all prototypes and generating a corresponding logit 172. Following the example of some embodiments, the readout subnetwork 132C sums the similarity scores for all prototypes of a particular class to generate class-specific similarity scores. These similarity scores are then converted into a set of class-specific probabilities using a softmax function. Given a classification of an input image, the prototypes that best match that class are shown along with the locations where they had the highest similarity, providing an explanation for why the image was classified in that way.

[0040] In some embodiments, the output data 180 may include classifications of portions of the image into one or more classes, and associated explainability / justifications behind each classification. That is, the explainability engine 132 predicts the object classes present in the input image and provides justification for its output with respect to image regions from the training images (referred to as prototype portions) for the output class that best fits the input image. The best-fitting prototype portion represents the reason for the network's classification decision, i.e., "This output class was selected because the input image has portions that closely match these prototype portions found in the training images of this class."

[0041] The aspects of the operation of the explainability engine 132 can be understood from the description of a deep neural network as a concrete example for interpretable classification of input images for object recognition. In this regard, Figure 2A shows a network architecture 200A of one such deep neural network for interpretable classification of input images for object recognition, according to an example of several embodiments, and Figure 2B shows a flowchart 200B of steps performed by an AI-based image processing system for interpretable classification of input images, according to an example of several embodiments.

[0042] Referring to Figures 2A and 2B, the deep neural network is implemented by a backbone subnetwork 232A, a prototype subnetwork 232B, and a readout subnetwork 232C. The deep neural network can be considered trained in a manner similar to that described with reference to Figure 1B, such that the backbone subnetwork 232A is trained to generate feature embeddings by processing the input image 202 with an incomplete sequence of active convolutional layers (272). The feature embeddings are compared with prototype feature embeddings obtained from training images and learned by the deep neural network during training (274), thereby obtaining a similarity score as part of the result of the comparison. The result of the comparison in the form of similarity scores is analyzed (276), thereby generating an interpretable classification of the input image (278). According to the example of some embodiments, the trained deep neural network is configured to be executed by a computing circuit to process the input image 202 to generate one or more classifications of the input image 202 (278), and to output this classification along with an explanation of the reasoning behind the classification (280). In the following, the output class and the explanation behind that classification may be referred to as an interpretable classification.

[0043] Referring to Figure 2A, the backbone subnetwork 232A comprises multiple convolutional neural network (CNN) layers that are responsible for embedding the image. The backbone subnetwork 232A takes the input image 202 as input.

number

[0044] The complete feature embedding function for the backbone subnetwork 232A is f = f add (f core ) and can be shown as such. This function f yields [Number] which has D channels, height H z , and width W z . Z i is a feature map and corresponds to the final output of the backbone subnetwork 232A. The feature map has H z ×W z pixels, and each pixel is a real-valued length D vector.

[0045] Figure 3A shows a diagram of the feature maps of the convolutional layers constituting the backbone subnetwork 232A of the deep neural network in Figure 2A, according to some examples of embodiments. As shown in Figure 3A, a pixel 306A in feature map 306 is computed from a neighboring pixel 304B in the preceding feature map 304. Similarly, a pixel 304A is computed from a neighboring pixel in the preceding feature map. This continues for the intermediate feature maps 303 until the input image 202 is reached. The size of the neighborhoods is determined by the details of the selected backbone convolutional neural network. For example, in some embodiments, the size of the neighborhoods may typically be 3x3 or 5x5, but is not limited to these. In this case, the pixels of the second-to-last layer feature map 304 are determined by larger neighboring pixels in the preceding feature map (of the intermediate feature maps 303). The set of pixels in the input image 202 used to compute the pixels in the final feature map 306 is called the receptive field. For example, the set of pixels 302B is the receptive field for pixel 306A in feature map 306. Similarly, the set of pixels 302A is the receptive field for pixel 304A in feature map 304.

[0046] As shown in Figure 3A, the number of convolutional layers in the backbone subnetwork directly affects the size of the receptive field of each pixel in the final feature map. The more convolutional layers present in the backbone layer, the larger the receptive field 302B for pixel 306A in the final feature map 306. In some scenarios, if the number of backbone layers is sufficiently large, each pixel in the final feature map 306 will have a receptive field the size of the entire input image 202. Such a number of layers may be called a "complete" number of layers. This is shown in Figure 3B, where for pixel 326A in the final feature map 326, the receptive field 322A corresponds to the entire input image 322. In such a scenario, the count of convolutional layers constituting the backbone subnetwork (i.e., all intermediate layers 303 and 304) is above a threshold. Thus, the number of convolutional layers constituting a backbone subnetwork where the receptive field of a pixel in the final feature map covers the entire input image (i.e., all pixels in the input image) can be called a complete number of layers.

[0047] However, in some scenarios, if the number of layers is less than the complete number of layers, the backbone layer can be considered to have an "incomplete" number of layers. Referring to Figure 3A, for an incomplete number of layers, the pixels in the final feature map 306 have receptive fields corresponding to local regions 302B of the input image 202, which can also be called parts of the input image 202. For example, in the scenario shown in Figure 3A, for an incomplete number of active convolutional layers, the pixels 304A in the feature map 304 have their receptive field 302A corresponding to local regions of the input image 202. As shown in Figure 3B, for a complete number of layers, the final feature map has receptive fields that cover the entire input image and therefore do not correspond to local regions or parts. A complete number of layers typically results in higher accuracy, but with lower interpretability and greater processing delay. An incomplete number of layers results in higher interpretability because the pixels in the final feature map 304 represent local regions (such as 302B) of the input image 202 used as prototypes in the system in Figure 2A. Furthermore, an incomplete number of layers results in smaller processing delays. Therefore, the number of layers used may be fine-tuned depending on the desired level of accuracy in the explainability of the decision. In this regard, some embodiments consider the number of active convolutional layers used in the decision as a user-defined or adjustable parameter. In some embodiments, the optimal number of layers to use may be dynamically determined by comparing the final output with a threshold result, and the process may be repeated if the output is not similar to the result. Thus, in some embodiments, the same input image may be processed iteratively by different layers of the backbone subnetwork, with a different number of active convolutional layers selected in each iteration. In some embodiments, the number of layers may be selected as a parameter, for example, based on a user-defined input.

[0048] Figure 3C shows a flowchart of the steps performed by the backbone layer in Figure 3A, relating to some examples of several embodiments. Figure 3C will be described with reference to Figures 2A and 3A. The backbone subnetwork 232A takes a multichannel image 332 as its input. Depending on the particular embodiment, the backbone subnetwork 232A may consist of several convolutional and pooling layers, along with a nonlinear activation function that constitutes the core layer 252. As part of the processing 334 by the backbone subnetwork, the core layer 252 performs processing 334A of the input image to compute a large number of channels of high-level features from the input image 202. This may be followed by processing 334B by an add-on layer 254 that maps the large number of channels output from the core layer 252 into a smaller set of channels to reduce the size of the feature embedding. In step 336, the backbone subnetwork 232A provides the final feature map Z to the prototype subnetwork 232B for further processing. i Outputs.

[0049] Referring again to Figure 2A, the prototype subnetwork 232B finds the most similar embedded patch z of the input image 202 for each prototype. In the context of the present invention, each prototype can be understood as an embedding of some prototype portion of an object. Each prototype is class-specific, class y i A set of prototypes belonging to

number

[0050] Figure 4A shows a diagram of the prototype subnetwork 232B of the deep neural network in Figure 2A, according to some examples of embodiments. Figure 4B shows a flowchart of the steps performed by the prototype layer in Figure 4A, according to some examples of embodiments.

number

[0051]

number

number

[0052]

number

[0053] Figure 5A shows a diagram of the read subnet 232C of the deep neural network of Figure 2A, according to an example of several embodiments. Figure 5B shows a flowchart of the steps performed by the read subnet 232C of Figure 5A, according to an example of several embodiments. The read subnet 232C of Figures 2A and 5A reads the logit 526,

number

number

[0054] As illustrated with reference to Figure 1A, the explainability engine 132 may be implemented as an end device for AI-based image processing used in several applications. In this regard, the explainability engine 132 may be implemented as an AI-based image processing engine. These use cases will now be described with reference to Figures 6A and 6B.

[0055] Figure 6A shows a use case as a concrete example of generating context-based audio for video using an AI-based image processing engine 610, according to some examples of embodiments. The image processing engine 610 is supplied with input video provided from a suitable imaging source 602A. In some examples of embodiments, the input video may be live video, such as video 604A of an ongoing soccer match, and may be provided directly to the image processing engine 610. In some examples of embodiments, the video may be pre-captured and stored in a database, such as the input image database 140 in Figure 1A.

[0056] Regardless of the video source, the image processing engine 610 processes the video by dividing it into individual frames or images, applying any appropriate object detector algorithm to discover image patches containing objects, each of which is processed according to the workflow and process described with reference to Figures 1B to 5B to generate multiple classification results corresponding to video 604A. According to the example of some embodiments, the classification results 612 may include one or more object classes corresponding to objects in the video, and their explainability data that provides the basis for the validity of the corresponding classification. For example, the classification results 612 may include identification of object classes and attributes such as human, gender, football / soccer player, player's ethnicity, age, ball, soccer ball, goalpost, grass, ground, crowd, etc. Along with these object classes, the image processing engine 610 may output an explanation of the reasoning behind the class selection. This explanation may show contextual inference, indicating each patch and region of the video frame in which the object class was found to exist, the location of the object class represented by pixels in such frames, and a mapping between such regions and corresponding regions of learned prototypes in training images.

[0057] The classification results 612 may be further processed for object identification and scene analysis 614. For example, according to the examples of some embodiments, the indicated object classes and their locations and regions may be processed for the identification of the underlying objects. In this regard, the system may generate the names of players identified in the frame, the positions of players identified in the frame relative to the ground, and their spatial relationships with respect to each other and the soccer ball and / or goalposts. Contextual analysis of scenes captured in the video may be provided by associating the identification results for a set number of consecutive frames of video 604A. The results of the contextual analysis may be obtained as part of a subprocess of object identification and scene analysis 614 and provided to an AI-based speech generator 616 to generate audio, which is synthesized in 614 using a speech scheme then generated based on the results of the contextual analysis. In this way, audio data corresponding to video 604A may be generated and provided to a video processing system 620, which, in a frame-synchronous manner, overlays the generated audio onto video 604A and outputs the overlaid video to a broadcast system 622 for output or storage as needed. In this way, some embodiments of the present invention may be useful for generating online commentary for sports, events, or for describing scenes.

[0058] Figure 6B shows another specific use case for detecting anomalies in a scene using an AI-based image processing engine 660, according to some examples of embodiments. The image processing engine 660 is supplied with input video provided from a suitable imaging source 602B. In some embodiments, the input video may be live video, such as video 604B of a street sidewalk captured by a closed-circuit television camera, and may be provided directly to the image processing engine 660. In some embodiments, the video may be pre-captured and stored in a database, such as the input image database 140 in Figure 1A.

[0059] Regardless of the video source, the image processing engine 660 processes the video frame by frame by dividing the video into individual frames or images, applying any appropriate object detector algorithm to discover image patches containing objects, each of which is processed according to the workflow and process described with reference to Figures 1B to 5B to generate multiple classification results corresponding to video 604B. According to the example of some embodiments, the classification results 662 may include one or more object classes corresponding to objects in the video, and their explainability data that provides the basis for the validity of the corresponding classification. For example, the classification results 662 may include identification of object classes such as humans, animals, vehicles, structures, plants, sex, human and vehicle traffic categories, and structure categories. Along with these object classes, the image processing engine 660 may output an explanation of the reasoning behind the class selection. This explanation may show contextual inference, indicating each patch and region of the video frame in which the object class was found to exist, the location of the object class represented by pixels in such frames, and a mapping between such regions and corresponding regions of learned prototypes in the training images.

[0060] The classification results 662 may be further processed for object identification and scene analysis 664. For example, according to the examples of some embodiments, the indicated object classes as well as their locations and regions may be processed for the identification of the underlying objects. In this regard, the system may identify objects and subjects in a frame as humans >> pedestrians / cyclists / drivers >> pedestrians / disabled persons, gender and age, structures >> roads / sidewalks / buildings / trees, etc., and the relative positions of objects to each other. Contextual analysis of scenes captured in the video may be provided by associating the identification results for a set number of consecutive frames of the video 604B. The results of the contextual analysis may be obtained as part of a subprocess of object identification and scene analysis 664 and may be provided to an AI-based anomaly detector 666 to identify anomalies in the video based on the results of the contextual analysis in 664.

[0061] For example, the image processing engine 660 may provide object classes such as humans, pedestrians, non-pedestrians, lost animals, pet animals, disabled persons, prohibited persons, roads, sidewalks, and retail stores / specialty shops, along with the regions within the frame in which objects of these classes are found. The object identification and scene analysis module 664 may further refine the object classes, identify them within the frame, and build contextual relationships between the identified objects based on the classification results and their explainability. The anomaly detector 666 uses appropriate techniques to detect anomalies in one or more frames of the video 604B. For example, the anomaly detector 666 may detect an anomaly if a prohibited entity or activity is identified on a sidewalk or road. As shown in Figure 6B, the anomaly detector 666 may consider it an anomaly because it has identified a cyclist 654 on the sidewalk 652 of road 650. However, the anomaly detection unit 666 does not consider it an anomaly if a pedestrian 656 is detected on the sidewalk. In this way, AI-assisted anomaly detection may be performed on video 604B, and the detected anomaly may be indicated to a control system 670 that generates an alarm 672 for further action.

[0062] Thus, embodiments of the present invention provide improvements in the field of image processing by providing means for processing image patches more quickly for feature generation. The resulting architecture of the disclosed neural network brings about performance improvements across image processing tasks. Furthermore, by providing explainability using incomplete layers with receptive fields from local regions of the input image rather than the entire image, the examples of embodiments described herein provide a more accurate explanation of the reasoning behind AI-based decision-making.

[0063] Figure 6C shows another specific use case illustrating anomalies detected in a factory using the AI-based image processing engine 680, relating to some examples of embodiments. An imaging device, such as a video camera 602C, may capture an industrial process being carried out in the factory setup. The industrial process may have predetermined setpoints on a timeline in which several actions performed on an object lead to predetermined deliverables. The deliverables may be stored as prototypes for the setpoints, and the AI ​​image processing engine 680 may be trained using these prototypes. The video camera 602C may capture images 604C at each of the setpoints and provide them to the AI-based image processing engine 680, which processes the input images 604C according to the workflow and process described earlier with respect to Figures 1A to 5B and outputs classification results for the input images 604C. In this regard, the AI-based image processing engine 680 may first perform object classification using a full number of layers of the backbone subnetwork to output an accurate classification of the identified objects in the input images 604C. The anomaly detector 686 may examine the object class for a setpoint to determine whether it is identical to a predetermined prototype for that setpoint. If no anomalies are detected, the anomaly detector 686 may communicate with the control system 690 and proceed with the industrial process according to a predetermined workflow. However, if the object classified in 682 does not match a predetermined prototype for that setpoint, the anomaly detector 686 instructs the control system 690 to stop the industrial process or start another subprocess so that the industrial process does not proceed to the next stage. In this case, the control system 690 may reduce the number of active convolutional layers used in the backbone subnetwork of the AI-based image processing engine 680 to less than the total number of layers, and then run the image classification process again with the reduced number of active layers to output better explainability of the classification.The explainability results may be provided to the output device 692 so that the operator can perform a diagnosis according to the reasoning provided behind the classification.

[0064] In this way, examples of embodiments of the present invention may be used to help understand the reasons behind anomalies, and thus may help in the rapid adoption of corrective techniques in such mission-critical processes.

[0065] It is widely acknowledged that a lack of explainability in an AI model can lead to a lack of trust in the model. Therefore, some embodiments of the present invention reflect the recognition that AI explainability can help reduce the opacity of the decision-making process in a black-box ML model in a way that is understandable to humans. As used herein, less opacity with respect to the decision-making process of a black-box ML model broadly means providing sufficient visibility into how a particular decision was made, the factors contributing to the decision, and the respective effects on the decision, so that the user can understand how and why the decision was made. Some embodiments of the present invention reflect the recognition that the range or degree of detail that may need to be provided can vary depending on the user's specific needs, the complexity of the decision, the context of the decision, or a combination thereof. Therefore, the degree of such visibility, and the way it is generated and provided, is a matter of design choice. Thus, the accuracy and interpretability of AI-based decision-making, which are influenced by tunable parameters (the choice of the number of layers), are also tunable and can be adjusted according to desired needs. Thus, the examples of embodiments provide scalability and flexibility in the adoption and integration of such AI-based models into various application areas and tasks, which standard / conventional models have not been able to achieve.

[0066] The various devices, systems, modules, and units described with reference to examples of several embodiments may be implemented using some or all components of a computing system. One such computing system is disclosed with reference to Figure 7, which shows a block diagram of a system for implementing some components of an AI-based image processing system according to examples of several embodiments.

[0067] Figure 7 shows a block diagram of a system 700 for implementing some components of the present invention, according to examples of several embodiments. Component 711 includes a processor 740, computer-readable memory 712, storage 758, and a user interface 749 having an optional display device 752 and a keyboard 751, all connected via a bus 756. For example, the user interface 749, communicating with the processor 740 and the computer-readable memory 712, receives user input from the surface of the keyboard 753 of the user interface 757, retrieves the data, and stores it in the computer-readable memory 712.

[0068] Component 711 may include a power supply 754, which may optionally be located outside component 711, depending on the application. A user input interface 757 adapted to connect to a display device 748 may, among other things, include a computer monitor, camera, television, projector, or mobile device via bus 756. A network interface controller (NIC) 734 adapted to connect to a network 736 via bus 756 may, among other things, render image data or other data onto a third-party display device, third-party imaging device, and / or third-party printing device outside component 711.

[0069] Continuing with reference to Figure 7, in particular, electronic data can be transmitted through the communication channels of network 736 and / or stored in the storage system 758 for storage and / or further processing. Furthermore, time-series data or other data can be received wirelessly or via wire from receiver 746 (or external receiver 738) or transmitted wirelessly or via wire through transmitter 747 (or external transmitter 739), both of which are connected via bus 756. Component 711 can be connected to an external sensing device 744 and an external input / output device 741 via input interface 708. For example, the external sensing device 704 may include a sensor that collects data before and after the collected time-series data of the machine. Component 711 can be connected to another external computer 742. Processed data from processor 740 can be output using output interface 709. Furthermore, the user interface 749, which communicates with the processor 740 and the non-temporary computer-readable storage medium 712, receives input from the user from the surface of the user interface 749, acquires the data, and stores it in the non-temporary computer-readable storage medium 712.

[0070] The above description provides only specific embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of specific embodiments will provide a description that enables the realization of one or more specific embodiments for those skilled in the art. Various modifications are intended to be made to the function and configuration of the elements without departing from the spirit and scope of the subject matter disclosed in the appended claims.

[0071] Specific details are provided in the following description to ensure a full understanding of the embodiments. However, those skilled in the art will understand that embodiments can be performed without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in the form of block diagrams to avoid obscuring the embodiments with unnecessary details. In other examples, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. Furthermore, similar reference numbers and names in various drawings refer to similar elements. Also, individual embodiments may be described as processes shown as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may terminate when its operations are complete, but it may have additional steps that are not discussed or included in the diagrams. Furthermore, not all operations in any specifically described process may occur in all embodiments. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, the termination of the function may correspond to returning the function to the calling function or the main function.

[0072] Furthermore, embodiments of the disclosed subject matter may be implemented manually or automatically, at least in part. Manual or automatic implementation may be performed, or at least assisted, through a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. If implemented in software, firmware, middleware, or microcode, program code or code segments for performing the required tasks may be stored in machine-readable media. A processor(s) may perform the required tasks. The various methods or processes outlined herein may be encoded as software executable on one or more processors employing any one of various operating systems or platforms. In addition, such software may be written using any of several suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functions of program modules may be combined or distributed as desired in the various embodiments.

[0073] Embodiments of this disclosure may be implemented as methods, and an example thereof is provided. The order of operations performed as part of this method may be determined in any suitable manner. Thus, embodiments may be configured such that operations are performed in an order different from the order illustrated, which may include performing some operations simultaneously, even if they are shown as a series of operations in the illustrated embodiments. Furthermore, in a claim, terms indicating order, such as “first,” “second,” etc., that modify an element of a claim do not themselves imply superiority, precedence, or order of one element of the claim over another, or the temporal order in which the operations of the method are performed, but are merely used as labels to distinguish the elements of the claim, to distinguish one element of the claim having a particular name from another element having the same name (when no terms indicating order are used). While this disclosure has been described with reference to several preferred embodiments, it should be understood that various other adaptations and modifications can be made within the spirit and scope of this disclosure. Thus, it is the nature of the appended claims to cover all such variations and modifications that fall within the true spirit and scope of this disclosure.

Claims

1. An artificial intelligence (AI) image processing system for interpretable classification of input images, wherein the AI ​​image processing system comprises a processor and a memory storing instructions, and when the instructions are executed by the processor, the AI ​​image processing system performs the following: The system performs the following actions: classifying an input image using a prototype sub-neural network including a backbone subnetwork, a prototype subnetwork, and a readout subnetwork, and generating an interpretable classification of the input image, which includes either or a combination of the classification result of the input image and an interpretation of the classification result. The backbone subnetwork is trained using machine learning to generate feature embeddings that represent features extracted from pixels in different regions of the input image by processing the input image using an incomplete sequence of active convolutional layers, the complete sequence of active convolutional layers causing each of the feature embeddings to contain features derived from pixels of the entire input image. The prototype subnetwork is trained to compare the feature embedding with the prototype feature embedding and generate a comparison result. The readout subnetwork is configured to analyze the results of the comparison to generate the interpretable classification of the input image, and the instruction further sends to the AI ​​image processing system: An AI image processing system that performs the operation of outputting the interpretable classification of the input image.

2. The AI ​​image processing system according to claim 1, wherein the processor is configured to iteratively classify the input image by the prototype partial neural network using a different number of the active convolutional layers in each iteration.

3. The AI ​​image processing system according to claim 2, wherein the processor is further configured to accept the number of active convolutional layers as a parameter.

4. The AI ​​image processing system according to claim 1, wherein the backbone subnetwork includes a plurality of core layers and an add-on layer, the add-on layer modifies the number of channels in the output provided by the plurality of core layers.

5. The AI ​​image processing system according to claim 4, wherein the add-on layer includes a first layer having a normalized linear unit (ReLU) activation function and a second layer having a sigmoid activation function.

6. The AI ​​image processing system according to claim 1, wherein the prototype subnetwork generates a plurality of feature vectors corresponding to the features extracted from pixels in different regions of the input image, calculates a pairwise distance between the feature vectors and the prototype feature embedding, and determines the closest embedded patch for each prototype based on the pairwise distance between the feature vectors and the prototype feature embedding.

7. The AI ​​image processing system according to claim 1, wherein the readout subnetwork includes a fully connected layer that assigns positive weights to prototype units of the same class and negative weights to prototype units of no class among the results of the comparison.

8. The AI ​​image processing system according to claim 1, wherein the classification result includes at least one object class present in the input image, and the interpretation of the classification result is defined with respect to an image region from a prototype portion of the at least one object class that best fits the input image.

9. The AI ​​image processing system according to claim 8, wherein the prototype portion for the at least one object class that best fits the input image represents the reason why it is associated with the classification decision of the prototype portion neural network for classifying the input image into the at least one object class.

10. The AI ​​image processing system according to claim 8, wherein the prototype portion corresponds to a training image associated with the at least one object class.

11. The aforementioned processor further, From the aforementioned memory, a reference image divided into multiple reference patches is fetched, The image region from the prototype portion is compared with the reference patch to determine whether each of the image regions is similar to at least one of the reference patches. If at least one of the aforementioned image regions is not similar to all of the aforementioned reference patches, an anomaly will be declared. The AI ​​image processing system according to claim 10, configured to perform the following actions.

12. An image processing method for interpretable classification of an input image, wherein the image processing method is: This includes classifying an input image using a prototype sub-neural network that includes a backbone subnetwork, a prototype subnetwork, and a readout subnetwork, and generating an interpretable classification of the input image, which includes either or a combination of the classification result of the input image and an interpretation of the classification result. The backbone subnetwork is trained using machine learning to generate feature embeddings that represent features extracted from pixels in different regions of the input image by processing the input image with an incomplete sequence of active convolutional layers, the complete sequence of active convolutional layers causing each of the feature embeddings to include features derived from pixels of the entire input image. The prototype subnetwork is trained to compare the feature embedding with the prototype feature embedding and generate a comparison result. The readout subnetwork is configured to analyze the results of the comparison to generate the interpretable classification of the input image, and the image processing method further, An image processing method comprising outputting the interpretable classification of the input image.

13. The image processing method according to claim 12, wherein the processor is configured to iteratively classify the input image using the prototype partial neural network with a different number of active convolutional layers in each iteration.

14. The image processing method according to claim 13, wherein the processor is further configured to accept the number of active convolutional layers as a parameter.

15. The image processing method according to claim 12, wherein the backbone subnetwork includes a plurality of core layers and an add-on layer, and the add-on layer modifies the number of channels in the output provided by the plurality of core layers.

16. The image processing method according to claim 12, wherein the prototype subnetwork generates a plurality of feature vectors corresponding to the features extracted from pixels in different regions of the input image, calculates a pairwise distance between the feature vectors and the prototype feature embedding, and determines the closest embedded patch for each prototype based on the pairwise distance between the feature vectors and the prototype feature embedding.

17. The readout sub-network according to claim 12, wherein the readout sub-network includes a fully-connected layer that assigns positive weights to prototype units of the same class and negative weights to prototype units without a class among the results of the comparison.

18. The classification result includes at least one object class present in the input image, and the interpretation of the classification result is defined with respect to an image region from a prototype portion for the at least one object class that best fits the input image. The image processing method according to claim 12.

19. The prototype portion for the at least one object class that best fits the input image represents a reason associated with a classification determination of the prototype portion neural network for classifying the input image into the at least one object class. The image processing method according to claim 18.

20. Fetching a reference image divided into a plurality of reference patches from a memory; Comparing the image region from the prototype portion with the reference patches to determine whether each of the image regions is similar to at least one of the reference patches; Declaring an abnormality when at least one of the image regions is not similar to all of the reference patches. The image processing method according to claim 12, further comprising.

Images