Program, information processing system and information processing method

Abstract

To visualize the effect level given to an output by an evaluation model.SOLUTION: A program causes a computer to accomplish: an image obtaining unit that obtains image information subjected to processing; a pixel extracting unit that extracts, based on extraction setting information relating to a setting for extracting multiple pixels from pixels in the image information, a subset pixel from the pixels in the image information; and a calculating unit that calculates, based on the subset pixel and an evaluation model that associates a particular region in the image information with a first class, an effect level score relating to an effect level to a score output by the evaluation model in one subjected pixel, the effect level score being calculated in accordance with the score output by the evaluation model relating to a pixel group including the one subjected pixel in the image information and the subset pixel for each pixel in the image information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, an information processing system, and an information processing method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for analyzing an image and detecting a specific object contained in the image.

[0003] For example, the image analysis device described in Patent Document 1 includes a detection means for detecting an object from an image using a plurality of different object models, and a display control means for causing a display to display a screen showing information representing each of the plurality of different object models in association with the detection results related to the object models. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP2019-139588A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the object model in the image analysis device described in Patent Document 1 is an evaluation model used when detecting an object. However, the image analysis device described in Patent Document 1 merely displays the result of object detection on a screen. Therefore, the user cannot grasp to what extent which part of the image has influenced the output of the evaluation model.

[0006] Therefore, an object of the present invention is to provide a program, an information processing system, and an information processing method that are capable of visualizing the degree of influence on the output of an evaluation model. [Means for solving the problem]

[0007] A program according to one embodiment of the present invention causes a computer to include an image acquisition unit that acquires image information to be analyzed, a pixel extraction unit that extracts subset pixels from the pixels of the image information based on extraction setting information related to settings for extracting multiple pixels from the pixels of the image information, and a calculation unit that calculates, for each pixel of the image information, an influence score related to the influence of one target pixel on the score output by the evaluation model for a group of pixels including one target pixel and the subset pixels, based on the subset pixels and an evaluation model that associates a first class with a specific area included in the image information. The influence score is calculated in accordance with the score output by the evaluation model for one target pixel of the image information. Effect of the Invention

[0008] According to the present invention, it is possible to visualize the degree of influence of each pixel or region on the output of an evaluation model. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a configuration of a score calculation system 100 according to an embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram showing the relationship between a target pixel and a subset pixel. [Figure 3A] FIG. 13 is a diagram illustrating an example of object detection using an evaluation model. [Figure 3B] FIG. 2 illustrates an example of a saliency map. [Figure 4A] FIG. 13 is a diagram showing an example of an error in object detection by an evaluation model. [Figure 4B] FIG. 2 illustrates an example of a saliency map. [Figure 5A] FIG. 2 illustrates an example of a saliency map. [Figure 5B] FIG. 13 is a diagram illustrating an example of improvement of object detection using an evaluation model. [Figure 6] 1 is a flowchart showing an example of processing in the score calculation system 100. [Figure 7] 1 is a flowchart showing an example of processing in the score calculation system 100. [Figure 8] 1 is a flowchart showing an example of processing in the score calculation system 100. [Figure 9] FIG. 9 illustrates an example of the hardware configuration of a computer 900. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A preferred embodiment of the present invention will now be described with reference to the accompanying drawings.

[0011] The score calculation system 100 is an information processing system realized by a program, and is an information processing system that calculates an influence score regarding the influence of a target pixel included in the image information relative to a score output by an evaluation model that associates a first class with a specific area included in the image information.

[0012] Conventionally, an artificial intelligence technology using an evaluation model that associates a class (for example, a first class described later) with a specific area included in image information is known. The artificial intelligence technology can associate a first class (for example, "dog") with a specific area included in image information (for example, an area showing a dog) using the evaluation model. However, the artificial intelligence technology has a problem in that it is difficult for humans to fully understand the evaluation algorithm and output results of the artificial intelligence technology due to the parameters of the evaluation model being a black box. Therefore, in recent years, so-called explainable AI (eXplainable Artificial Intelligence (XAI)) has been attracting attention. The score calculation system 100 is an information processing system related to XAI.

[0013] Here, the evaluation model is a model that associates a specific area included in the image information with the first class. In other words, the evaluation model is a model that detects an area included in the image information that corresponds to the first class.

[0014] The evaluation model may be, for example, an object detection model. In this case, the class may be a class corresponding to a specific object detected by the object detection model, more specifically, a class indicating the object. The object detection model is, for example, a model that associates a first class (for example, "dog") corresponding to a specific object with a specific area included in the image information, and detects an object corresponding to the first class from the image information. Note that, in the following, an object detection model will be described as an example of the evaluation model, but the evaluation model is not limited to this.

[0015] 1 is a diagram showing a configuration of a score calculation system 100 according to an embodiment of the present invention. The score calculation system 100 is communicably connected to a user device 200 via a network such as the Internet. Details of the score calculation system 100 will be described later.

[0016] The user device 200 is a computer used by a user, such as a smartphone, a tablet terminal, a personal computer, etc. The user accesses the score calculation system 100 using the user device 200.

[0017] The user device 200 provides, for example, image information to the score calculation system 100. In addition, the user device 200 acquires output results (for example, an influence score, a saliency map, mask information, and a class detection result, which will be described later) generated in the score calculation system 100 based on the image information.

[0018] Although only one user device 200 is shown in FIG. 1, a plurality of user devices 200 may be provided.

[0019] Next, details of the score calculation system 100 will be described. The score calculation system 100 includes a storage unit 110, an image acquisition unit 120, a pixel extraction unit 130, a calculation unit 140, a map generation unit 150, a map output unit 160, a subset identification unit 170, a mask generation unit 180, a mask output unit 190, and a class detection unit 195. Each unit shown in Fig. 1 can be realized, for example, by using a storage area or by a processor executing a program stored in the storage area.

[0020] The storage unit 110 stores information processed in the score calculation system 100. The storage unit 110 can store, for example, image information, extraction setting information, influence score, saliency map information, and mask information, which will be described later.

[0021] The image acquisition unit 120 acquires image information to be processed, and stores the image information in the storage unit 110 .

[0022] The image information is, for example, information in which a numerical value corresponding to a color is associated with each pixel.

[0023] The image information may be a photograph, an illustration, a painting, or even a single frame of a video.

[0024] The pixel extractor 130 extracts a subset of pixels made up of a portion of the pixels included in the image information, based on extraction setting information relating to settings for extracting a plurality of pixels from the pixels (for example, all pixels) of the image information.

[0025] Here, the subset pixels are pixels that are made up of a portion of the pixels included in the image information, that is, the subset pixels can be said to be a subset of the pixels included in the image information.

[0026] The extraction setting information may include layer information regarding layers when extracting a plurality of pixels from pixels of the image information. The layer information may be, for example, an interval of a ratio of the subset pixels extracted by the pixel extraction unit 130 to the pixels of the image information (for example, all pixels). The layer information may be information displayed as a percentage, such as "20%". That is, when the layer information is "20%", the pixel extraction unit 130 can extract subset pixels including 20%, 40%, 60%, and 80% of the pixels of the image information. In other words, in this case, the pixel extraction unit 130 extracts subset pixels of four layers. Note that the pixel extraction unit 130 may extract one or more subset pixels for each layer.

[0027] The layer information may also be information indicating the number of layers, such as "4." That is, when the layer information is "20%," the pixel extraction unit 130 can extract subset pixels of four layers that include 20%, 40%, 60%, and 80% of the pixels of the image information.

[0028] In this way, the pixel extraction unit 130 can extract a plurality of subset pixels having different ratios of the number of subset pixels to the number of pixels of the image information based on the layer information. In other words, the pixel extraction unit 130 can extract subset pixels corresponding to a plurality of layers. This allows the score calculation system 100 to reduce the amount of calculation of the influence score and calculate an approximation of the Shapley value, as shown in Equation (2) described later.

[0029] The extraction setting information may be set by an administrator of the score calculation system 100 before the extraction process by the pixel extraction unit 130, for example, or may be set according to an operation received from the user device 200.

[0030] The extraction setting information may be set according to the first class associated with the evaluation model. Specifically, for example, when the first class is "person" (i.e., when the evaluation model detects "person"), a predetermined first extraction setting information may be set, and when the first class is "dog" (i.e., when the evaluation model detects "dog"), a predetermined second extraction setting information may be set. This allows the score calculation system to set appropriate extraction setting information according to the first class, and to calculate a more accurate impact score.

[0031] The pixel extractor 130 randomly extracts subset pixels from the pixels of the image information based on the extraction setting information. At this time, the pixel extractor 130 extracts subset pixels that do not include the target pixel.

[0032] The pixel extraction unit 130 extracts a plurality of subset pixels. For example, when the extraction setting information is “20%”, the pixel extraction unit 130 extracts a plurality of subset pixels each consisting of a set of pixels different from each other, each including 20% ​​of the pixels (e.g., all pixels) of the image information.

[0033] The calculation unit 140 calculates an influence score for each pixel of the image information, which relates to the influence on the score output by the evaluation model for one target pixel, based on the subset pixels and an evaluation model that associates a first class with a specific region included in the image information, and is calculated in accordance with the score output by the evaluation model for one target pixel of the image information and a group of pixels including the subset pixels, and stores the calculated influence scores in the memory unit 110.

[0034] The calculation section 140 calculates the influence score for each pixel (target pixel) included in the image information.

[0035] The calculation unit may calculate an influence score related to the influence on the score output by the evaluation model for one target pixel, the influence score being calculated according to the difference between the score output by the evaluation model for a pixel group including one target pixel and the subset pixel of the image information and the score output by the evaluation model for the subset pixel, based on the subset pixel and the evaluation model that associates the first class with a specific region included in the image information. In this case, the calculation unit 140 first inputs the target pixel and the subset pixel to the evaluation model to obtain a score (first score). Next, the calculation unit 140 inputs the subset pixel to the evaluation model to obtain a score (second score). Then, the calculation unit 140 calculates the influence score according to the difference between the first score and the second score.

[0036] Since the influence score is a value according to the difference between the first score and the second score, the influence score can be considered as the influence of the target pixel on the score output by the evaluation model.

[0037] The influence score calculated by the calculation unit 140 includes a positive value and a negative value.

[0038] That is, for example, a higher positive value of the influence score indicates that the evaluation model has a more positive impact on the evaluation model when outputting the evaluation model (e.g., detecting an object corresponding to the first class). Here, a positive impact is an impact that makes it easier for the evaluation model to detect an object corresponding to the first class. On the other hand, a lower negative value of the influence score indicates that the evaluation model has a more negative impact on the evaluation model when outputting the evaluation model (e.g., detecting an object corresponding to the first class). Here, an area with an influence score that has a negative impact is an area that the evaluation model perceives as, for example, not likely to be an object corresponding to the first class.

[0039] Furthermore, for example, the closer the influence score is to 0, the less the output of the evaluation model (for example, detection of an object corresponding to the first class) has a positive or negative influence on the evaluation model.

[0040] In this way, the score calculation system 100 can function as an XAI that explains the evaluation algorithm and output result of the evaluation model by calculating the influence score for each target pixel. In addition, since the influence score includes positive and negative values, the score calculation system 100 can identify target pixels that have a positive or negative influence on the output result of the evaluation model.

[0041] The calculation unit 140 can calculate the Shapley value for each target pixel as the influence score. i is expressed, for example, by the following formula (1).

[0042]

number

[0043] Here, i indicates the target pixel. N f denotes pixels of image information (e.g., all pixels); S denotes the subset pixels excluding the target pixel i; and v(x) denotes the output score of the evaluation model for pixel group x. Thus, v(S∪i) in equation (1) is the first score obtained by inputting the target pixel and the subset pixels into the evaluation model, and v(S) in equation (1) is the second score obtained by inputting the subset pixels into the evaluation model.

[0044] FIG. 2 is a diagram showing the relationship between the target pixel and the subset pixels.

[0045] N findicates pixels of the image information (e.g., all pixels). The target pixel i is one pixel included in the image information. The subset pixel S is a collection of multiple pixels excluding the target pixel i. Note that the subset pixel S may be a collection of adjacent pixels as shown in FIG. 2, or may be a collection of pixels in which at least some of the pixels are not adjacent to each other, so to speak, a collection of pixels that are worm-eaten.

[0046] As shown in equation (1), the influence score is a value corresponding to the difference between the first score and the second score, and more specifically, is a value obtained by weighting the first score and the second score according to the number of pixels (e.g., all pixels) and subset pixels of the image information, and then adding them up for all subset pixels.

[0047] When the pixel extraction unit 130 extracts subset pixels of all combinations without using the extraction setting information, the Shapley value s i The computational complexity of Nf ), which is an unrealistic amount of calculation. Therefore, the score calculation system 100 (particularly the pixel extraction unit 130) extracts a subset pixel based on the extraction setting information, thereby obtaining the Shapley value s i In addition, the Shapley value s i We succeeded in making the computation time O(N). In this case, the Shapley value s i The approximation of the Shapley value s is expressed by, for example, the following formula (2). i An approximation of can be calculated.

[0048]

number

[0049] Here, K is the number of layers indicated by the extraction setting information, particularly the layer information. The pixel extraction unit 130 extracts subset pixels of K layers based on the extraction setting information. For example, when K is "4", the pixel extraction unit 130 extracts subset pixels of four layers consisting of 20%, 40%, 60%, and 80% of the pixels of the image information.

[0050] Also, M k denotes pixels to be masked based on the subset pixels corresponding to a layer k among the K layers, and X denotes pixels of image information. k Denote the Hadamard product of and X.

[0051]

number

[0052] M k (i) indicates the state of the mask at the target pixel i; for example, "0" indicates that the pixel is ignored, and "1" indicates that the pixel is included in the calculation.

[0053] Moreover, f is a more specific rewriting of v(S) in formula (1) as a function that outputs a value related to an evaluation model that receives an image as an input. The value calculated by f is, for example, a value that quantitatively indicates the degree of similarity between a detection result outputted by inputting an image to be evaluated into the evaluation model and a detection result outputted by inputting an image of the class to be detected (so to speak, a ground truth image) into the evaluation model. The value calculated by f can be appropriately defined by the user of the score calculation system 100.

[0054] Furthermore, Z is a predetermined coefficient, for example, a normalization coefficient, and may be, for example, a hyperparameter of granularity when generating a mask (for example, a value of 32 pixels by 32 pixels, etc.).

[0055] In addition, the upper line (bar) in formula (2) indicates the average value obtained from N samples by Monte Carlo sampling. For example, formula (4) indicates the M k The average value of (i) is shown.

[0056]

number

[0057] In this way, the calculation section 140 can calculate an approximation of the Shapley value for each target pixel as the influence score, based on a plurality of subset pixels corresponding to a layer.

[0058] The map generating unit 150 generates a saliency map regarding the evaluation result for the image information by the evaluation model, based on the influence score calculated for each pixel of the image information, and stores the generated saliency map in the storage unit 110.

[0059] The saliency map is, for example, a heat map that visualizes the influence scores.

[0060] The map output unit 160 outputs the saliency map.

[0061] The map output unit 160 can output the saliency map to, for example, the user device 200. This allows the user to refer to the saliency map and understand, for example, the degree of influence (positive influence and negative influence) that each of the target pixels has on the evaluation model.

[0062] The subset identification unit 170 identifies at least one subset pixel from among the multiple subset pixels based on statistical attributes of influence scores calculated for each pixel of the image information corresponding to each of the multiple subset pixels corresponding to the multiple extraction setting information.

[0063] Depending on the first class, appropriate extraction setting information, particularly, for example, an appropriate number of layers, may differ. Specifically, appropriate extraction setting information when the first class is "person" may differ from appropriate extraction setting information when the first class is "dog". When subset pixels extracted according to inappropriate extraction setting information are used, an appropriate saliency map may not be generated. An inappropriate saliency map is, for example, a map in which pixels having similar influence scores are distributed throughout the entire image and do not form a certain cluster. Therefore, the subset identification unit 170 identifies at least one subset pixel from among the multiple subset pixels based on the statistical attributes of the influence scores.

[0064] In this case, the statistical attribute of the impact score may be an attribute based on a value according to the distribution of the impact scores. The value according to the distribution of the impact scores may be, for example, the mean, median, mode, variance, standard deviation, or quartile of the impact scores.

[0065] The mask generating unit 180 generates mask information for masking pixels having an influence score that satisfies a predetermined condition, and stores the generated mask information in the storage unit 110. Here, the mask information may be, for example, mask information for ignoring a predetermined pixel (for example, a pixel having an influence score that satisfies a predetermined condition), and may be, for example, mask information for setting the mask state of the pixel to "0".

[0066] The mask output unit 190 outputs the mask information.

[0067] The mask output unit 190 can output mask information to, for example, the user device 200. The user device 200 that has received the output of the mask information can use, for example, the mask information to input image information into an evaluation model and evaluate the specific region again.

[0068] Furthermore, the mask output unit 190 can output mask information to a class detection unit 195, which will be described later.

[0069] The class detection unit 195 inputs the image information and mask information into the evaluation model, and detects the specific region as a second class different from the first class.

[0070] The processing by the mask generating unit 180, the mask output unit 190, and the class detecting unit 195 will be described in detail later.

[0071] Next, a specific example of the process of the score calculation system 100 will be described.

[0072] FIG. 3A is a diagram showing an example of object detection by an evaluation model. FIG. 3A shows an example of image information including a "dog" region. The evaluation model receives the input of the image information shown in FIG. 3A and detects a region 301 including the "dog" region as a "dog". However, while a user can understand the result of object detection by the evaluation model by referring to FIG. 3A, the user cannot understand the degree of influence (positive influence and negative influence) that each of the target pixels has on the evaluation model.

[0073] Fig. 3B is a diagram showing an example of a saliency map. The saliency map shown in Fig. 3B is a heat map that visualizes the influence score. Fig. 3B shows a heat map when the evaluation model detects a "dog" from the image information shown in Fig. 3A.

[0074] The heat map shown in Fig. 3B is displayed in color, for example. In this case, the heat map may be displayed in red for positive values, blue for negative values, and colorless for values ​​closer to 0. Note that in Fig. 3B, since the heat map is displayed in grayscale, values ​​closer to 0 are displayed colorless, and values ​​closer to positive or negative are displayed white.

[0075] As shown in FIG. 3B, for example, pixels in the dog's head region 302 and tail region 303 have positive influence scores. This allows the user to understand that the pixels in the head region 302 and tail region 303 have a positive influence on the output result of the evaluation model, and pixels in other regions have influence scores close to 0. This allows the user to understand that the pixels in the other regions do not affect the output result of the evaluation model. In other words, the score calculation system 100 can function as an XAI.

[0076] 4A is a diagram showing an example of an object detection error by the evaluation model. Although the image of a dog is shown in FIG. 4A, the evaluation model erroneously detects it as a "horse."

[0077] FIG. 4B is a diagram showing an example of a saliency map. The saliency map shown in FIG. 4B is a heat map that visualizes the influence score. FIG. 4B shows the heat map when the evaluation model detects a "horse" from the image information shown in FIG. 4A. Note that in FIG. 4B, the heat map is displayed in grayscale, so values ​​closer to 0 are displayed in white, and positive or negative values ​​are displayed in black.

[0078] As shown in FIG. 4B, for example, pixels in the dog's leg region 401 and tail region 402 have positive influence scores. This indicates that the evaluation model has a positive influence on the output result of the evaluation model when it erroneously detects the dog as a "horse." On the other hand, pixels in the dog's head region 403 and body region 404 have negative influence scores. This indicates that the evaluation model considers the head region 403 and body region 404 to be unlikely regions corresponding to a "horse."

[0079] FIG. 5A is a diagram showing an example of a saliency map. The saliency map shown in FIG. 5A is a heat map that visualizes the influence score. FIG. 5A shows a heat map in the case where the evaluation model detects a "dog" from the image information shown in FIG. 4A. As shown in FIG. 4A, the evaluation model is a model that erroneously detects a "horse" from the image information shown in FIG. 4A, but FIG. 5A shows a saliency map in the case where the evaluation model is made to detect a "dog" (in other words, the image is experimentally or forcibly assumed to be an image of a "dog"). In addition, since the heat map is displayed in grayscale in FIG. 5A, the closer to 0, the whiter the color, and the more positive or negative the value, the blacker the color.

[0080] As shown in FIG. 5A, for example, pixels in the leg region 501 and tail region 502 of the dog have a negative influence score. This indicates that the evaluation model has a negative influence on the output result of the evaluation model when detecting a "dog". In other words, it can be said that the evaluation model cannot correctly detect a "dog" from the image information shown in FIG. 4A due to the pixels in the leg region 501 and tail region 502, for example. Also, as shown in FIG. 5A, pixels in the head region 503 and body region 504 of the dog have a positive influence score. This indicates that the evaluation model has a positive influence on the output result of the evaluation model when detecting a "dog".

[0081] Therefore, the mask generation unit 180 generates mask information for masking the regions having the negative influence (in this case, the leg region 501 and the tail region 502), and the mask output unit 190 outputs the mask information. At this time, the mask generation unit 180 may generate mask information for masking pixels having an influence score smaller than a predetermined value, for example. Then, the class detection unit 195 inputs the mask information and image information into the evaluation model, and detects a second class (in this case, "dog") different from the first class (in this case, "horse"). A specific description will be given using the figures.

[0082] FIG. 5B is a diagram showing an example of improvement of object detection by the evaluation model. FIG. 5B is a diagram showing the result of inputting mask information and image information into the evaluation model. As shown in FIG. 5B, a mask region 505 corresponding to a leg region 501 and a mask region 506 corresponding to a tail region 502 are masked. Also, as shown in FIG. 5B, the evaluation model correctly detects the object as a "dog" different from a "horse". Note that the mask generation unit 180 may generate mask information that masks regions that have a positive effect on false detection (for example, the leg region 401 and the tail region 402 in FIG. 4B).

[0083] In this way, a series of processes in the score calculation system 100 allows the user to identify pixels that have a common positive or negative effect on the detection of multiple classes (e.g., "dog" and "horse"), pixels that have a positive or negative effect on the detection of a correct class, and pixels that have a positive or negative effect on the detection of an incorrect class. This allows the user to understand the nature of the evaluation model, and for example, to create a more accurate evaluation model or use the evaluation model in accordance with its nature.

[0084] Furthermore, even if the evaluation model erroneously detects a class due to a series of processes in the score calculation system 100, the user can specify pixels that have a negative effect on the correct detection or pixels that have a positive effect on the erroneous detection, and adjust the pixel to detect the correct class by masking the pixel. That is, the score calculation system 100 can visualize the degree of influence (positive and negative influence) of each target pixel on the evaluation model, and can make the algorithm and output result of the evaluation model fully comprehensible to humans. This allows the user to understand the nature of the evaluation model, and can, for example, create a more accurate evaluation model or use the evaluation model according to its nature.

[0085] Furthermore, the score calculation system 100 can calculate an influence score based on a score obtained by inputting image information into an evaluation model (e.g., an object detection model). This allows a user to visualize the degree of influence (positive and negative influence) of each target pixel on the evaluation model without needing to understand the details of the evaluation model (e.g., the algorithm or parameters of the evaluation model). In other words, a user can use the score calculation system 100 as an XAI that can explain an evaluation model without needing information about the details of the evaluation model.

[0086] In this regard, when using Grad-CAM (Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, pages 618-626, 2017.), which is a prior art related to XAI, the user needs to access the internal parameters of the evaluation model, which makes the XAI unusable. On the other hand, the score calculation system 100 is advantageous in that the evaluation model can be used as an explainable XAI without requiring information about the details of the evaluation model.

[0087] Fig. 6 is a flowchart showing an example of processing in the score calculation system 100. The flowchart shown in Fig. 6 is a flowchart showing an example of processing up to outputting a saliency map.

[0088] The image acquisition unit 120 acquires image information to be processed (S601). The pixel extraction unit 130 extracts subset pixels (S602). The calculation unit 140 calculates an influence score for each target pixel (S603). The map generation unit 150 generates a saliency map based on the influence score, and the map output unit 160 outputs the saliency map (S604).

[0089] Fig. 7 is a flowchart showing an example of processing in the score calculation system 100. The flowchart shown in Fig. 7 is a flowchart showing an example of a subset specifying process.

[0090] The pixel extraction unit 130 extracts a plurality of subset pixels corresponding to a plurality of pieces of extraction setting information (S701). The calculation unit 140 calculates an influence score for each of the plurality of subset pixels (S702). The subset identification unit 170 identifies at least one subset pixel from among the plurality of subset pixels based on the statistical attributes of the influence score (S703).

[0091] Fig. 8 is a flowchart showing an example of processing in the score calculation system 100. The flowchart shown in Fig. 8 is a flowchart showing an example of second class detection processing.

[0092] The mask generating unit 180 generates mask information for masking pixels having influence scores that satisfy a predetermined condition, and the mask output unit 190 outputs the mask information (S801). The class detecting unit 195 inputs the mask information and image information to an evaluation model, and detects a specific region as a second class different from the first class (S802).

[0093] Next, an example of a hardware configuration in the case where the score calculation system 100 is realized by a computer 900 will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of a hardware configuration of the computer 900.

[0094] As shown in FIG. 9, a computer 900 includes, for example, a processor 901, a memory 902, a storage device 903, an input I / F unit 904, a data I / F unit 905, a communication I / F unit 906, and a display device 907.

[0095] Computer 900 may be, for example, a server computer, a personal computer (e.g., a desktop, laptop, tablet, etc.), a media computing platform (e.g., a cable, satellite set-top box, digital video recorder, etc.), a handheld computing device (e.g., a PDA, email client, etc.), or some other type of computing or communications platform.

[0096] The processor 901 is a control unit that controls various processes in the computer 900 by executing programs stored in the memory 902 .

[0097] The memory 902 is a storage medium such as a RAM (Random Access Memory), etc. The memory 902 temporarily stores the program code of the program executed by the processor 901 and data required when the program is executed.

[0098] The storage device 903 is a non-volatile storage medium such as a hard disk drive (HDD), a flash memory, etc. The storage device 903 stores an operating system and various programs for implementing the above-mentioned configurations.

[0099] The input I / F unit 904 is a device for receiving input from a user. The input I / F unit 904 is, for example, a keyboard, a mouse, a touch panel, various sensors, a wearable device, etc. The input I / F unit 904 may be connected to the computer 900 via an interface such as a USB (Universal Serial Bus).

[0100] The data I / F unit 905 is a device for inputting data from outside the computer 900. The data I / F unit 905 is, for example, a drive device for reading data stored in various storage media. The data I / F unit 905 may be provided outside the computer 900. When the data I / F unit 905 is provided outside the computer 900, the data I / F unit 905 is connected to the computer 900 via an interface such as a USB.

[0101] The communication I / F unit 906 is a device for performing data communication via a network such as the Internet, either wired or wirelessly, with an external device of the computer 900. The communication I / F unit 906 may be provided outside the computer 900. When the communication I / F unit 906 is provided outside the computer 900, the communication I / F unit 906 is connected to the computer 900 via an interface such as a USB.

[0102] The display device 907 is a device for displaying various types of information. The display device 907 is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, or the like. The display device 907 may be provided outside the computer 900. When the display device 907 is provided outside the computer 900, the display device 907 is connected to the computer 900 via, for example, a display cable. In addition, when a touch panel is adopted as the input I / F unit 904, the display device 907 may be configured integrally with the input I / F unit 904.

[0103] An embodiment of the present invention has been described above. The score calculation system 100 can acquire image information, extract subset pixels based on extraction setting information, and calculate an influence score for each pixel of the image information. This allows the score calculation system 100 to visualize the influence on the output of the evaluation model, and the user can understand which part of the image affects the output of the evaluation model.

[0104] In addition, the score calculation system 100 can extract a plurality of subset pixels based on the layer information and calculate the influence score based on the plurality of subset pixels. This allows the score calculation system 100 to calculate the influence score with a reduced amount of calculation.

[0105] Furthermore, the score calculation system 100 can calculate an approximation of the Shapley value for each pixel of the image information as the influence score. This allows the score calculation system 100 to calculate the influence score with a reduced amount of calculation.

[0106] Furthermore, the score calculation system 100 can output a saliency map based on the influence score, which allows the score calculation system 100 to visualize the influence on the output of the evaluation model.

[0107] In addition, the score calculation system 100 can calculate influence scores including positive and negative values ​​and output a saliency map based on the influence scores including positive and negative values, allowing a user to understand the positive and negative influences that pixels of an image have on the output of an evaluation model.

[0108] In addition, the score calculation system 100 can output mask information for masking pixels having influence scores that satisfy a predetermined condition. This allows a user to input image information masked using the mask information to an evaluation model and understand the properties of the evaluation model.

[0109] Furthermore, the score calculation system 100 can input image information and mask information to the evaluation model to detect a specific region as a second class different from the first class. This allows the user to obtain a correct class detection result by using the mask information even if the class detection by the evaluation model is incorrect.

[0110] Furthermore, the score calculation system 100 can identify at least one subset pixel among the plurality of subset pixels based on statistical attributes of the influence scores corresponding to each of the plurality of subset pixels corresponding to the plurality of extraction setting information. This allows the score calculation system 100 to visualize the influence on the output of the evaluation model based on appropriate extraction setting information according to the first class.

[0111] It should be noted that the present embodiment is provided to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

[0112] In the present invention, a "unit" does not simply mean a physical means, but also includes cases where the functions of the "unit" are realized by software. Furthermore, the functions of one "unit" or device may be realized by two or more physical means, devices, or software, and the functions of two or more "units" or devices may be realized by one physical means, device, or software.

[0113] In addition, this embodiment and the accompanying drawings use images from the COCO (Common Objects in Context) dataset (Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In CVPR, pages 740-755. 2014). [Explanation of symbols]

[0114] 100 score calculation system, 110 storage unit, 120 image acquisition unit, 130 pixel extraction unit, 140 calculation unit, 150 map generation unit, 160 map output unit, 170 subset identification unit, 180 mask generation unit, 190 mask output unit, 195 class detection unit, 200 user device

Claims

1. On the computer, an image acquisition unit that acquires image information to be processed; a pixel extractor configured to extract a subset of pixels from the pixels of the image information based on extraction setting information related to settings for extracting a plurality of pixels from the pixels of the image information; a calculation unit that calculates, for each pixel of the image information, an influence score related to an influence on a score output by the evaluation model at the one target pixel of the image information and a pixel group including the subset pixels, based on an evaluation model that associates a first class with the subset pixels and a specific region included in the image information; the influence score is calculated according to a score output by the evaluation model with respect to the one target pixel of the image information and a pixel group including the subset pixels; A program to achieve this.

2. the extraction setting information includes layer information regarding a layer when extracting a plurality of pixels from the pixels of the image information; The pixel extraction unit extracts a plurality of subset pixels having different ratios of the number of subset pixels to the number of pixels of the image information based on the layer information, The calculation unit calculates the influence score based on the plurality of subset pixels. The program according to claim 1.

3. The program according to claim 1 , wherein the calculation unit calculates an approximation of a Shapley value for each pixel of the image information as the influence score.

4. The program according to claim 1 or 2, further causing the computer to realize a map output unit that outputs a saliency map regarding the evaluation result for the image information by the evaluation model based on the influence score calculated for each pixel of the image information.

5. The impact score includes positive and negative values; The map output unit outputs the saliency map based on the influence score including the positive value and the negative value. The program according to claim 4.

6. The program according to claim 1 or 2, further causing the computer to realize a mask output unit that outputs mask information for masking pixels having the influence score that satisfies a predetermined condition.

7. The program according to claim 6 , further causing the computer to realize a class detection unit that inputs the image information and the mask information to the evaluation model and detects the specific region as a second class different from the first class.

8. The pixel extraction unit extracts a plurality of subset pixels corresponding to a plurality of extraction setting information; The calculation unit calculates the influence score for each of the pixels of the image information for each of the plurality of subset pixels; The program further causes the computer to realize a subset identification unit that identifies at least one subset pixel among the plurality of subset pixels based on a statistical attribute of the influence score calculated for each pixel of the image information corresponding to each of the plurality of subset pixels.

3. The program according to claim 1 or 2.

9. an image acquisition unit that acquires image information to be processed; a pixel extractor configured to extract a subset of pixels from the pixels of the image information based on extraction setting information related to settings for extracting a plurality of pixels from the pixels of the image information; a calculation unit that calculates, for each pixel of the image information, an influence score related to an influence on a score output by the evaluation model at the one target pixel of the image information and a pixel group including the subset pixels, based on an evaluation model that associates a first class with the subset pixels and a specific region included in the image information; the influence score is calculated according to a score output by the evaluation model with respect to the one target pixel of the image information and a pixel group including the subset pixels; An information processing system comprising:

10. The computer Acquire image information to be processed; Extracting a subset of pixels from the pixels of the image information based on extraction setting information related to settings for extracting a plurality of pixels from the pixels of the image information; Calculating an influence score for each pixel of the image information, the influence score being calculated according to a score output by the evaluation model for a pixel group including the one target pixel of the image information and the subset pixels, based on an evaluation model that associates a first class with the subset pixels and a specific region included in the image information; and Information processing methods.

Images