Image recognition device and image recognition method

Abstract

To improve the accuracy of recognition results for an input image by appropriately correcting an attention map.SOLUTION: An image recognition device comprises: an input unit which inputs an image to an unsupervised learned neural network; a map correction unit which makes corrections according to a correction operation of a user or a prescribed program for the attention map representing a focus region of the input image input by the neural network when the attention map is generated; and an output unit which outputs the generated recognition results when the corrected attention map and the image recognition results based on the image are generated by the neural network.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image recognition device and an image recognition method. [Background technology]

[0002] Patent Document 1 discloses a network model including an attention unit and a recognition unit. The attention unit of Patent Document 1 outputs an attention map indicating the distribution of attention levels on an input image. The recognition unit outputs a recognition result for the input image based on the attention map and a feature map indicating features extracted from the input image. The accuracy of the recognition result can be improved by outputting the recognition result for the input image taking the attention map into consideration. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 220990 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, if the fixation area in the attention map is not set appropriately, the accuracy of the recognition result may deteriorate. For example, when a network model is generated by unsupervised learning, the correct answer is not given to the learning data, so it is not easy to appropriately correct the attention map.

[0005] The present disclosure aims to improve the accuracy of recognition results for input images by appropriately modifying the attention map. [Means for solving the problem]

[0006] An image recognition device according to one embodiment includes an input unit that inputs an image into a neural network that has been trained in an unsupervised manner, a map correction unit that, when an attention map representing the gaze area of ​​the input image is generated by the neural network, corrects the attention map in accordance with a user's correction operation or a predetermined program, and an output unit that, when an image recognition result is generated by the neural network based on the corrected attention map and image, outputs the generated recognition result. Effect of the Invention

[0007] According to the present disclosure, by appropriately modifying the attention map, the accuracy of the recognition result for the input image can be improved. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a configuration diagram of an image recognition device according to this embodiment. [Diagram 2] FIG. 2 is a diagram showing a configuration of a neural network. [Diagram 3] FIG. 3 is a diagram showing the configuration of the attention unit. [Figure 4] FIG. 4 is a flowchart of the process executed by the processing unit. [Diagram 5] FIG. 5 is a flowchart of the model training process. [Figure 6] FIG. 6 is a diagram showing a feature space before the attention map is corrected. [Figure 7] FIG. 7 is a diagram showing the feature amount space after the attention map is corrected. [Figure 8] FIG. 8 is a diagram for explaining image recognition of the harness. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [Description of the embodiments of the present disclosure] First, the contents of the embodiments of the present disclosure will be listed and described.

[0010] [1] An image recognition device according to one embodiment includes an input unit that inputs an image to a neural network that has been trained in an unsupervised manner, a map correction unit that, when an attention map representing a gaze area of ​​the input image is generated by the neural network, corrects the attention map in accordance with a user's correction operation or a predetermined program, and, when an image recognition result is generated by the neural network based on the corrected attention map and image, an output unit that outputs the generated recognition result.

[0011] In the image recognition device described in [1] above, an image is input to a neural network that has been trained without a teacher, an attention map that represents a gaze area of ​​the input image is corrected according to a user's correction instruction or a predetermined program, and an image recognition result is generated and output based on the corrected attention map and image. When a neural network is generated by unsupervised learning, since a correct answer is not given to the learning data, it may be difficult to properly correct the attention map when the gaze area in the attention map is not properly set. In this regard, the attention map can be properly corrected by correcting the attention map according to a user's correction instruction or a predetermined program. Then, the recognition result is output using the attention map corrected in this way, thereby improving the accuracy of the recognition result for the input image.

[0012] [2] In the image recognition device described in [1] above, the neural network may have a feature extraction unit, an attention unit, a synthesis unit, and a recognition unit, the feature extraction unit includes a plurality of convolution layers and generates a feature map indicating image features by propagating image information through the plurality of convolution layers, the attention unit generates the attention map based on the feature map, the synthesis unit generates a synthesis map by synthesizing the feature map and the corrected attention map, and the recognition unit generates a recognition result based on the synthesis map. With this configuration, it is possible to appropriately generate a recognition result using the attention map.

[0013] [3] In the image recognition device described in [2] above, the map correction unit may group the features shown in the feature map, identify a group containing fewer feature values ​​than a predetermined value, and identify the feature values ​​of the group as feature values ​​to be corrected. The map correction unit may output the feature values ​​corresponding to the feature values ​​to be corrected in the attention map as correction candidates, and accept a user correction operation or a correction instruction by a predetermined program. The feature values ​​(correction candidates) in the attention map corresponding to the feature values ​​of the group containing fewer feature values ​​are likely to be feature values ​​that should be corrected (not appropriate as a gaze area). Therefore, by outputting such feature values ​​as correction candidates, the user can be prompted to perform a correction operation or the like for the feature values ​​to be corrected, and the attention map can be appropriately corrected.

[0014] [4] In the image recognition device according to any one of [1] to [3] above, the multiple recognition targets of the image may be of the same color or similar colors. When the multiple recognition targets are of the same color or similar colors, it may be difficult to output a recognition result with high accuracy. Even in the case of multiple recognition targets that are generally difficult to recognize with high accuracy, the image recognition device according to the present disclosure can appropriately correct the attention map, thereby improving the accuracy of the recognition result.

[0015] [5] An image recognition method according to one embodiment is an image recognition method executed by an image recognition device, and includes the steps of: inputting an image into a neural network that has been trained in an unsupervised manner; when an attention map representing the gaze area of ​​the input image is generated by the neural network, modifying the attention map in accordance with a user's modification operation or a predetermined program; and when an image recognition result is generated by the neural network based on the modified attention map and image, outputting the generated recognition result.

[0016] [6] In the image recognition method described in [5] above, the step of making corrections may include grouping each feature shown in a feature map indicating the features of the image, identifying a group containing features less than a predetermined value and identifying the features of the group as features to be corrected, outputting features corresponding to the features to be corrected in an attention map generated based on the feature map as correction candidates, and accepting a user correction operation or a correction instruction by a predetermined program in response to the output of the correction candidates.

[0017] [Details of the embodiment of the present disclosure] Specific examples of the embodiments of the present disclosure will be described below with reference to the drawings. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope of the claims. In the description of the drawings, the same elements are given the same reference numerals, and duplicated descriptions are omitted.

[0018] 1 is a configuration diagram of an image recognition device 1 according to this embodiment. As shown in FIG. 1, the image recognition device 1 includes an operation device 2, a display device 3, a memory 4, and a processing unit 5.

[0019] The operation device 2 is a device that receives an operation from a user and outputs a signal corresponding to the received operation to the processing unit 5. The operation device 2 may be, for example, a mouse, a keyboard, a touch panel, etc. The display device 3 is a device that displays an image to the user.

[0020] The memory 4 includes a RAM, which is a rewritable volatile storage medium, a ROM, which is a non-rewritable non-volatile storage medium, and a flash memory, which is a rewritable non-volatile storage medium. The RAM, ROM, and flash memory are non-transient tangible storage media. The flash memory stores data of the trained neural network 10 in advance.

[0021] The processing unit 5 executes a program (not shown) stored in a ROM or a flash memory, and uses the RAM as a working area during the execution to realize various processes, which will be described later.

[0022] Fig. 2 is a diagram showing the configuration of the neural network 10. As shown in Fig. 2, the neural network 10 is a deep neural network including a feature extraction unit 11, an attention unit 12, a synthesis unit 13, and a recognition unit 14.

[0023] When an input image 51 is input, the neural network 10 generates an attention map 53. The attention map 53 is data expressing a fixation area during inference by the neural network 10. In other words, the attention map 53 is visual explanatory data explaining which area of ​​the input image 51 is emphasized during inference by the neural network 10.

[0024] The neural network 10 outputs a classification result of the input image 51 based on the input image 51 and the attention map 53. The classification result of the input image 51 is a plurality of likelihoods corresponding to a plurality of classes corresponding to the recognition target of the image. For example, when the input image is an image related to a harness 500 (cable harness) as shown in FIG. 8, the plurality of classes corresponding to the recognition target may include a harness 500A (see FIG. 8(a)) in which no tape 510 is wound, a harness 500B (see FIG. 8(b)) having a portion 511 where the tape 510 is wound and a portion 512 where the tape 510 is not wound, and a harness 500C (see FIG. 8(c)) in which the tape 510 is completely wound. In this case, the plurality of recognition targets in the image may be of the same color or similar colors. For example, the tape 510 and the harnesses 500A, 500B, and 500C may all be black. Here, the number of classes is K.

[0025] The feature extraction unit 11 is a neural network having multiple layers. These multiple layers include at least multiple convolution layers. Furthermore, these multiple layers may be components of multiple residual blocks, or may have multiple pooling layers, etc. The feature extraction unit 11 generates a feature map 52 by propagating information of the input image 51 that has been input to these multiple layers.

[0026] The feature map 52 is a map with K resolutions of h×w, each of which corresponds to K classes. h and w are any integers. Therefore, the number of channels of the feature map 52 is K. The resolution of the feature map 52 may be the same as the resolution of the input image 51, or may be lower than the resolution of the input image 51.

[0027] The feature extraction unit 11 may be configured by a portion of the baseline model starting from the input layer and preceding the first fully connected layer. As the baseline model, one having multiple convolutional layers and generating the same types of multiple class likelihoods as the neural network 10 is selected. For example, as the baseline model, VGGNet, ResNet, or another CNN (Convolutional Neural Network) may be used.

[0028] The attention unit 12 generates an attention map 53 from the feature map 52 generated by the feature extraction unit 11. The attention unit 12 is a neural network having multiple layers. FIG. 3 is a diagram showing the configuration of the attention unit 12. As shown in FIG. 3, the multiple layers include a first part 12a having one or more convolution layers or one or more residual blocks, and a K×1×1 convolution layer 12b at the rear of the first part. Here, if L, a, and b are any natural numbers, the L×a×b convolution layer means a convolution layer using an a×b kernel in each of the L channels.

[0029] The attention unit 12 has two K×1×1 convolution layers 12c and a 1×1×1 convolution layer 12d branched out after the convolution layer 12b. The attention unit 12 has a GAP (Global Average Pooling) layer 12e after the convolution layer 12c.

[0030] The information of the feature map 52 input to the attention unit 12 propagates through the first part 12a, the convolution layer 12b, the convolution layer 12c, and the GAP layer 12e, and the output of the GAP layer 12e is input to a Softmax function, whereby likelihoods of multiple classes of the same type as those of the neural network 10 are generated as classification results. The classification results are a type of recognition results.

[0031] In addition, the information of the feature map 52 input to the attention unit 12 is propagated to the first part 12a, the convolution layer 12b, and the convolution layer 12d to generate an attention map 53. By generating the attention map 53 via the convolution layer 12b instead of the fully connected layer, the information of the attention area is propagated to the attention map 53 while remaining localized. In addition, by passing through the 1×1×1 convolution layer 12e, a one-channel attention map 53 is generated as a weighted sum of the attention areas corresponding to all classes. Each value of the kernel of the convolution layer 12d may be all 1 or may be other than that.

[0032] The attention unit 12 is configured so that the resolution of each map of the feature map 52 is the same as the resolution of the attention map 53. In the attention map 53, a relatively high pixel value is given to pixels that correspond to the attention area, and a lower pixel value is given to pixels that do not correspond to the attention area compared to the attention area. The value that each pixel value of the attention map 53 can take may be a binary value or a value of 256 levels. The higher the pixel value of a pixel, the higher the attention degree at the position of that pixel.

[0033] The synthesis unit 13 synthesizes the feature map 52 and the attention map 53. Specifically, the attention map 53 is multiplied by a map of resolution h×w in each of the K channels in the feature map 52. The attention map 53 is multiplied by the map of resolution h×w between pixels at the same position coordinates. The synthesis may be multiplication as described above, addition, or a calculation consisting of a combination of addition and multiplication. A synthesis map 54 is obtained by this synthesis. The number of channels and resolution of the synthesis map 54 are the same as those of the feature map 52.

[0034] The perception unit 14 outputs the likelihood of each class based on the composite map 54. The perception unit 14 is a neural network having multiple layers. The multiple layers include at least multiple convolution layers. The multiple layers also include one or both of a fully connected layer and a GAP layer. Furthermore, the multiple layers may further be components of multiple residual blocks, or may have multiple pooling layers. The perception unit 14 outputs the likelihood of each class as a classification result by propagating the information of the input composite map 54 to the multiple layers. The classification result is also a recognition result. The perception unit 14 may be configured by a portion of the above-mentioned baseline model from the stage immediately following the portion used in the attention unit 12 to the output layer.

[0035] The functions described above as being performed by the neural network 10, the feature extraction unit 11, the attention unit 12, the synthesis unit 13, and the recognition unit 14 are actually realized by the processing unit 5 performing processing in accordance with the structure and parameters of the neural network 10. The feature extraction unit 11, the attention unit 12, and the synthesis unit 13 are trained by unsupervised learning so as to realize the functions described above.

[0036] The image classification process of the processing unit 5 using the thus configured unsupervised trained neural network 10 will be described below.

[0037] Fig. 4 is a flowchart of the process executed by the processing unit 5. When a predetermined condition, such as a user's operation to start execution on the operation device 2, is satisfied, the processing unit 5 starts the process shown in Fig. 4, which is defined in a predetermined program recorded in the memory 4. In this process, the processing unit 5 first reads out the neural network 10 from the memory 4 (step S1).

[0038] Next, the processing unit 5 functions as an input unit that acquires an input image 51 and inputs the input image 51 to the unsupervised trained neural network 10 (step S2). The input image 51 may be an image selected by a user's operation on the operation device 2 from among a plurality of images previously recorded in the memory 4, or may be an image received from another device via a communication network (not shown).

[0039] When an input image 51 is input to the neural network 10, as described above, the feature extraction unit 11 of the neural network 10 generates a feature map 52 and a classification result from the input image 51, and the attention unit 12 generates an attention map 53 from the feature map 52.

[0040] The processing unit 5 acquires the attention map 53 thus generated (step S3). That is, the processing unit 5 copies or moves the attention map 53 generated in the memory 4 by the neural network 10 to another area in the memory 4.

[0041] Next, the processing unit 5 functions as a map correction unit that corrects the acquired (i.e., copy destination or move destination) attention map 53 based on the user's correction operation on the operation device 2 (step S4). As a result, the attention map 53 is corrected based on the user's knowledge.

[0042] Specifically, the processing unit 5 displays the attention map 53 and the pointer before the correction on the display device 3. The pointer is an image that moves within the display range of the attention map 53 displayed on the display device 3 in response to the user's operation on the operation device 2. The user corrects the value of the position range overlapping with the pointer in the displayed attention map 53 by performing a predetermined correction operation (e.g., a delete operation, an add operation, etc.) on the operation device 2. At this time, the processing unit 5 may reflect the correction according to the correction operation on the attention map 53 in a state in which the input image 51 is transparently aligned and superimposed on the attention map 53 and displayed on the display device 3. At this time, if the resolutions of the input image 51 and the attention map 53 are different, the processing unit 5 lowers the resolution of the input image 51 to match the attention map 53, and then transparently superimposes it on the attention map 53.

[0043] In this way, by displaying the input image 51 on the display device 3 while being superimposed on the attention map 53, if the user can determine which part of the input image 51 should be set as the fixation area, the user can efficiently use that knowledge to easily specify the fixation area in the attention map 53. By the processing of step S4, the attention map 53 acquired in step S3 is corrected in the memory 4.

[0044] Next, the processing unit 5 inputs the attention map 53 corrected in the immediately preceding step S4 to the synthesis unit 13 (step S5). The synthesis unit 13 then synthesizes the feature map 52 and the attention map 53 as described above to generate a synthesis map 54, which it inputs to the first layer of the recognition unit 14. The recognition unit 14, to which the synthesis map 54 has been input, generates a classification result based on the synthesis map 54 as described above.

[0045] The processing unit 5 functions as an output unit that acquires and outputs the classification result generated by the recognition unit 14 (step S6). The output destination may be another device via a communication network (not shown), the memory 4, or the display device 3.

[0046] In this way, by correcting the attention map 53 using the user's knowledge, the recognition unit 14 assigns a higher weight to the area intended by the user. As a result, image recognition can be performed according to the user's intention. In other words, the recognition result can be adjusted by using the attention map manually corrected based on the user's knowledge.

[0047] 5 is a flowchart of the model training process executed when correcting the attention map. The model (neural network 10) training process when correcting the attention map will be described with reference to FIG.

[0048] In the model training process, first, an image is input to the neural network 10 (step S101). The image may be the above-mentioned input image 51 or another training image. As a result, the feature extraction unit 11 generates a feature map indicating the features of the image.

[0049] The processing unit 5 functioning as the map correction unit acquires the feature amounts shown in the feature amount map 52 (step S102) and groups the feature amounts shown in the feature amount map 52 (step S103). The feature amount map 52 may be a feature amount map obtained by compressing K channels into two dimensions, or may be a feature amount map of K channels without compression. FIG. 6 is a diagram showing a feature amount space in the feature amount map 52 before the attention map is corrected. As shown in FIG. 6, the processing unit 5 groups the feature amounts shown in the feature amount map 52 based on their mutual similarity. In the example shown in FIG. 6, three groups are specified as groups with many feature amounts and one group is specified as a group with few feature amounts. In this case, the processing unit 5 may specify a group with fewer feature amounts than a predetermined value as a "group with fewer feature amounts". The processing unit 5 specifies each feature amount of the group with fewer feature amounts as a correction target feature amount, and sets the feature amount in the attention map corresponding to the correction target feature amount as a correction candidate (correction target) (step S104). The processing unit 5 outputs a correction candidate (correction target) and accepts a correction instruction by a user's correction operation. Based on the output of the correction candidate (correction target) from the processing unit 5, the user performs, for example, an operation to delete the correction candidate (correction target) on the operation device 2.

[0050] Next, the processing unit 5 judges whether or not the user has performed a correction operation (correction instruction) for the attention map (step S105). If a correction instruction has been received, the processing unit 5 regenerates the attention map in response to the correction instruction from the user (step S106). If there is no correction instruction in step S105, or if the regeneration of the attention map has been completed in step S106, it is judged whether or not the correction process has been completed (step S107), and if completed, retraining is performed using training data (step S108). The training data here includes the attention map after regeneration. Then, when the retraining is completed (step S109), the model training process is completed.

[0051] Fig. 7 is a diagram showing a feature space in the feature map 52 after the attention map has been corrected. As shown in Fig. 7, in the corrected feature map 52, there are no groups that were previously considered to belong to a small number of features, and only groups with many features remain. Features that were included in groups with few features before the correction have been corrected to be included in groups with many features after the correction.

[0052] Next, the effects of the image recognition device 1 according to this embodiment will be described.

[0053] The image recognition device 1 includes a processing unit 5 that functions as an input unit that inputs an image to a neural network 10 that has been trained in an unsupervised manner, a map correction unit that, when an attention map 53 representing the gaze area of ​​the input image is generated by the neural network 10, corrects the attention map 53 in accordance with a user's correction operation, and, when an image recognition result is generated by the neural network 10 based on the corrected attention map 53 and image, outputs the generated recognition result.

[0054] In the image recognition device 1, an image is input to the neural network 10 that has been trained without a teacher, and the attention map 53 that represents the gaze area of ​​the input image is corrected according to a correction instruction from the user, and a recognition result of the image is generated and output based on the corrected attention map 53 and the image. When the neural network 10 is generated by unsupervised learning, since the correct answer is not given to the learning data, it may be difficult to appropriately correct the attention map 53 when the gaze area in the attention map 53 is not appropriately set. In this regard, the attention map 53 can be appropriately corrected by correcting the attention map 53 according to a correction instruction from the user. Then, the recognition result is output using the attention map 53 corrected in this way, and the accuracy of the recognition result for the input image can be improved.

[0055] In the image recognition device 1, the neural network 10 may have a feature extraction unit 11, an attention unit 12, a synthesis unit 13, and a recognition unit 14, the feature extraction unit 11 includes a plurality of convolution layers and generates a feature map 52 indicating image features by propagating image information through the plurality of convolution layers, the attention unit 12 generates an attention map 53 based on the feature map 52, the synthesis unit 13 generates a synthesis map 54 by synthesizing the feature map 52 and the corrected attention map 53, and the recognition unit 14 may generate a recognition result based on the synthesis map 54. With this configuration, it is possible to appropriately generate a recognition result using the attention map 53.

[0056] In the image recognition device 1, the processing unit 5 functioning as a map correction unit may group the features shown in the feature map 52, identify a group containing fewer feature amounts than a predetermined value, and identify the feature amounts of the group as feature amounts to be corrected, output the feature amounts corresponding to the feature amounts to be corrected in the attention map 53 as correction candidates, and accept a correction instruction by a correction operation by the user. The feature amounts (correction candidates) in the attention map 53 corresponding to the feature amounts of the group containing fewer feature amounts are likely to be feature amounts that should be corrected (are not appropriate as a gaze area). Therefore, by outputting such feature amounts as correction candidates, the user can be prompted to perform a correction operation or the like for the feature amounts to be corrected, and the attention map 53 can be appropriately corrected.

[0057] In the image recognition device 1, the multiple recognition targets of the image may be the same or similar colors. In this embodiment, the multiple recognition targets include, for example, a harness 500A with no tape 510 wrapped around it (see FIG. 8(a)), a harness 500B having a portion 511 with tape 510 wrapped around it and a portion 512 with no tape 510 wrapped around it (see FIG. 8(b)), and a harness 500C with tape 510 completely wrapped around it (see FIG. 8(c)). These multiple recognition targets are the same color (for example, the tape 510 and the harnesses 500A, 500B, and 500C are all black). When the multiple recognition targets are the same or similar colors, it may be difficult to output a recognition result with high accuracy. Even with multiple recognition targets that are generally difficult to recognize with high accuracy, the image recognition device 1 according to this embodiment can appropriately correct the attention map 53, thereby improving the accuracy of the recognition result.

[0058] Although various embodiments and modifications according to the present disclosure have been described above, the present disclosure is not limited to the exemplified embodiments.

[0059] For example, the map correction unit has been described as correcting the attention map in response to a correction operation by a user, but is not limited to this. For example, the map correction unit may correct the attention map in response to a predetermined program.

[0060] In addition, although the multiple recognition targets have been described as being the same color or similar colors, this is not limiting. Furthermore, although the multiple recognition targets have been described as being harnesses, this is not limiting, and the recognition targets may be something other than harnesses.

[0061] The various embodiments and modified examples described above may be combined as appropriate without departing from the spirit of the present disclosure. [Explanation of symbols]

[0062] 1. Image recognition device 2...Operating device 3...Display device 4. Memory 5...Processing section (input section, map correction section, output section) 10. Neural Networks 11...Feature extraction unit 12…Attention section 12a…first part 12b, 12c, 12d...convolutional layers 12e…GAP layer 13...Synthesis section 14…Cognitive part 51...Input image 52…Feature map 53…Attention Map 54…Composite map 500, 500A, 500B, 500C…Harness 510…Tape 511…where the tape is wrapped 512...Area where tape is not wrapped.

Claims

1. An input unit that inputs an image into an unsupervised trained neural network; a map correction unit that corrects the attention map according to a user's correction operation or a predetermined program when an attention map representing a gaze area of ​​the input image is generated by the neural network; An image recognition device comprising: an output unit that outputs the generated recognition result of the image when the neural network generates a recognition result of the image based on the corrected attention map and the image.

2. The neural network includes a feature extraction unit, an attention unit, a synthesis unit, and a recognition unit, the feature extraction unit includes a plurality of convolution layers, and generates a feature map indicating features of the image by propagating information of the image through the plurality of convolution layers; The attention unit generates the attention map based on the feature map; The synthesis unit generates a synthesis map by synthesizing the feature map and the corrected attention map; The image recognition device according to claim 1 , wherein the recognition unit generates the recognition result based on the composite map.

3. 3. The image recognition device according to claim 2, wherein the map correction unit groups the features shown in the feature map, identifies a group including features less than a predetermined value, and identifies the features of the group as features to be corrected, outputs features in the attention map corresponding to the features to be corrected as correction candidates, and accepts a correction operation by the user or a correction instruction by the predetermined program.

4. 4. The image recognition device according to claim 1, wherein the plurality of recognition targets in the image are of the same color or similar colors.

5. An image recognition method executed by an image recognition device, Inputting an image into an unsupervised trained neural network; When an attention map representing a gaze area of ​​the input image is generated by the neural network, modifying the attention map according to a user's modification operation or a predetermined program; When a recognition result for the image is generated by the neural network based on the corrected attention map and the image, a step of outputting the generated recognition result.

6. The step of modifying includes: grouping each feature amount shown in a feature amount map showing the feature amount of the image, identifying a group containing a feature amount less than a predetermined value, and identifying the feature amount of the group as a correction target feature amount; outputting, as a correction candidate, a feature quantity corresponding to the correction target feature quantity in the attention map generated based on the feature quantity map; The image recognition method according to claim 5 , further comprising: accepting a correction operation by the user or a correction instruction by the predetermined program in response to the output of the correction candidate.

Images