Image classification device, image classification method, and image classification program

Abstract

This provides a machine learning technique that can improve classification accuracy for images with different characteristics from those previously trained. [Solution] In an image classification device 100 that classifies an input image, the image classification model 10 calculates a prediction vector for the class to which the input image is classified. The confidence evaluation unit 20 calculates a probability vector and the confidence level of the classification based on the prediction vector of the input image. The image conversion unit 30 converts the input image to generate a converted image if the confidence level is less than a first criterion value. The integrated evaluation unit 40 calculates an integrated prediction vector by summing the prediction vector of the input image and the prediction vector of the converted image calculated based on the image classification model 10, and calculates an integrated probability vector from the integrated prediction vector. The classification determination unit 50 determines the classification of the input image based on the probability vector or the integrated probability vector.

Description

Technical Field

[0004] , , , ,

[0006] , , , ,

[0005] , , , , , , , ,

[0001] The present invention relates to image classification technology.

Background Art

[0002] As a learning method generally known in the field of machine learning, there is ensemble learning (for example, Patent Document 1). Ensemble learning is a learning method in which a plurality of inferencers are learned for one piece of data, and at the time of inference, the results of the plurality of inferencers are aggregated to obtain a final inference result by majority vote or the like. This is a concept of obtaining a final evaluation by aggregating evaluations of a plurality of inferences for one piece of data.

[0003] In addition, as a technique used in machine learning, there is a technique for increasing data called data augmentation, which is used to expand the amount of teacher data (for example, Patent Document 2).

Prior Art Documents

Non-Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] It is known that a deep learning model inductively learned using recent large-scale learning data can achieve high enough accuracy even with a single deep learning model. However, an inductively learned deep learning model has the characteristics of a black box, and the causal relationship between the input image and the inference result has not been fully elucidated. Therefore, it is known that when operating in the real world, the accuracy is not high enough for images with characteristics different from the learned images.

[0006] This invention has been made in view of these circumstances, and its purpose is to provide a machine learning technique that can improve classification accuracy for images with different characteristics from those previously trained. [Means for solving the problem]

[0007] To solve the above problems, an image classification device in one aspect of the present invention is an image classification device for classifying an input image, and includes: an image classification model that calculates a prediction vector of the class to which an image is classified for the input image; a confidence evaluation unit that calculates a probability vector and a confidence level of the classification based on the prediction vector of the input image; an image transformation unit that, if the confidence level is less than a first criterion value, transforms the input image to generate a transformed image; an integrated evaluation unit that calculates an integrated prediction vector by summing the prediction vector of the input image and the prediction vector of the transformed image calculated based on the image classification model, and calculates an integrated probability vector from the integrated prediction vector; and a classification determination unit that determines the class classification of the input image based on the probability vector or the integrated probability vector.

[0008] Another aspect of the present invention is an image classification method. This method is an image classification method for classifying an input image, and includes the steps of: calculating a prediction vector of the class to which an image is classified for an input image using an image classification model; a confidence evaluation step of calculating a probability vector and a confidence level of classification based on the prediction vector of the input image; an image transformation step of generating a transformed image by transforming the input image if the confidence level is less than a first threshold; an integrated evaluation step of calculating an integrated prediction vector by summing the prediction vector of the input image and the prediction vector of the transformed image calculated based on the image classification model, and calculating an integrated probability vector from the integrated prediction vector; and a classification determination step of determining the class classification of the input image based on the probability vector or the integrated probability vector.

[0009] Furthermore, any combination of the above components, as well as conversions of the expression of the present invention between methods, apparatus, systems, recording media, computer programs, etc., are also valid embodiments of the present invention. [Effects of the Invention]

[0010] According to the present invention, classification accuracy can be improved for images with different characteristics from those previously trained. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram showing the configuration of an image classification device according to an embodiment. [Figure 2] Figure 1 is a diagram illustrating the configuration of the image classification model. [Figure 3] This flowchart shows the procedure for classifying input images using an image classification device. [Figure 4] This flowchart shows a modified example of the input image classification procedure using an image classification device. [Figure 5] Figure 1 illustrates the classification of images using the image classification model. [Figure 6] Figures 6(a) to 6(c) illustrate the values ​​of each component of the probability vectors of images P, Q, and R. [Figure 7] This diagram illustrates another method for deriving the confidence score for image classification. [Figure 8] Figures 8(a) to 8(e) illustrate the clipping process during the conversion. [Modes for carrying out the invention]

[0012] Figure 1 is a diagram showing the configuration of an image classification device 100 according to an embodiment. The image classification device 100 takes image data as input and outputs the image class. The image classification device 100 includes an image classification model 10, a confidence evaluation unit 20, an image conversion unit 30, an integrated evaluation unit 40, and a classification determination unit 50.

[0013] The image classification model 10 calculates a prediction vector for an input image and outputs the prediction vector of the input image to the confidence evaluation unit 20 and the integrated evaluation unit 40. The image classification model 10 has already learned the classification classes of images in advance by machine learning. Each component of the prediction vector is the logit (predicted value) of each class into which the image is classified. Here, the image classification model 10 has been learned to classify L classes. For example, let L = 200, but it may be any integer value of 2 or more, and is not limited to this.

[0014] Figure 2 is a configuration diagram of the image classification model 10. The image classification model 10 includes a feature extraction unit 60 and a prediction unit 70.

[0015] The image classification model 10 is, as an example, a deep neural network, and here, it is assumed to be ResNet-50. The image classification model 10 is not limited to ResNet-50, and may be a deep neural network using a CNN such as ResNet-18, ResNet-101, ResNet-50x4, etc., or a deep neural network using a transformer of ViT (Vision Transformer). The image classification model 10 is assumed to have been learned in advance with a large-scale training dataset.

[0016] When image data is input, the feature extraction unit 60 outputs an M-dimensional feature vector indicating the feature amount of the image. For example, M = 2048.

[0017] When the feature vector is input, the prediction unit 70 outputs an L-dimensional prediction vector indicating the predicted value of the class.

[0018] Refer to FIG. 1 again. The confidence evaluation unit 20 multiplies the L-dimensional prediction vector by the softmax function, which is an activation function, to calculate an L-dimensional probability vector. The confidence evaluation unit 20 derives the confidence of class classification based on the L-dimensional probability vector. Here, the value of the i-th component (i = 1 to L) of the L-dimensional probability vector is the probability of being classified into class i. The confidence evaluation unit 20 derives the value of the highest probability among the components of the probability vector as the confidence of class classification.

[0019] If the confidence of class classification by the confidence evaluation unit 20 is equal to or higher than the reference value, the confidence evaluation unit 20 outputs the L-dimensional probability vector to the classification determination unit 50. Details of the confidence and the reference value will be described later.

[0020] The classification determination unit 50 determines the class classification of the input image based on the L-dimensional probability vector. Here, on the premise that the confidence of class classification is sufficiently high, the input image is classified into the class corresponding to the component having the maximum probability among the components of the L-dimensional probability vector.

[0021] If the confidence of class classification by the confidence evaluation unit 20 is less than the reference value, the confidence evaluation unit 20 issues an instruction to the image conversion unit 30 to generate a converted image from the input image. The confidence evaluation unit 20 sets the number N of converted images according to the confidence of class classification and issues an instruction to the image conversion unit 30. Details of the number N of converted images according to the confidence of class classification will be described later.

[0022] The image conversion unit 30 performs image conversion processing on the input image to generate N converted images. The image conversion unit 30 outputs the N generated converted images to the image classification model 10. The image conversion processing includes image clipping, rotation, etc., and details will be described later. <​​​​​The integrated evaluation unit 40 calculates an integrated prediction vector by adding the prediction vectors of the input image and the prediction vectors of the N transformed images. The integrated evaluation unit 40 calculates an integrated probability vector by multiplying the integrated prediction vector by the softmax function, which is an activation function. The integrated evaluation unit 40 outputs the integrated probability vector to the classification decision unit 50.

[0025] More specifically, the integrated evaluation unit 40 calculates a prediction vector PVn (n=1,2,···,N) for each of the N converted images. If the integrated evaluation unit 40 takes the prediction vector of the input image as PV0, it calculates an integrated prediction vector PVa by summing the prediction vector of the input image and the prediction vector of the converted image as shown in the following equation.

number

[0026] The integrated evaluation unit 40 calculates an integrated probability vector from the integrated prediction vector PVa and supplies it to the classification decision unit 50.

[0027] The classification determination unit 50 determines the class classification of the input image based on the integrated probability vector. Here, the input image is classified into the class corresponding to the component with the highest probability among the components of the L-dimensional integrated probability vector.

[0028] In this way, by inputting multiple transformed images in addition to the input image into a single image classification model 10, and considering the probability distribution based on the features of the multiple transformed images in addition to the input image, the input image can be evaluated from multiple perspectives. By using the confidence level evaluated from multiple perspectives, the reliability of image classification can be improved. In particular, by clipping a part of the image and evaluating the detailed features of the image, the reliability of classifying images that are on the boundary of the classification in the feature space can be improved.

[0029] Figure 3 is a flowchart showing the procedure for classifying input images by the image classification device 100.

[0030] The image classification model 10 calculates a prediction vector for the input image (S100). The confidence evaluation unit 20 calculates a probability vector from the prediction vector (S101). The confidence evaluation unit 20 derives the confidence level from the probability vector (S102).

[0031] The confidence evaluation unit 20 checks whether the confidence level is equal to or greater than the threshold value (S103). If the confidence level is equal to or greater than the threshold value (Y in S103), the classification determination unit 50 determines the classification based on the probability vector (S130).

[0032] If the confidence level is below the threshold value (N in S103), the confidence level evaluation unit 20 sets the number of converted images N according to the confidence level and instructs the image conversion unit 30 to generate the converted images.

[0033] The process from step S111 to step S112 is repeated N times. The image conversion unit 30 performs image conversion processing on the input image to generate a converted image (S111). The image classification model 10 calculates the prediction vector of the converted image (S112).

[0034] The integrated evaluation unit 40 aggregates the predicted vector of the input image and the predicted vectors of the N transformed images to calculate an integrated predicted vector (S120). The integrated evaluation unit 40 calculates an integrated probability vector from the integrated predicted vector (S121). The classification determination unit 50 determines the classification based on the integrated probability vector (S122).

[0035] Figure 4 is a flowchart showing a modified version of the input image classification procedure by the image classification device 100. In the classification procedure in Figure 3, the number of transformed images N is set, N transformed images are generated, and the predicted vectors of the input image and the N transformed images are aggregated to calculate a combined predicted vector. In the modified version of the classification procedure in Figure 4, a pattern for generating transformed images is determined in advance, and transformed images are created one by one. Each time, the predicted vectors of the transformed images are added to calculate a combined predicted vector, and the confidence level is derived from the combined predicted vector. The generation of transformed images is repeated until the confidence level exceeds a threshold value, but when the confidence level reaches the threshold value, the generation of transformed images is stopped, and the classification is determined based on the combined predicted vector at that point.

[0036] The image classification model 10 calculates a prediction vector for the input image (S100). The confidence evaluation unit 20 calculates a probability vector from the prediction vector (S101). The confidence evaluation unit 20 derives the confidence level from the probability vector (S102).

[0037] The confidence evaluation unit 20 checks whether the confidence level is equal to or greater than the threshold value (S103). If the confidence level is equal to or greater than the threshold value (Y in S103), the classification determination unit 50 determines the classification based on the probability vector (S130).

[0038] If the confidence level is below the threshold value (N in S103), the confidence level evaluation unit 20 instructs the image conversion unit 30 to generate one converted image.

[0039] The image conversion unit 30 performs image conversion processing on the input image to generate a converted image (S111). The image classification model 10 calculates the prediction vector of the converted image (S112).

[0040] The integrated evaluation unit 40 aggregates the prediction vectors of the transformed images and calculates an integrated prediction vector (S120). More specifically, initially, the integrated prediction vector is calculated by adding the prediction vector of the first transformed image to the prediction vector of the input image, and for the second and subsequent transformed images, the integrated prediction vector is calculated by sequentially adding the prediction vectors of the second and subsequent transformed images to the integrated prediction vector calculated so far.

[0041] The integrated evaluation unit 40 calculates an integrated probability vector from the integrated prediction vector (S121). The integrated evaluation unit 40 derives the confidence level from the integrated probability vector (S123).

[0042] Returning to step S103, the integrated evaluation unit 40 checks whether the confidence level is above the threshold value (S103). If the confidence level is above the threshold value (Y in S103), the classification determination unit 50 determines the classification based on the probability vector (S130). If the confidence level is below the threshold value (N in S103), steps S111 to S123 are executed again to generate the next transformed image, the prediction vectors of the transformed images are added together to calculate the integrated prediction vector, and the confidence level is derived.

[0043] In the modified classification procedure shown in Figure 4, the transformed images are generated sequentially in order of priority of patterns that generate transformed images, and for each transformed image, the predicted vector is sequentially added to the integrated predicted vector, and the confidence level is evaluated. The integrated predicted vector is calculated by sequentially adding the predicted vectors of the transformed images with higher priority, and the transformation process is terminated when the confidence level derived from the integrated predicted vector reaches a threshold value. This eliminates the need to calculate the predicted vectors of the transformed images with lower priority, thereby reducing the overall processing load. Furthermore, while transformed images with lower priority may cause a decrease in the accuracy of the integrated predicted vector, the classification procedure in Figure 4 prevents this decrease in accuracy.

[0044] Figure 5 illustrates the classification process performed by the image classification model 10. The feature extraction unit 60 of the image classification model 10 converts a 2D input image into an M-dimensional feature vector. This is equivalent to mapping a 2D input image to a 2048-dimensional feature space when M=2024.

[0045] Figure 5 is a conceptual diagram of the M-dimensional feature space, schematically representing in two dimensions how the five classes, Class A to Class E, are mapped to the feature space. The feature extraction unit 60 is pre-trained using training data so that the image of each class is mapped into the circle corresponding to that class.

[0046] The positions of the feature vectors of images P, Q, and R of class E are shown as points P, Q, and R in the feature space. Image P is a typical image of class E, and point P is located near the center of the circle represented by class E. Image Q is an image of class E but also slightly similar to class B, and point Q is located near the boundary of class B within class E. Image R is an image of class E but also similar to class B, and point R is located in the overlapping region of class E and class B.

[0047] Figures 6(a) to 6(c) illustrate the values ​​of each component of the probability vectors for images P, Q, and R. Each component of the probability vector represents the probability of each class A to E. Figures 6(a), 6(b), and 6(c) graph the values ​​of each component of the probability vectors for images P, Q, and R, respectively. The vertical axis represents probability, the horizontal axis represents class, and the dashed line represents the baseline value.

[0048] In the case of the probability vector for image P, the probability of class E is sufficiently large and exceeds the threshold, while the probabilities of the other classes A to D are sufficiently small.

[0049] In the case of the probability vector for image Q, the probability of class E is the highest, but it is below the threshold, and the probability of class B is also relatively high.

[0050] In the case of the probability vector for image R, the probabilities of both class E and class B are large, and the difference between them is small.

[0051] In other words, if the probability of image P being in class E is sufficiently large, it is easy to definitively classify image P as class E. However, if the probability of image Q or R being in class E is not sufficiently large, it is not easy to definitively classify them as class E.

[0052] The confidence level is derived from the highest probability value among the components of the probability vector. Here, in any case of images P, Q, and R, the highest probability value among the components of the probability vector is the probability of class E, so the confidence level is the probability value of class E. The baseline value is, for example, 0.75, but it is not limited to this value. For convenience, below we will represent the confidence level as a number from 0 to 100, which is obtained by multiplying the probability values ​​from 0 to 1 by 100, and explain using a baseline value of 75.

[0053] In the case of image P, since the probability of class E, i.e., the confidence level, is greater than or equal to the threshold value among the components of the probability vector in Figure 6(a), the confidence level evaluation unit 20 supplies the probability vector of image P to the classification determination unit 50. The classification determination unit 50 determines that image P should be classified into class E, which corresponds to the component with the highest probability among the components of the probability vector of image P.

[0054] In the case of images Q and R, the probability of class E, which has the highest component among the probability vectors in Figures 6(b) and 6(c), does not exceed the threshold value. Therefore, the class classification cannot be determined as is, and the confidence evaluation unit 20 instructs the image transformation unit 30 to generate N transformed images for images Q and R.

[0055] Figure 7 illustrates another method for deriving the confidence score for image classification. As a variation of the confidence score, the difference between the first highest probability value and the second highest probability value in the probability vector may be derived as the confidence score.

[0056] The confidence evaluation unit 20 determines the number of transformed images N according to the confidence level of the image classification, as shown in Table 1 as an example. The higher the confidence level, the smaller the number of transformed images N is, and the lower the confidence level, the larger the number of transformed images N is. However, if the confidence level is too low, generating transformed images will increase noise, so the number of transformed images N is set to 0 and no image transformation processing is performed.

[0057] [Table 1]

[0058] In the example in Table 1, the confidence level of image Q is 60, so the number of transformed images N is 3, and the confidence level of image R is 45, so the number of transformed images N is 5.

[0059] In the case of Table 1, if the confidence level is less than 25, a transformed image is not generated, so all components of the integrated prediction vector become 0, and therefore no classification result is obtained, resulting in unclassifiable. As a variation, if the confidence level is less than 25, a transformed image may not be generated, but the classification may be determined based on the probability vector of the input image. The criterion value described so far is set as the first criterion value (here, 75), and a new second criterion value (smaller than the first criterion value, here, 25) is established, and instead of step S103 in Figure 3, the confidence evaluation unit 20 checks whether the confidence level is greater than or equal to the first criterion value, or whether the confidence level is less than the second criterion value (S103'). If the confidence level is greater than or equal to the first criterion value, or if the confidence level is less than the second criterion value (Y in S103'), the confidence level evaluation unit 20 outputs the probability vector of the input image to the classification decision unit. If the confidence level is greater than or equal to the second criterion value and less than the first criterion value (N in S103'), the confidence level evaluation unit 20 instructs the image conversion unit 30 to generate a converted image.

[0060] Furthermore, if the confidence level is too low, instead of setting the number of image conversions N to 0, you may set a smaller number of conversion images when the confidence level is below the second criterion, as shown in Table 2.

[0061] [Table 2]

[0062] Let's explain another way to set the number of converted images N. As shown in Table 3, when the confidence level is above a predetermined threshold (here, 45), the higher the confidence level, the smaller the number of converted images N should be, and the lower the confidence level, the larger the number of converted images N should be. When the confidence level is below a predetermined threshold (here, 45), the higher the confidence level, the larger the number of converted images N should be, and the lower the confidence level, the smaller the number of converted images N should be. In this way, when the confidence level is above a predetermined threshold, increasing the number of converted images as the confidence level decreases increases the amount of information that can be used for class classification decisions, while when the confidence level is below a predetermined threshold, decreasing the number of converted images as the confidence level decreases prevents an increase in noise, thereby improving the signal-to-noise ratio.

[0063] [Table 3]

[0064] The conversion process for input images is predetermined, and as an example, the conversion process is determined in order of the conversion image type, as shown in Table 4. The smaller the conversion image type number, the higher the priority. If the number of conversion images N is 3, 3 conversion images will be generated by the conversion processes corresponding to conversion image types 1 to 3. If the number of conversion images N is 5, 5 conversion images will be generated by the conversion processes corresponding to conversion image types 1 to 5.

[0065] [Table 4]

[0066] In the examples in Table 4, the correspondence between the converted image type and the conversion process is as follows: Image conversion type 1: Clip the center of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 2: Clip the top left corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 3: Clip the top right corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 4: Clip the bottom left corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 5: Clip the bottom right corner of the image and reduce its horizontal and vertical size to 3 / 4.

[0067] Figures 8(a) to 8(e) illustrate the clipping process during the transformation. Each figure shows an input image with vertical dimensions H and horizontal dimensions clipped to 3 / 4 of its original size, generating a clipped image (indicated by diagonal lines) with dimensions of 3H / 4 vertically and 3W / 4 horizontally. Figure 8(a) clips the center of the image, Figure 8(b) the top left, Figure 8(c) the top right, Figure 8(d) the bottom left, and Figure 8(c) the bottom right. The size and position of the clipping are not limited to these figures. A region of interest (ROI) may also be extracted and clipped.

[0068] As part of the image transformation process, in addition to image clipping, image rotation may also be performed as shown in Table 5. Vertical and horizontal flipping may also be performed. The priority order is clipping, vertical flipping, rotation (rotations of 15, 30, 45, 60, 90 degrees, etc. are possible, but it is preferable that the rotation angle is not too large), horizontal flipping, and combinations thereof. Vertical flipping is equivalent to flipping upside down, and horizontal flipping is equivalent to flipping left and right.

[0069] [Table 5]

[0070] In the examples in Table 5, the correspondence between the converted image type and the conversion process is as follows: Image conversion type 1: Clip the center of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 2: Clip the top left corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 3: Clip the top right corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 4: Clip the bottom left corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 5: Clip the bottom right corner of the image and reduce its horizontal and vertical size to 3 / 4. Image conversion type 6: Flipping the image vertically. Image conversion type 7: Rotates the image 15 degrees clockwise. Image conversion type 8: Rotate the image 15 degrees counterclockwise. Image conversion type 9: Clip the center of the image, reduce the horizontal and vertical size to 3 / 4, and rotate it 45 degrees clockwise. Image conversion type 10: Clip the center of the image, reduce the horizontal and vertical size to 3 / 4, and rotate it 45 degrees counterclockwise.

[0071] As described above, the image classification device 100 of this embodiment utilizes a highly accurate inductively trained deep learning model and obtains a final evaluation by comprehensively evaluating and aggregating input data using a single deep learning model. This enables improved accuracy even for images with different characteristics from those previously trained in the real world.

[0072] The various processes of the image classification device 100 described above can, of course, be implemented using hardware such as a CPU and memory, but can also be implemented using firmware stored in ROM (read-only memory) or flash memory, or software on a computer. The firmware program and software program can be recorded on a recording medium readable by a computer and provided, transmitted and received with a server via a wired or wireless network, or transmitted and received as data broadcasting on terrestrial or satellite digital broadcasting.

[0073] The present invention has been described above based on embodiments. The embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their components and processing processes, and that such modifications also fall within the scope of the present invention. [Explanation of symbols]

[0074] 10 Image classification model, 20 Confidence evaluation unit, 30 Image conversion unit, 40 Integrated evaluation unit, 50 Classification decision unit, 60 Feature extraction unit, 70 Prediction unit, 100 Image classification device.

Claims

1. An image classification device that classifies input images into classes, An image classification model that calculates a prediction vector for the class to which an image is classified based on the input image, A confidence evaluation unit calculates a probability vector and a confidence level for classification based on the predicted vector of the input image. If the confidence level is less than the first reference value, an image conversion unit converts the input image to generate a converted image, An integrated evaluation unit calculates an integrated prediction vector by summing the prediction vector of the input image and the prediction vector of the transformed image calculated based on the image classification model, and calculates an integrated probability vector from the integrated prediction vector. An image classification apparatus characterized by including a classification determination unit that determines the classification of an input image based on the probability vector or the combined probability vector.

2. The image classification device according to claim 1, characterized in that the number of converted images generated decreases as the confidence level increases, and the number of converted images increases as the confidence level decreases.

3. The image classification device according to claim 1 or 2, characterized in that, if the confidence level is less than a second reference value which is less than a first reference value, the converted image is not generated, or the number of converted images is reduced as the confidence level decreases.

4. An image classification method for classifying input images, The steps include: using an image classification model to calculate a prediction vector of the class to which the input image is classified; A confidence evaluation step that calculates a probability vector and the confidence level of classification based on the prediction vector of the input image, If the confidence level is less than the first reference value, the image conversion step involves converting the input image to generate a converted image. An integrated evaluation step which involves calculating an integrated prediction vector by summing the prediction vector of the input image and the prediction vector of the transformed image calculated based on the image classification model, and then calculating an integrated probability vector from the integrated prediction vector, An image classification method characterized by comprising a classification decision step of determining the class classification of an input image based on the probability vector or the combined probability vector.

5. An image classification program that classifies input images into classes, The steps include: using an image classification model to calculate a prediction vector of the class to which the input image is classified; A confidence evaluation step that calculates a probability vector and the confidence level of classification based on the prediction vector of the input image, If the confidence level is less than the first reference value, the image conversion step involves converting the input image to generate a converted image. An integrated evaluation step which involves calculating an integrated prediction vector by summing the prediction vector of the input image and the prediction vector of the transformed image calculated based on the image classification model, and then calculating an integrated probability vector from the integrated prediction vector, An image classification program characterized by causing a computer to perform a classification decision step of determining the class classification of an input image based on the probability vector or the combined probability vector.

Images