Recognition result display device, recognition result display method, and program

Abstract

To provide an identification result display apparatus, an identification result display method, and a program capable of displaying identification accuracy according to an identification target in a compound identifier combining a plurality of identification models.SOLUTION: A identification result display apparatus comprises: an identification result acquisition unit that acquires an identification result, which is a result of causing a compound identifier consisting of a plurality of identifiers to identify a verification image consisting of a plurality of classes, including a compound identification result by the compound identifier and an individual identification result by the identifier included in the compound identifier; an identification index calculation unit that calculates an identification index including a compound identification index and an individual identification index for each of the classes by using the identification result, wherein the compound identification index is an index indicating identification accuracy of the compound identifier, wherein the individual identification index is an index indicating the identification accuracy of the identifier included in the compound identifier; and a display unit that displays the identification index calculated by the identification index calculation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a recognition result display device, a recognition result display method, and a program. [Background technology]

[0002] In recent years, classification technologies using machine learning, such as deep learning, have been developing rapidly. For example, there is ensemble learning, which improves predictive capabilities for untrained data by combining multiple classifiers that have been trained individually. Development using complex machine learning technologies such as ensemble learning requires the knowledge of experts such as developers and engineers who have deep knowledge of machine learning technologies. In other words, it is difficult for non-experts, such as workers in factories where machine learning technologies can be introduced, to fine-tune classifiers that use machine learning.

[0003] However, workers who work in factories and the like have a deep understanding of the problems that the field faces, and there are many cases where fine adjustments are required in the factory. Furthermore, since factories need to deal with various cases in a short period of time, there are many cases where early adjustments to the classification model are required through quick judgment. For this reason, there has been a demand for the introduction of a support tool that allows even non-experts to adjust a classifier created using machine learning. Patent Document 1 discloses a technology that displays the classification result for a classification target input to a classifier. Patent Document 2 discloses a technology that compares the learning results of neural networks of multiple models. By displaying the classification results and comparison results, it is expected that even non-experts will be able to intuitively understand the classification results and accuracy, and will be able to adjust the classification model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 181458 [Patent Document 2] Patent No. 6574942 Summary of the Invention [Problem to be solved by the invention]

[0005] However, Patent Document 1 is an invention that displays the classification result for one classifier, and does not support a configuration using multiple classifiers. In addition, Patent Document 2 can compare the learning results of neural networks of multiple models, but cannot present the classification accuracy according to the input classification target. For this reason, there is a problem that it is difficult for non-experts in the field to adjust the classifier so as to improve the classification accuracy of the classification target being handled in the field.

[0006] The present invention has been made in consideration of the above-described circumstances, and provides a classification result display device, a classification result display method, and a program that are capable of displaying classification accuracy according to a classification target in a composite classifier that combines a plurality of classification models. [Means for solving the problem]

[0007] In order to solve the above-described problems, a classification result display device of the present invention includes: a classification result acquisition unit that acquires classification results obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation unit that uses the classification results to calculate, for each class, classification indexes including a composite classification index that is an index indicating classification accuracy of the composite classifier and individual classification indexes that are indexes indicating classification accuracy of classifiers included in the composite classifier; and a display unit that displays the classification indexes calculated by the classification index calculation unit. the identification result acquisition unit acquires performance information on signal processing performance related to the identification of the verification image, the display unit displays the performance information, and the performance information includes at least one of a memory capacity required for signal processing related to the identification of the verification image, a processing time required for signal processing related to the identification of the verification image, and an average processing time required to identify the verification images per unit number. It is characterized by: a classification result display device according to the present invention, the classification result being a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation unit that calculates, for each class, a classification index including a composite classification index that is an index indicating a classification accuracy of the composite classifier and an individual classification index that is an index indicating a classification accuracy of a classifier included in the composite classifier, using the classification result; a display unit that displays the classification index calculated by the classification index calculation unit; and a visualization condition acquisition unit that acquires visualization conditions indicating a manner in which the classification index is to be displayed, the display unit displays the classification index based on the visualization conditions, the visualization conditions including a multidimensional graph of at least two or more dimensions and a composite graph, the multidimensional graph including at least one of a bar graph, a line graph, a pie chart, an area graph, a scatter plot, and a radar chart, and the composite graph including a combination of at least one or more of the multidimensional graphs.

[0008] Further, a classification result display method of the present invention is a classification result display method by a computer device related to a classification result display device, in which a classification result acquisition unit acquires classification results obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification results including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier, a classification index calculation unit calculates classification indexes for each of the classes using the classification results, the classification indexes including a composite classification index that is an index indicating the classification accuracy of the composite classifier and individual classification indexes that are indexes indicating the classification accuracy of classifiers included in the composite classifier, and a display unit displays the classification indexes calculated by the classification index calculation unit. the identification result acquisition unit acquires performance information on signal processing performance related to the identification of the verification image, the display unit displays the performance information, and the performance information includes at least one of a memory capacity required for signal processing related to the identification of the verification image, a processing time required for signal processing related to the identification of the verification image, and an average processing time required to identify the verification images per unit number. It is characterized by: Further, a classification result display method of the present invention is a classification result display method by a computer device related to a classification result display device, in which a classification result acquisition unit acquires classification results obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification results including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier, and a classification index calculation unit calculates classification indexes including a composite classification index which is an index indicating the classification accuracy of the composite classifier and individual classification indexes which are indexes indicating the classification accuracy of classifiers included in the composite classifier, using the classification results, a display unit displays the discrimination index calculated by the discrimination index calculation unit; a visualization condition acquisition unit acquires visualization conditions indicating the manner in which the discrimination index is to be displayed; the display unit displays the discrimination index based on the visualization conditions; the visualization conditions include a multidimensional graph of at least two or more dimensions and a composite graph; the multidimensional graph includes at least one of a bar graph, a line graph, a pie chart, an area graph, a scatter plot, and a radar chart; and the composite graph includes a combination of at least one of the multidimensional graphs.

[0009] Furthermore, a program of the present invention causes a computer device relating to a classification result display device to function as: a classification result acquisition means for acquiring a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation means for calculating, for each of the classes, a classification index including a composite classification index that is an index indicating the classification accuracy of the composite classifier and individual classification indexes that are indexes indicating the classification accuracy of classifiers included in the composite classifier, using the classification result; and a display means for displaying the classification index. A program, the classification result acquisition means acquiring performance information on signal processing performance related to classification of the verification image, and the display means displaying the performance information, the performance information including at least one of a memory capacity required for signal processing related to classification of the verification image, a processing time required for signal processing related to classification of the verification image, and an average processing time required to classify the verification images per unit number. It is a program. Further, a program of the present invention is a program that causes a computer device related to a classification result display device to function as: a classification result acquisition means for acquiring a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation means for calculating, for each class, a classification index including a composite classification index that is an index indicating the classification accuracy of the composite classifier and individual classification indexes that are indexes indicating the classification accuracy of classifiers included in the composite classifier, using the classification result; a display means for displaying the classification index; and a visualization condition acquisition means for acquiring visualization conditions indicative of a manner in which the classification index is to be displayed, wherein the visualization condition acquisition means acquires visualization conditions indicative of a manner in which the classification index is to be displayed, and the display means displays the classification index based on the visualization conditions, the visualization conditions including a multidimensional graph of at least two or more dimensions and a composite graph, the multidimensional graph including at least one of a bar graph, a line graph, a pie chart, an area graph, a scatter plot, and a radar chart, and the composite graph including a combination of at least one or more of the multidimensional graphs. Effect of the Invention

[0010] According to the present invention, in ensemble learning, it is possible to display the classification accuracy according to the classification target input to the classification model. Therefore, even a non-expert in the field can adjust the classifier so as to improve the classification accuracy of the classification target being handled in the field. [Brief description of the drawings]

[0011] [Figure 1]1 is a block diagram showing an example of the configuration of an adjustment support system 1 to which a discrimination result display device 30 according to an embodiment is applied. [Diagram 2] 2 is a block diagram showing an example of the configuration of an identification index calculation unit 33 according to the embodiment. FIG. [Figure 3A] FIG. 11 is a diagram showing an example of the configuration of identification result information 350-1 according to the embodiment. [Figure 3B] FIG. 13 is a diagram showing an example of the configuration of identification result information 350-2 according to the embodiment. [Figure 3C] FIG. 11 is a diagram showing an example of the configuration of identification result information 350-3 according to the embodiment. [Figure 3D] FIG. 13 is a diagram showing an example of the configuration of identification result information 350-4 according to the embodiment. [Figure 4A] 13 is a diagram showing an example of a recognition result displayed on a display unit 37 according to the embodiment. FIG. [Figure 4B] 13 is a diagram showing an example of a recognition result displayed on a display unit 37 according to the embodiment. FIG. [Figure 4C] 13 is a diagram showing an example of a recognition result displayed on a display unit 37 according to the embodiment. FIG. [Figure 4D] 13 is a diagram showing an example of a recognition result displayed on a display unit 37 according to the embodiment. FIG. [Figure 5A] 13 is a diagram showing an example of a recognition result displayed on a display unit 37 according to the embodiment. FIG. [Figure 5B] 13 is a diagram showing an example of a recognition result displayed on a display unit 37 according to the embodiment. FIG. [Figure 6] 10 is a flowchart showing the flow of processing performed by a discrimination result display device 30 according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, a recognition result display device according to an embodiment will be described with reference to the drawings.

[0013] FIG. 1 is a block diagram showing an example of the configuration of an adjustment support system 1 to which an identification result display device 30 according to an embodiment is applied. The adjustment support system 1 includes a verification image storage server 10, a composite identifier 20, and an identification result display device 30.

[0014] The verification image storage server 10 is a server device in which verification images (images for verification) are stored. The verification image is an image used to verify the performance of the composite identifier 20. The verification image is an image in which the identification result is pre-labeled (associated) with the image.

[0015] For example, consider a case where the verification image is a character image and the composite identifier 20 identifies the characters shown in the verification image. In this case, the verification image shows characters of various types (Chinese characters, hiragana, katakana, variant kana, etc.) in various fonts (regular script, running script, cursive script, etc.). And the identification result of the characters shown in the verification image is labeled on the verification image. Specifically, in the verification image showing the character "橋", it is labeled that the identification result is "橋". Also, for example, in the verification image showing the character "喬", it is labeled that the identification result is not "橋". And in this case, the composite identifier 20 is a model that identifies whether the character shown in the verification image is "橋" or not.

[0016] Hereinafter, a case where the verification image is a character image and the composite identifier 20 identifies the characters shown in the verification image will be described as an example. However, it is not limited to this. For example, an aspect in which animals or plants are shown in the image and the composite identifier 20 identifies the animals or plants shown in the image may also be possible.

[0017] The verification images are classified by class. The class is a division of the verification images, for example, a division by character type. In this case, for example, a verification image showing characters in Kanji is classified into the same class regardless of the font. A verification image showing characters in Hiragana is classified into the same class regardless of the font. A verification image showing characters in Katakana is classified into the same class regardless of the font. Alternatively, the class is a division by character font. In this case, for example, a verification image showing characters in Kaisho is classified into the same class regardless of the character type. A verification image showing characters in Gyosho is classified into the same class regardless of the character type. A verification image showing characters in Cursive is classified into the same class regardless of the character type.

[0018] The composite classifier 20 is a classifier that classifies a verification image. The composite classifier 20 is a so-called ensemble learner that includes a plurality of weak classifiers (weak classifiers 21 to 23) and a composite classification unit 24. The plurality of weak classifiers are classifiers that have undergone different learning processes. In the following description, when the plurality of weak classifiers are not to be distinguished from one another, they will be referred to as "weak classifiers" or the like without being assigned reference symbols.

[0019] The weak classifier may be a supervised learning model or an unsupervised learning model. The weak classifier is a trained machine learning model (trained discrimination model) generated by performing mutually different learning. Any machine learning method may be applied to the weak classifier. Examples of any machine learning method include SVM (support vector machine), k-means, k-means++ which is an improved K-means, neural network (convolutional neural network that performs multi-class classification), decision tree, etc.

[0020] Furthermore, each of the weak classifiers may be configured using the same machine learning technique, may be configured using different techniques, or may be configured using the same technique for some of them. For example, the weak classifier 21 may be configured using SVM, the weak classifier 22 may be configured using k-means, and the weak classifier 23 may be configured using a neural network. Alternatively, the weak classifiers 21 and 22 may be configured using decision trees, and the weak classifier 23 may be configured using k-means++.

[0021] The weak classifiers classify whether an input verification image is a classification target or not, and output the classification results to the composite classification unit 24. That is, the classification results of the weak classifiers 21, 22, and 23 are each output to the composite classification unit 24.

[0022] The composite classification unit 24 determines one classification result (whether or not it is a classification target) based on the classification results of each of the multiple weak classifiers. The composite classification unit 24 determines one classification result, for example, by majority decision (voting) of the classification results of each of the multiple weak classifiers. Alternatively, the composite classification unit 24 may weight the classification results of each of the multiple weak classifiers and determine one classification result by majority decision (voting) of the weighted results. The composite classification unit 24 outputs the determined classification result to the classification result display device 30.

[0023] In this embodiment, the classification results (individual classification results) of the multiple weak classifiers are output to the classification result display device 30 together with the classification result (composite classification result) determined by the composite classification unit 24. For example, the weak classifiers 21 to 23 may output their individual classification results to the composite classification unit 24 and also to the classification result display device 30, or the composite classification unit 24 may output the individual classification results acquired from the weak classifiers 21 to 23 together with the composite classification result.

[0024] Here, in the ensemble learning device, even if a weak classifier makes an incorrect classification, the ensemble learning device is configured to output a correct classification result if many other weak classifiers make correct classification. Therefore, it is possible to improve the classification performance for various classes of classification targets. However, even if a model with high classification performance for such various classes is created, there are cases where it does not meet the needs of the field.

[0025] For example, even if a classifier with a classification rate of over 90% for all classes can be created as a result of ensemble learning, in the field, a classification accuracy of 95% or more may be required for a certain class, while other classes may require classification accuracy of less than 90%. In such a case, an ensemble learning device with a classification rate of over 90% for all classes does not meet the needs of the field.

[0026] Alternatively, even if an ensemble learning machine with a performance exceeding 95% is created for every class, if the GPU (Graphics Processing Unit) memory capacity of the on-site server equipment is not sufficient to run the program corresponding to the ensemble learning machine, then the ensemble learning machine does not meet the needs of the field.

[0027] As a countermeasure against these problems, it is conceivable to select weak classifiers that compose an ensemble learning machine so that it can be customized to achieve a classification accuracy of 95% or more only for certain classes by selecting weak classifiers that compose the ensemble learning machine. It is also conceivable to customize the ensemble learning machine with a combination of weak classifiers that can be installed in the on-site environment by selecting weak classifiers that compose the ensemble learning machine.

[0028] That is, as a countermeasure against these problems, the adjustment support system 1 is configured to display on the classification result display device 30 a display that allows a general ensemble learning device (composite classifier 20) created to improve classification accuracy for various classes to be customized (adjusted) according to on-site needs. In this case, the classification result display device 30 displays information that allows the user to intuitively understand how to customize the device. This enables even non-experts in the field to customize the ensemble learning device according to on-site needs.

[0029] The classification result display device 30 is a computer device that performs display based on the classification result obtained by the composite classifier 20. The classification result display device 30 includes, for example, a classification result acquisition unit 31, a visualization condition acquisition unit 32, a classification index calculation unit 33, a display control unit 34, a storage unit 35, an input unit 36, and a display unit 37.

[0030] The classification result acquisition unit 31 acquires the classification result by the composite classifier 20. Here, the classification result by the composite classifier 20 includes the composite classification result determined by the composite classification unit 24 and the individual classification results classified by each of the weak classifiers 21 to 23. The classification result acquisition unit 31 outputs the acquired classification result to the classification index calculation unit 33.

[0031] The visualization condition acquisition unit 32 acquires the visualization conditions. The visualization conditions are a display mode related to the identification result, and are conditions indicating, for example, the type of graph to be displayed. Examples of the graph types include a bar graph, a line graph, a pie chart, an area graph, a scatter plot, and a radar chart.

[0032] Here, the graphs that can be visualization conditions are multidimensional graphs of two or more dimensions and combinations of multidimensional graphs (composite graphs). Here, the graphs that can be visualization conditions are multidimensional graphs in order to display the classification results of multiple classes in one graph. By displaying the classification results of multiple classes in one graph, it becomes easier to intuitively understand the classification performance for each class. In addition, the graphs that can be visualization conditions are composite graphs in order to display the classification results for each class of the weak classifier and the classification results for each class of the composite classifier 20 side by side, for example, so that they can be compared. This makes it easier to intuitively understand the classification accuracy for each class of each weak classifier and the classification accuracy for each class of the composite classifier 20.

[0033] The visualization condition acquisition unit 32 acquires visualization conditions input by a user operating an input device such as a mouse or a keyboard. The visualization condition acquisition unit 32 outputs the acquired visualization conditions to the display control unit 34.

[0034] The discrimination index calculation unit 33 calculates a discrimination index using the discrimination result. The discrimination index is an index that allows discrimination performance to be compared for each class to be classified. The discrimination index calculation unit 33 calculates a discrimination index of the weak classifier using individual discrimination results from the classification results, and calculates a discrimination index for each class in the composite classifier 20 using a composite discrimination result from the classification results. A method for calculating the discrimination index by the discrimination index calculation unit 33 will be described in detail later. The discrimination index calculation unit 33 outputs the calculated discrimination index to the display control unit 34.

[0035] The display control unit 34 controls the display unit 37. The display control unit 34 acquires the visualization conditions from the visualization condition acquisition unit 32. The display control unit 34 acquires the discrimination index from the discrimination index calculation unit 33. The display control unit 34 causes the display unit 37 to display the discrimination index corresponding to the visualization conditions.

[0036] The storage unit 35 stores programs for executing various processes of the classification result display device 30, and temporary data used when performing various processes. The storage unit 35 is configured with a storage medium, for example, a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 35 stores, for example, classification result information 350. The classification result information 350 is information related to the classification result. The classification result information 350 includes, for example, the classification result acquired from the composite classifier 20 and the classification index calculated by the classification index calculation unit 33.

[0037] The input unit 36 ​​acquires information that is input through an input device such as a mouse or a keyboard. The input unit 36 ​​acquires, for example, visualization conditions that are input through the input device, and outputs the acquired visualization conditions to the visualization condition acquisition unit 32.

[0038] The display unit 37 includes a display device such as a liquid crystal display, and displays an image according to the control of the display control unit 34. The display unit 37 displays the classification index based on the weak classifier and the classification index based on the composite classifier 20 in a mode according to the visualization conditions, for example.

[0039] Here, a method for calculating the discrimination index by the discrimination index calculation unit 33 will be described in detail with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the discrimination index calculation unit 33 according to an embodiment. The discrimination index calculation unit 33 includes, for example, a combined discrimination accuracy calculation unit 330, a correct discrimination accuracy calculation unit 331, an incorrect discrimination accuracy calculation unit 332, and a machine performance calculation unit 333.

[0040] The combined classification accuracy calculation unit 330 calculates the combined classification accuracy. The combined classification accuracy is the classification accuracy by the combined classifier 20, and corresponds to the classification index by the combined classifier 20. Any index that has been conventionally used as an index for evaluating the performance of a classifier may be used as the combined classification accuracy.

[0041] The composite classification accuracy is composed of indices represented by, for example, true positive (hereinafter also referred to as TP (True Positive)), false positive (hereinafter also referred to as FP (False Positive)), true negative (hereinafter also referred to as TN (True Negative)), and false negative (hereinafter also referred to as FN (False Negative)).

[0042] True positive (TP) is an index that indicates the degree to which a classification target image is correctly classified as a classification target. For example, TP is the number of images that are correctly classified as a classification target image.

[0043] False positive (FP) is an index that indicates the degree to which an image different from the target is mistakenly identified as the target. For example, FN is the number of images that are mistakenly identified as the target when they are different from the target.

[0044] True negatives (TN) are an index that indicates the degree to which images different from the target are correctly classified as not being the target. For example, TN is the number of images different from the target that are correctly classified as not being the target.

[0045] False negative (FN) is an index that indicates the degree to which a target image is mistakenly classified as not being a target image. For example, FP is the number of target images that are mistakenly classified as not being a target image.

[0046] The composite classification accuracy may include other indices than those mentioned above. For example, the accuracy rate Ac, the precision rate Pr, the recall rate Re, the specificity rate Sp, and the F-measure may be included. These indices are calculated by the following formula (1).

[0047] Ac=(TP+TN) / (TP+FP+FN+TN) Pr=TP / (TP+FP) Re=TP / (TP+FN) Sp = TN / (FP + TN) F = 2 × Pr × Re / (Pr + Re) … (1)

[0048] In equation (1), Ac is the accuracy rate, Pr is the precision rate, Re is the recall rate, Sp is the specificity rate, and F is the F-measure. Also, TP is true positive, TN is true negative, FP is false positive, and FN is false negative.

[0049] The correct identification accuracy calculation unit 331 calculates the correct identification accuracy. The correct identification accuracy is the degree to which the verification image is correctly identified, and includes, for example, the number of correctly identified images, the correct identification ratio, the degree of contribution of the composite learning to the correct identification, the degree of true positive (TP), and the degree of true negative (TN).

[0050] The number of correctly identified images is the number of correctly identified verification images. The correct identification ratio is the ratio of correctly identified verification images to the total number of identified verification images, and corresponds to the accuracy rate Ac. The degree of contribution of compound learning to correct identification is the degree to which the compound identification process in compound identification unit 24 contributed to correct identification.

[0051] The correct classification accuracy calculation unit 331 calculates the degree of contribution of the correct classification by the composite learning based on the composite classification result and the individual classification result. For example, the correct classification accuracy calculation unit 331 calculates the degree of correct classification in both the composite classification result and the individual classification result, and the degree of correct classification in the composite classification result but incorrect classification in the individual classification result as the degree of contribution of the correct classification by the composite learning.

[0052] The misclassification accuracy calculation unit 332 calculates the misclassification accuracy. The misclassification accuracy is the degree to which the verification image is misclassified, and includes, for example, the number of misclassified images, the misclassification ratio, the degree of the contribution of misclassification due to composite learning, the degree of false positives (FP), and the degree of false negatives (FN).

[0053] The number of misclassified images is the number of verification images that were misclassified. The misclassification ratio is the ratio of the misclassified verification images to the total number of recognized verification images, and is equivalent to (1-correct answer rate Ac). The degree of misclassification contribution due to compound learning is the degree to which the compound classification process in compound classification unit 24 contributed to erroneous classification.

[0054] The misclassification accuracy calculation unit 332 calculates the degree of misclassification contribution due to composite learning based on the composite classification result and the individual classification result. The correct classification accuracy calculation unit 331 calculates, as the degree of misclassification contribution due to composite learning, for example, the degree to which both the composite classification result and the individual classification result were misclassified, and the degree to which a classification was misclassified in the composite classification result but was correct in the individual classification result.

[0055] The machine performance calculation unit 333 calculates machine performance (signal processing performance). The machine performance is the performance required for signal processing related to classification, such as the memory capacity of a CPU (Central Processing Unit) required until a classification result is calculated, the memory capacity of a GPU, the processing time required until a classification result is calculated, the average processing time required until a classification result is calculated per unit number (e.g., one image) of verification images, etc.

[0056] The method by which the machine performance calculation unit 333 acquires the memory capacity, processing time, and the like may be arbitrary. For example, the memory capacity of the CPU and / or GPU of the composite classifier 20 is stored in advance in the storage unit 35 or the like. Then, the machine performance calculation unit 333 acquires the memory capacity of the CPU and / or GPU of the composite classifier 20 by referring to the storage unit 35. In this case, for example, the composite classifier 20 measures the processing time required for classification and the number of classified verification images, and notifies the classification result display device 30 of the measurement results. The machine performance calculation unit 333 calculates the processing time and average processing time based on the measurement results notified to the classification result display device 30.

[0057] Here, the information (identification result information 350) stored in the storage unit 35 will be described with reference to Fig. 3A to Fig. 3D. Fig. 3A to Fig. 3D are diagrams showing examples of the configuration of the identification result information 350 according to the embodiment.

[0058] 3A shows, as classification result information 350-1, information indicating the relationship between input and output (classification result) when a verification image to be classified is input to the composite classifier 20. The classification result information 350-1 includes items such as a classification model configuration ID, an input, and an output.

[0059] The discrimination model configuration ID is identification information that uniquely identifies the configuration of the discrimination model (composite classifier 20). The discrimination model configuration here is, for example, information indicating weak classifiers that have performed discrimination processing among the weak classifiers 22 to 23 of the composite classifier 20. In the process of customizing the composite classifier 20, adjustment is performed such that some of the multiple weak classifiers are set not to perform discrimination processing. The discrimination model configuration ID is identification information that specifies the configuration of the composite classifier 20 after such adjustment.

[0060] The input includes items such as a classification target image, an expected value, and a class. The classification target image indicates a verification image input to the composite classifier 20 as a classification target. The expected value is information labeled on the classification target image indicating whether it is a classification target or not. The class indicates the class to which the classification target image belongs (class #1 in this case). This example shows a case where the classification target image is an image of the kanji character "bridge" written in cursive script. The expected value in this case indicates that it is "bridge", that is, that it is a classification target.

[0061] The output includes items such as weak discriminators A to N and composite determination, and identification results are associated with each of them. For weak discriminators A to N, the identification results of each of the plurality of weak discriminators are shown. For composite determination, the composite identification result by the composite identification unit 24 is shown. The identification result indicates whether it is identified as "bridge" and which of true positive (TP) and false negative (FN) the identification result corresponds to. In this example, weak discriminator A is shown to correctly identify as "bridge" and be a true positive (TP). Weak discriminator B is shown to be incorrectly identified as not "bridge" and be a false negative (FN). Also, in the composite determination, it is shown to be correctly identified as "bridge" and be a true positive (TP).

[0062] In FIG. 3B, information showing the relationship between the input and output (identification result) when a verification image that is not an identification target is input to the composite identifier 20 is shown as identification result information 350-2. The identification result information 350-2 includes items such as an identification model configuration ID, an input, and an output. Since the items of the identification model configuration ID, input, and output are the same as those of the identification result information 350-1, their descriptions are omitted.

[0063] In this example of the input, a case is shown where the identification target image is an image with the Chinese character "Qiao" written in running script. The expected value in this case is shown to be not "bridge", that is, not an identification target.

[0064] In this example of the output, weak discriminator A is shown to be incorrectly identified as "bridge" and be a false positive (FP). Weak discriminator B is shown to be correctly identified as not "bridge" and be a true negative (TN). Also, in the composite determination, it is shown to be incorrectly identified as "bridge" and be a false positive (FP).

[0065] 3C, correct classification information based on the classification result of the composite classifier 20 is shown as classification result information 350-3. The classification result information 350-3 has items such as a classification model configuration ID, the number of classification target images, and correct classification information. The classification model configuration ID item is the same as the classification model configuration ID item in the classification result information 350-1, and therefore a description thereof will be omitted.

[0066] The number of images to be classified shows the total number of classified verification images and the breakdown of the classes they belong to. In this example, the total number of classified verification images is 100, with 20 in class #1 and 8 in class #8.

[0067] The correct identification information includes the following items: number of correctly identified images (ratio), number of images that were correctly identified by ensemble learning (ratio), TP (difference from previous time), and TN (difference from previous time). The number of correctly identified images (ratio) includes items such as weak classifiers A to N and composite judgment, and each item has a column showing the number of correctly identified images (ratio) for each class. In this example, it is shown that as a result of classification by weak classifier A, 16 of the verification images belonging to class #1 were correctly classified, and the ratio is 80%.

[0068] The number (ratio) of images that were correctly classified by ensemble learning is shown as an example of the "degree of contribution of composite learning to correct classification" by indicating the number and ratio of verification images that were incorrectly classified by one of multiple weak classifiers but correctly classified by the composite classification unit 24. In this example, it is shown that 20 verification images (20% as a ratio) were correctly classified in the composite judgment process by the composite classification unit 24.

[0069] TP (difference from previous time) indicates the number of verification images that were true positives (TP) and the difference from the previous time. The previous time here refers to the number of true positives (TP) based on the classification result before the configuration of the composite classifier 20 was changed. TN (difference from previous time) indicates the number of verification images that were true negatives (TN) and the difference from the previous time. The previous time here refers to the number of true negatives (TN) based on the classification result before the configuration of the composite classifier 20 was changed. By calculating the difference from the previous time, the user can grasp the changes in true positives (TP) and true negatives (TN) before and after the change in the configuration of the composite classifier 20, and can grasp the effect of the adjustment on the classification accuracy. In this example, it is indicated that the number of true positive (TP) verification images has increased by three from the previous time to 18, and the number of true negative (TN) verification images has increased by one from the previous time to 6.

[0070] 3D, misclassification information based on the classification result of the composite classifier 20 is shown as classification result information 350-4. The classification result information 350-4 includes items such as a classification model configuration ID, the number of images to be classified, and misclassification information. The items of the classification model configuration ID and the number of images to be classified are the same as the corresponding items in the classification result information 350-3, and therefore description thereof will be omitted.

[0071] The misclassification information includes items such as the number of misclassified images (ratio), the number of images misclassified by ensemble learning (ratio), FP (difference from previous time), and FN (difference from previous time). The number of misclassified images (ratio) includes items such as weak classifiers A to N and composite judgment, and each item has a column showing the number of misclassified images (ratio) for each class. In this example, it is shown that as a result of classification by weak classifier A, 4 of the validation images belonging to class #1 were misclassified, and the ratio is 20%.

[0072] The number (ratio) of images misclassified by ensemble learning shows, as an example of the "degree of contribution of misclassification by composite learning", the number and ratio of verification images that were correctly classified by one of multiple weak classifiers but were misclassified by the composite classification unit 24. In this example, it is shown that 8 verification images (8% as a ratio) were misclassified in the process of composite judgment by the composite classification unit 24.

[0073] FP (difference from previous) indicates the number of verification images that were false positives (FP) and the difference from the previous. The previous here refers to the number of false positives (FP) based on the classification result before the configuration of the composite classifier 20 was changed. FN (difference from previous) indicates the number of verification images that were false negatives (FN) and the difference from the previous. The previous here refers to the number of false negatives (FN) based on the classification result before the configuration of the composite classifier 20 was changed. By calculating the difference from the previous, the user can grasp the change in false positives (FP) and false negatives (FN) before and after the change in the configuration of the composite classifier 20, and can grasp the effect of the adjustment on the classification accuracy. In this example, it is indicated that the number of false positive (FP) verification images has decreased by two from the previous to two, and the number of false negative (FN) verification images has decreased by one from the previous to two.

[0074] Here, the aspects of the graphs displayed on the display unit 37 will be described with reference to Figures 4A to 4D. Figures 4A to 4D are diagrams showing examples of the recognition results displayed on the display unit 37 according to the embodiment.

[0075] 4A shows an example in which the correct classification ratio for each class in a certain weak classifier (or composite classifier 20) is displayed in a radar chart. In this example, the correct classification ratio for each of a plurality of classes (classes #1 to #8) is displayed in a radar chart, making it possible to grasp which class has a high correct classification ratio and which class has a low correct classification ratio.

[0076] 4B shows an example in which the correct classification ratio for each class in a certain weak classifier (or composite classifier 20) is shown in a bar graph. In this example, the correct classification ratio for each of a plurality of classes (classes #1 to #8) is shown in a bar graph, making it possible to grasp which class has a high correct classification ratio and which class has a low correct classification ratio.

[0077] FIG. 4C shows an example of a case where the numbers of true positive (TP) and true negative (TN) validation images before and after a change in the composite classifier 20 are displayed in a bar graph. In this example, the number of true positives (TP) this time is shown by a value V1, and a value V1P of the value V1 indicates that the number has increased from the previous time. Furthermore, the number of true negatives (TN) this time is shown by a value V2, and a value V2P of the value V2 indicates that the number has increased from the previous time. In this way, by clearly indicating the difference from the previous time in the bar graph, it is possible to display the change in classification accuracy before and after the configuration of the composite classifier 20 is changed in an easy-to-understand manner.

[0078] 4D shows an example of a case where the numbers of false positive (FP) and false negative (FN) validation images before and after a change in the composite classifier 20 are displayed in a bar graph. In this example, the number of false positives (FP) this time is shown as a value V3, and it is shown that the value V3 is a number that has decreased by a value V3P from the previous time. In addition, the number of false negatives (FN) this time is shown as a value V4, and it is shown that the value V4 is a number that has decreased by a value V4P from the previous time. In this way, by clearly showing the difference from the previous time in the bar graph, it is possible to display the change in classification accuracy before and after the configuration of the composite classifier 20 is changed in an easy-to-understand manner.

[0079] Here, a description will be given of an example of a display displayed on the display unit 37 with reference to Figures 5A and 5B. Figures 5A and 5B are diagrams showing an example of a display displayed on the display unit 37 according to the embodiment.

[0080] 5A shows an example in which images of a performance information display section 371, an output condition acquisition section 372, a name information display section 373, a classification accuracy display section 374, an incorrect classification accuracy display section 375, and a correct classification accuracy display section 376 are displayed on a screen 370 of the display section 37. The performance information display section 371 displays machine performance calculated by the machine performance calculation section 333. The output condition acquisition section 372 displays visualization conditions selected by a user or the like. The name information display section 373 displays each of the weak classifiers that constitute the composite classifier 20. The classification accuracy display section 374 displays the composite classification accuracy calculated by the composite classification accuracy calculation section 330.

[0081] The misclassification accuracy display unit 375 displays the misclassification accuracy calculated by the misclassification accuracy calculation unit 332. In this example, from the upper left of the column of the misclassification accuracy display unit 375, the misclassification ratio for each class is displayed in a radar chart for each of the five weak classifiers. The upper right of the column shows a radar chart in which the misclassification ratios for each class of the five weak classifiers on the left side are superimposed. The lower right of the column shows the misclassification ratio for each class of the composite classifier 20 in a radar chart. In addition, near the center of the column, the degree of misclassification contribution due to composite learning is displayed as the number of images and the ratio thereof as the ensemble contribution. And, in the lower left of the column, false positives (FP) and false negatives (FN) are displayed together with the difference from the previous time.

[0082] The correct classification accuracy display unit 376 displays the correct classification accuracy calculated by the correct classification accuracy calculation unit 331. In this example, from the upper left of the column of the correct classification accuracy display unit 376, the correct classification ratio for each class is displayed in a radar chart for each of the five weak classifiers. The upper right of the column shows a radar chart in which the correct classification ratios for each class of the five weak classifiers on the left side are superimposed. The lower right of the column shows the correct classification ratio for each class of the composite classifier 20 in a radar chart. In addition, near the center of the column, the degree of the correct classification contribution by the composite learning is displayed as the number of images and the ratio thereof as the ensemble contribution. And, in the lower left of the column, true positives (TP) and true negatives (TN) are displayed together with the difference from the previous time.

[0083] 5B shows an example in which images of a performance information display section 371, an output condition acquisition section 372, a name information display section 373, a classification accuracy display section 374, an error classification accuracy display section 375, and a correct classification accuracy display section 376 are displayed on a screen 370 of the display section 37. The performance information display section 371, the output condition acquisition section 372, the name information display section 373, and the classification accuracy display section 374 are the same as those in FIG. 5A, and therefore their description will be omitted. The error classification accuracy display section 375 shows the radar chart in FIG. 5A as a bar graph. By performing a display according to the user's selection in this way, it is possible to perform a display that makes it easy for the user to grasp the classification accuracy.

[0084] FIG. 6 is a flowchart showing the flow of processing performed by the classification result display device 30 of this embodiment. Step S10: The classification result display device 30 acquires the classification result. The classification result display device 30 acquires the individual classification results by the weak classifiers and the composite classification result by the composite classification unit 24. Step S11: The classification result display device 30 acquires visualization conditions. The classification result display device 30 acquires visualization conditions selected by a user or the like. Step S12: The classification result display device 30 calculates a classification index based on the classification result. The classification result display device 30 calculates the composite classification accuracy, correct classification accuracy, incorrect classification accuracy, and machine performance as classification indices. The classification result display device 30 calculates the correct classification accuracy for each class for each weak classifier. The classification result display device 30 calculates the correct classification accuracy for each class in the composite classifier 20. The classification result display device 30 calculates the incorrect classification accuracy for each class for each weak classifier. The classification result display device 30 calculates the incorrect classification accuracy for each class in the composite classifier 20. Step S13: The classification result display device 30 stores the calculated classification index in the storage unit 35 as classification result information 350. Step S14: The classification result display device 30 calculates the difference between the calculated classification index and the previously calculated classification index. Step S15: The classification result display device 30 displays the correct classification accuracy based on the classification result information 350 and the visualization conditions. In this case, the classification result display device 30 may display the difference from the previous time for each index of the correct classification accuracy. Step S16: The classification result display device 30 displays the misclassification accuracy based on the classification result information 350 and the visualization conditions. In this case, the classification result display device 30 may display the difference from the previous time for each index in the misclassification accuracy. Step S17: The recognition result display device 30 displays the machine performance (performance information) based on the recognition result information 350 and the visualization conditions. In this case, the recognition result display device 30 may display the difference from the previous time for each indicator of the machine performance (memory capacity, processing time, etc.).

[0085] As described above, the classification result display device 30 of this embodiment includes the classification result acquisition unit 31, the classification index calculation unit 33, and the display unit 37. The classification result acquisition unit 31 acquires the classification result. The classification result is a classification result obtained by causing the composite classifier 20 to classify a verification image consisting of a plurality of classes. The classification result includes a composite classification result by the composite classifier 20 and individual classification results by the classifiers (weak classifiers 21 to 23) included in the composite classifier 20. The classification index calculation unit 33 calculates a classification index for each class using the classification result. The classification index includes a composite classification index and an individual classification index. The composite classification index is an index that indicates the classification accuracy of the composite classifier 20. The individual classification index is an index that indicates the classification accuracy of the weak classifiers. The display unit 37 displays the classification index calculated by the classification index calculation unit 33.

[0086] As a result, the classification result display device 30 of the embodiment can calculate a classification index for each class. This makes it possible to display the classification index for each class, allowing the user to understand the classification accuracy for each class of the classification target. Therefore, in the composite classifier 20 that combines multiple classification models, it is possible to display the classification accuracy according to the classification target.

[0087] (First Modification of the Embodiment) Here, Modification 1 will be described. This modification differs from the above-described embodiment in that it displays the frequency of appearance of a class. The frequency of appearance here refers to the frequency at which an image corresponding to a specific class appears among all verification images classified by the composite classifier 20.

[0088] By displaying the occurrence frequency, the user can understand the occurrence frequency of a specific class. Therefore, the user can understand the reliability of the classification accuracy in a specific class. For example, if the occurrence frequency of a specific class is high and the classification accuracy is high, the user can determine that the classification accuracy in the specific class is a value calculated from a large number of verification images and is a reliable value. On the other hand, if the occurrence frequency of a specific class is low and the classification accuracy is high, the user can determine that the classification accuracy in the specific class is a value calculated from a small number of verification images and is not very reliable.

[0089] In this modification, the discrimination index calculation unit 33 calculates the occurrence frequency for each class. In the following, an example will be described in which the occurrence frequency is binary information indicating whether it is greater than the average occurrence number X. The average occurrence number X is the average number of occurrences per unit class, and is a value expressed by the following formula (2).

[0090] X = (total number of identified validation images) / (total number of classes) ... (2)

[0091] The classification index calculation unit 33 compares the number of verification images corresponding to each class with the average occurrence number X. The classification index calculation unit 33 stores the comparison result in the classification result information 350. For example, the classification index calculation unit 33 stores the comparison result in association with the item of the number of images per class in the column of the number of images to be classified in the classification result information 350-3 in FIG. 3C. For example, in the case of the number of images to be classified in FIG. 3C, the total number of verification images used for classification is 100, and the number of classes is 8, so the average occurrence number X is (100 / 8=) 12.5. Since class #1 has 20 images, it is larger than the average occurrence number X, and class #1 is a class with a high occurrence frequency. On the other hand, since class #8 has 8 images, it is smaller than the average occurrence number X, and class #8 is a class with a low occurrence frequency.

[0092] When displaying the classification accuracy for each class, the display control unit 34 changes the display mode according to the frequency of occurrence of the class. For example, when displaying the correct classification ratio for each class as a bar graph as shown in Fig. 4B, the display control unit 34 displays the correct classification ratio of a class with a high frequency of occurrence with a dark bar and the correct classification ratio of a class with a low frequency of occurrence with a light bar. This makes it possible to indicate whether the correct classification ratio is reliable by the shade of the color.

[0093] (Modification 2 of the embodiment) Here, a description will be given of Modification 2. This modification differs from the above-described embodiment in that the discrimination accuracy is calculated with a focus on overdetection and non-detection, and the result is displayed.

[0094] Here, overdetection is an index showing the degree to which images that were classified as classification targets were actually not classification targets, i.e., the degree of unnecessary detection (overdetection).Furthermore, undetection is an index showing the degree to which images that were classified as not classification targets were actually classification targets, i.e., the degree to which they have not yet been detected (undetected).

[0095] Consider a case where the discrimination model (composite classifier 20 and weak classifier) ​​is a model that discriminates whether or not the character shown in a verification image is "bridge". In this case, "overdetection" refers to the degree to which a verification image that is classified as a "bridge" by the discrimination model does not actually show the character "bridge". Also, "undetected" refers to the degree to which a verification image that is classified as not a "bridge" by the discrimination model actually shows the character "bridge".

[0096] The correct classification accuracy calculation unit 331 calculates the correct classification accuracy focusing on overdetection and the correct classification accuracy focusing on non-detection as examples of the correct classification accuracy. The correct classification accuracy calculation unit 331 calculates the correct classification accuracy focusing on overdetection and the correct classification accuracy focusing on non-detection for, for example, the composite classifier 20 and each of the individual weak classifications. The correct classification accuracy focusing on overdetection is calculated, for example, by the following formula (3). In formula (3), KS is the correct classification accuracy focusing on overdetection. N1 is the number of images in which the classification target is actually shown among the images classified as the classification target. N2 is the number of images in which the classification target is shown.

[0097] K S = N1 / N2 …(3)

[0098] For example, if there are 8 images that actually depict a "bridge" among the images classified as "bridge" and there are 100 images classified as "bridge", then N1 = 8 and N2 = 100. In this case, the correct classification accuracy KS focusing on overdetection is (8 / 100). In this case, the correct classification accuracy KS focusing on overdetection is 8%. Also, in this case, 92 (N2-N1=100-8=92) images are overdetected.

[0099] The correct classification accuracy focusing on undetected areas is calculated using the following formula (4). In formula (4), MS is the correct classification accuracy focusing on undetected areas. N3 is the number of images that are classified as the classification target among the images showing the classification target. N4 is the number of images showing the classification target.

[0100] MS = N3 / N4 … (4)

[0101] For example, if there are a total of 10 images showing "bridge" and 8 of those images are classified as "bridge", then N3 = 8 and N4 = 10. In this case, the correct classification accuracy MS focusing on undetected images is (8 / 10). In this case, the correct classification accuracy MS focusing on undetected images is 80%. Also, in this case, 2 (N4-N2=10-8=2) images are undetected.

[0102] The misidentification accuracy calculation unit 332 calculates the misidentification accuracy focusing on overdetection and the misidentification accuracy focusing on non-detection as examples of the misidentification accuracy. The misidentification accuracy calculation unit 332 calculates the misidentification accuracy focusing on overdetection and the misidentification accuracy focusing on non-detection for, for example, the composite classifier 20 and each of the individual weak classifications. The misidentification accuracy focusing on overdetection is calculated, for example, by the following formula (5). In formula (5), KG is the misidentification accuracy focusing on overdetection. N3 is the number of images that do not actually show the classification target among the images classified as the classification target. N6 is the number of images classified as the classification target.

[0103] KG = N5 / N6 … (5)

[0104] For example, if the total number of images classified as "bridge" that do not actually show a "bridge" is 92, and the total number of images classified as "bridge" is 100, then N5 = 92 and N6 = 100. In this case, the misclassification accuracy KG focusing on overdetection is (92 / 100). In this case, the misclassification accuracy KG focusing on overdetection is 92%. Also, in this case, 92 (N5 = 92) images are overdetected.

[0105] The misclassification accuracy focusing on undetected items is calculated by the following formula (6). In formula (6), MG is the misclassification accuracy focusing on undetected items. N7 is the number of images that are classified as not being the target of classification among the images showing the target of classification. N8 is the number of images showing the target of classification.

[0106] MG = N7 / N8 …(6)

[0107] For example, if there are a total of 10 images showing a "bridge" and two of those images are classified as not being a "bridge," then N7 = 2 and N8 = 10. In this case, the misclassification accuracy MG focusing on undetected images is (2 / 10). In this case, the misclassification accuracy MG focusing on undetected images is 20%. Also, in this case, 2 (N7 = 2) images are undetected.

[0108] The identification result display device 30 in the above-mentioned embodiment may be realized in whole or in part by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA.

[0109] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0110] 1. Coordination support system 10…Verification image storage server 20...Composite classifier 21~23…Weak classifier 24…Composite identification section 30...Identification result display device 31...Recognition result acquisition unit 32…Visualization condition acquisition unit 33...Identification index calculation unit 330...Composite identification accuracy calculation unit 331…Correct identification accuracy calculation unit 332…Misclassification accuracy calculation unit 333…Machine performance calculation section 34...Display control unit 35...Storage section 350…Identification result information 36...Input section 37…Display section

Claims

1. a classification result acquisition unit that acquires classification results obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification results including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation unit that calculates, for each of the classes, classification indices including a composite classification index that is an index indicating classification accuracy of the composite classifier and individual classification indices that are indexes indicating classification accuracy of classifiers included in the composite classifier, using the classification result; a display unit that displays the discrimination index calculated by the discrimination index calculation unit; Equipped with the classification result acquisition unit acquires performance information on a signal processing performance related to classification of the verification image; The display unit displays the performance information, The performance information includes at least one of a memory capacity required for signal processing related to the identification of the verification image, a processing time required for signal processing related to the identification of the verification image, and an average processing time required to identify the verification image per unit number. Identification result display device.

2. a classification result acquisition unit that acquires classification results obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification results including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation unit that calculates, for each of the classes, classification indices including a composite classification index that is an index indicating classification accuracy of the composite classifier and individual classification indices that are indexes indicating classification accuracy of classifiers included in the composite classifier, using the classification result; a display unit that displays the discrimination index calculated by the discrimination index calculation unit; and a visualization condition acquisition unit that acquires visualization conditions indicating a mode in which the identification index is to be displayed; Equipped with The display unit displays the identification index based on the visualization condition, The visualization conditions include at least two-dimensional or more multidimensional graphs and composite graphs; the multidimensional graph includes at least one of a bar graph, a line graph, a pie chart, an area chart, a scatter plot, and a radar chart; The composite graph includes a combination of at least one of the multidimensional graphs. Identification result display device.

3. The composite classifier is a trained classification model generated by ensemble learning using a plurality of weak classifiers. The identification result display device according to claim 1 or 2.

4. the discrimination index calculation unit calculates, as the discrimination index, a correct discrimination accuracy indicating a degree to which the verification image is correctly discriminated or an incorrect discrimination accuracy indicating a degree to which the verification image is incorrectly discriminated, for each of the classes; The identification result display device according to any one of claims 1 to 3.

5. the classification index calculation unit calculates, as the correct classification accuracy, at least one of a degree of true positive, a degree of true negative, and a degree of contribution to correct classification by composite learning based on the composite classification result and the individual classification result; The identification result display device according to claim 4 .

6. the classification index calculation unit calculates, as a degree of contribution of the composite learning to correct classification, at least one of a degree of correct classification in both the composite classification result and the individual classification result, and a degree of correct classification in the composite classification result but incorrect classification in the individual classification result. The identification result display device according to claim 5 .

7. the classification index calculation unit calculates, as the misclassification accuracy, at least one of a degree of false positive, a degree of false negative, and a degree of contribution to misclassification due to composite learning based on the composite classification result and the individual classification result. The identification result display device according to any one of claims 4 to 6.

8. the classification index calculation unit calculates, as a degree of contribution of classification errors due to the composite learning, at least one of a degree of classification errors in both the composite classification result and the individual classification result, and a degree of classification errors in the composite classification result but correct classification in the individual classification result. The identification result display device according to claim 7 .

9. The weak classifier is a trained classification model generated by a method including at least one of a support vector machine, k-means, k-means++, a convolutional neural network that performs multi-class classification, and a decision tree. The identification result display device according to claim 3 .

10. A method for displaying a recognition result by a computer device related to a recognition result display device, comprising the steps of: a classification result acquisition unit acquires a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation unit calculates, for each of the classes, a classification index including a composite classification index that is an index indicating a classification accuracy of the composite classifier and an individual classification index that is an index indicating a classification accuracy of a classifier included in the composite classifier, using the classification result; a display unit displays the discrimination index calculated by the discrimination index calculation unit; the classification result acquisition unit acquires performance information on a signal processing performance related to classification of the verification image; The display unit displays the performance information, The performance information includes at least one of a memory capacity required for signal processing related to the identification of the verification image, a processing time required for signal processing related to the identification of the verification image, and an average processing time required to identify the verification image per unit number. Identification result display method.

11. A method for displaying a recognition result by a computer device related to a recognition result display device, comprising the steps of: a classification result acquisition unit acquires a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation unit calculates, for each of the classes, a classification index including a composite classification index that is an index indicating a classification accuracy of the composite classifier and an individual classification index that is an index indicating a classification accuracy of a classifier included in the composite classifier, using the classification result; a display unit displays the discrimination index calculated by the discrimination index calculation unit; a visualization condition acquisition unit acquires visualization conditions indicating a mode of displaying the identification index; The display unit displays the identification index based on the visualization condition, The visualization conditions include at least two-dimensional or more multidimensional graphs and composite graphs; the multidimensional graph includes at least one of a bar graph, a line graph, a pie chart, an area chart, a scatter plot, and a radar chart; The composite graph includes a combination of at least one of the multidimensional graphs. Identification result display method.

12. A computer device related to the recognition result display device, a classification result acquisition means for acquiring a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation means for calculating, for each of the classes, classification indices including a composite classification index which is an index indicating classification accuracy of the composite classifier, and individual classification indices which are indexes indicating classification accuracy of classifiers included in the composite classifier, using the classification result; A display means for displaying the identification indicator; A program that functions as The classification result acquisition means acquires performance information on a signal processing performance related to classification of the verification image, The display means displays the performance information; The performance information includes at least one of a memory capacity required for signal processing related to the identification of the verification image, a processing time required for signal processing related to the identification of the verification image, and an average processing time required to identify the verification image per unit number. program.

13. A computer device related to the recognition result display device, a classification result acquisition means for acquiring a classification result obtained by causing a composite classifier consisting of a plurality of classifiers to classify a verification image consisting of a plurality of classes, the classification result including a composite classification result by the composite classifier and individual classification results by classifiers included in the composite classifier; a classification index calculation means for calculating, for each of the classes, classification indices including a composite classification index which is an index indicating classification accuracy of the composite classifier, and individual classification indices which are indexes indicating classification accuracy of classifiers included in the composite classifier, using the classification result; A display means for displaying the identification indicator; a visualization condition acquisition means for acquiring a visualization condition indicating a mode in which the identification index is to be displayed; A program that functions as a visualization condition acquisition means for acquiring a visualization condition indicating a mode of displaying the identification index; The display means displays the discrimination index based on the visualization conditions; The visualization conditions include at least two-dimensional or more multidimensional graphs and composite graphs; the multidimensional graph includes at least one of a bar graph, a line graph, a pie chart, an area chart, a scatter plot, and a radar chart; The composite graph includes a combination of at least one of the multidimensional graphs. program.

Images