Selection device, selection method, and program

Abstract

To provide a selection device, a selection method, and a program which can select a combination of weak discriminators constituting an ensemble discriminator, depending on identification accuracy for each class.SOLUTION: A composite discriminator is constituted by combining discriminators included in a discriminator group consisting of three or more discriminators. Each of the discriminator comprises: an evaluation result acquisition unit which acquires an evaluation result which is obtained by evaluating each of the discriminators, on the basis of an identification result in which evaluation images are classified into a plurality of classes; a required condition acquisition unit which acquires required conditions required of the composite discriminator, and including identification accuracy designated for each class; and a selection unit which selects the discriminator constituting the composite discriminator from the discriminator group, on the basis of the evaluation result and the required condition.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a selection device, a selection method, and a program. [Background technology]

[0002] In recent years, classification technologies using machine learning technologies, such as deep learning, have been rapidly developing. In the development of such technologies using machine learning technologies, adjustments based on the knowledge of experts such as developers and engineers who have deep knowledge of machine learning technologies are essential. For example, Patent Documents 1 and 2 describe technologies for optimizing the combination of weak classifiers that make up an ensemble classifier, thereby efficiently advancing the development of machine learning.

[0003] However, it is difficult for non-experts, i.e., workers at factories and other facilities where machine learning technology is being introduced, to fine-tune the machine learning identification model. However, there are many cases where the workers have a deep understanding of the challenges faced by the facility and require fine-tuning at the facility. Therefore, there is a need for a system that allows non-experts to customize models for each project and fine-tune the identification model without the need for experts. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-91083 [Patent Document 2] Japanese Patent Application Publication No. 2019-106112 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned Patent Document 1 describes a technique for determining a combination of weak classifiers from a plurality of given weak classifiers that has the highest average classification accuracy for all classes. However, Patent Document 1 does not allow the average classification accuracy, execution time, or memory conditions to be specified. Patent Document 2 describes a technique for specifying the average classification accuracy, execution time, and memory conditions. However, it does not allow the classification accuracy for each class to be specified.

[0006] The present invention has been made in consideration of the above circumstances, and provides a selection device, a selection method, and a program that are capable of selecting a combination of weak classifiers that constitute an ensemble classifier in accordance with the classification accuracy for each class. [Means for solving the problem]

[0007] In order to solve the above-described problems, a selection device of the present invention includes an evaluation result acquisition unit that acquires evaluation results of each of the classifiers, which is a composite classifier formed by combining classifiers included in a classifier group consisting of three or more classifiers, based on classification results obtained by each of the classifiers classifying an evaluation image into a plurality of classes; a requirement condition acquisition unit that acquires requirement conditions required of the composite classifier, which requirement conditions include classification accuracy specified for each class; and a selection unit that selects classifiers that constitute the composite classifier from the classifier group based on the evaluation results and the requirement conditions. the selection unit determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result, the evaluation result acquisition unit acquires, as the evaluation result, a memory capacity used by each of the classifiers for classification, the required condition acquisition unit acquires, as the required condition, an upper limit of a memory capacity that the composite classifier can use for classification, and the selection unit selects a plurality of classifiers in accordance with the ranking within a range that does not exceed the memory capacity indicated in the required condition. It is characterized by:

[0008] Further, a selection method of the present invention is a selection method by a computer device related to a selection device, in which an evaluation result acquisition unit acquires evaluation results of each of a composite classifier formed by combining classifiers included in a classifier group consisting of three or more classifiers, based on classification results obtained by each of the classifiers classifying an evaluation image into a plurality of classes; a required condition acquisition unit acquires required conditions for the composite classifier, the required conditions including classification accuracy specified for each class; and a selection unit selects classifiers that constitute the composite classifier from the classifier group based on the evaluation results and the required conditions. The selection unit determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result, the evaluation result acquisition unit acquires the memory capacity used by each of the classifiers for classification as the evaluation result, the required condition acquisition unit acquires an upper limit of the memory capacity that the composite classifier can use for classification as the required condition, and the selection unit selects a plurality of classifiers in accordance with the ranking within a range that does not exceed the memory capacity indicated in the required condition.It is characterized by:

[0009] Furthermore, the program of the present invention causes a computer device related to a selection device to function as: evaluation result acquisition means for acquiring evaluation results of each of three or more classifiers included in a classifier group, the classifiers being a composite classifier formed by combining the classifiers; requirement condition acquisition means for acquiring requirement conditions required of the composite classifier, the requirement conditions including a classification accuracy specified for each class; and selection means for selecting classifiers constituting the composite classifier from the classifier group based on the evaluation results and the requirement conditions. a program, wherein the selecting means determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result; the evaluation result acquiring means acquires, as the evaluation result, a memory capacity used by each of the classifiers for classification; the required condition acquiring means acquires, as the required condition, an upper limit of memory capacity that the composite classifier can use for classification; and the selecting means selects a plurality of classifiers in accordance with the ranking within a range that does not exceed the memory capacity indicated in the required condition. It is a program. [Effects of the Invention]

[0010] According to the present invention, a combination of weak classifiers that constitute an ensemble classifier can be selected according to the classification accuracy for each class. Therefore, even workers at factories who are not experts in machine learning technology can make adjustments to improve the classification accuracy for objects being handled on-site. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of a selection system 1 to which a selection device 20 according to an embodiment is applied. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of image information 120 according to the embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of classifier information 121 according to the embodiment. [Figure 4] FIG. 2 is a diagram showing an example of the configuration of evaluation result information 220 according to the embodiment. [Figure 5] FIG. 2 is a diagram illustrating an example of the configuration of composite classifier information 221 according to the embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a required condition according to the embodiment. [Figure 7]1 is a flowchart showing the flow of processing performed by the output device 10 according to the embodiment. [Figure 8] 10 is a flowchart showing the flow of processing performed by a selection device 20 according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] A selection system 1 and a selection device 20 according to an embodiment will be described below with reference to the drawings.

[0013] The selection system 1 is a system that selects a combination of weak classifiers that constitute an ensemble classifier. The ensemble classifier is an example of a "composite classifier" and is a model that performs classification using classification results from multiple classifiers. The weak classifiers that constitute the ensemble classifier are an example of a "classifier" and are machine learning models that perform classification using machine learning techniques.

[0014] The weak classifier may be any machine learning model, for example, a supervised learning model or an unsupervised learning model. Any machine learning method may be applied to the weak classifier. Examples of any machine learning method include SVM (support vector machine), k-means, k-means++ which is an improved k-means, neural networks (convolutional neural networks that perform multi-class classification), and decision trees.

[0015] The multiple weak classifiers (an example of a "classifier group") that make up the ensemble classifier use models that have learned different content. By comprehensively performing classification using classification results from multiple weak classifiers that have learned different content, the ensemble classifier can accurately classify various classification targets. For example, the ensemble classifier determines the classification result by majority voting. The ensemble classifier outputs the results of classification by many weak classifiers as the ensemble classifier's classification result. As a result, even if individual weak classifiers make an incorrect classification for some classification targets, the ensemble classifier can output a correct classification result if many other weak classifiers make a correct classification. Alternatively, the ensemble classifier may determine the classification result using a result obtained by weighting the classification results from the weak classifiers. As a result, even if many weak classifiers make an incorrect classification for some classification targets, the ensemble classifier can output a correct classification result if a specific high-precision weak classifier makes a correct classification. With such an approach, the ensemble classifier can correctly classify various classification targets. Therefore, the classification accuracy can be improved.

[0016] In this embodiment, a combination of weak classifiers constituting an ensemble classifier is determined depending on a classification target. For example, a combination of weak classifiers constituting an ensemble classifier is determined depending on a classification target so that the classification target can be classified with high accuracy.

[0017] The target of classification by the ensemble classifier is an image. The ensemble learning device classifies, for example, characters shown in the image. In the following description, an image that is the target of classification by the ensemble classifier is referred to as a target image. The target image is, for example, an image in which various types of characters (kanji, hiragana, katakana, variant kana, etc.) are shown in various writing styles (regular script, cursive script, semi-cursive script, etc.). The ensemble classifier classifies the characters shown in the input target image and outputs the classification result.

[0018] Here, although the case where the ensemble discriminator is the characters shown in the image has been described, it is not limited to this. The ensemble discriminator may, for example, be one that discriminates animals or plants shown in the image. In this case, animals or plants are shown in the target image.

[0019] In this embodiment, the accuracy (discrimination accuracy) of discrimination by the ensemble discriminator is evaluated. The evaluation is performed, for example, using evaluation images (referred to as evaluation images). Similar to the target images, the evaluation images show characters of various types (Chinese characters, hiragana, katakana, variant kana, etc.) in various fonts (regular script, running script, cursive script, etc.). And for the evaluation images, the discrimination results (correct values) of the classes of the characters shown in the images are labeled (associated).

[0020] Specifically, when the ensemble discriminator is a model that discriminates between two classes, whether the character shown in the image is "橋" or "喬", for the evaluation image showing the character "橋", it is labeled as being "橋". In this case, for the evaluation image showing the character "喬", it is labeled as being "喬". Also, for the evaluation image showing the character "僑", it is labeled as being neither "橋" nor "喬".

[0021] FIG. 1 is a block diagram showing an example of the configuration of a selection system 1 to which a selection device 20 according to an embodiment is applied. The selection system 1 includes, for example, an output device 10 and a selection device 20. In the selection system 1, based on the evaluation result of evaluating the discrimination by the weak discriminator output from the output device 10, the user selects a combination of weak discriminators that make up the ensemble discriminator so as to satisfy the conditions required for the ensemble discriminator.

[0022] The output device 10 is a computer device that outputs an evaluation result of evaluating the ensemble learner. The evaluation result here is the result of evaluating the discrimination by the ensemble learner from various viewpoints. The output device 10 includes, for example, a communication unit 11, a storage unit 12, and a control unit 13.

[0023] The communication unit 11 is realized by, for example, a general-purpose communication IC (Integrated Circuit). The communication unit 11 communicates with the selection device 20. The storage unit 12 is realized by, for example, a storage device (a storage device having a non-transitory storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a combination thereof. The storage unit 12 stores programs for realizing each component of the output device 10, variables used when executing the programs, and various types of information. The storage unit 12 stores, for example, image information 120 and classifier information 121.

[0024] FIG. 2 shows an example of the configuration of image information 120. Image information 120 is information related to the evaluation image. Image information 120 is created for each image. Image information 120 includes items such as an image information ID, an image, and a class. The image information ID is identification information that arbitrarily identifies the evaluation image. The image is information indicating the content of the evaluation image, such as information indicating the RGB values ​​for each pixel or information indicating the file name of the image file. The class is information indicating the correct identification result of the characters associated with the evaluation image.

[0025] FIG. 3 shows an example of the configuration of the classifier information 121. The classifier information 121 is information related to a weak classifier. The classifier information 121 includes items such as a classifier information ID, a configuration, and internal parameters. The classifier information ID is identification information that uniquely identifies a weak classifier. The configuration is information indicating the configuration of the weak classifier, and for example, if the weak classifier is a model using a neural network, the configuration is information indicating the configuration of the input layer, intermediate layer, and output layer, and whether or not convolution is performed. The internal parameters are setting values ​​of the internal parameters of the weak classifier, and for example, if the weak classifier is a model using a neural network, the internal parameters are information indicating weighting coefficients, bias values, and the like.

[0026] Returning to the explanation of Figure 1, the control unit 13 realizes its functions by having a Processing Unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) as hardware provided in the output device 10 execute a program stored in the memory unit 12.

[0027] The control unit 13 includes, for example, an acquisition unit 130, a classification unit 131, an evaluation unit 132, and a device control unit 133. The acquisition unit 130 acquires image information of an evaluation image. Here, the image information acquired by the acquisition unit 130 is, for example, image information 120 stored in the storage unit 12. Alternatively, the acquisition unit 130 may be configured to acquire image information input from outside via the communication unit 11. The acquisition unit 130 outputs the acquired image information to the classification unit 131.

[0028] The classification unit 131 causes each weak classifier to classify the evaluation image. The weak classifiers here are multiple classifiers that are components of an ensemble classifier. The classification unit 131 generates a classifier based on classifier information 121 stored in the storage unit 12, for example, and inputs image information into the generated classifier to obtain an output, which is the classification result. The classification result is binary information indicating whether or not the image shows the object to be classified. Alternatively, the classification result is information indicating the probability that the image shows the object to be classified. The classification unit 131 outputs the classification result to the evaluation unit 132.

[0029] The evaluation unit 132 evaluates the classification by the weak classifier using the classification result. The evaluation unit 132 evaluates the classification by the weak classifier, for example, with respect to the following evaluation items. Specifically, for the following evaluation items, the evaluation unit 132 calculates an index associated with each evaluation item.

[0030] (Evaluation item 1) Statistics of classification accuracy across classes (Evaluation item 2) Classification accuracy by class (Evaluation item 3) Memory capacity (Evaluation Item 4) Execution Time

[0031] (Evaluation item 1) is a statistic that can be calculated using the classification accuracy for each class, such as the average classification accuracy for the entire class. Examples of the statistical value of the classification accuracy for the entire class include the average, mode, maximum, minimum, and variance of the classification accuracy for each class.

[0032] An example of an issue that arises in situations where ensemble classifiers are introduced is the importance of the class to be classified. Generally, a model that has high classification accuracy for all classes is considered to be an excellent classifier, and the average value of classification accuracy for all classes (average classification accuracy) is often used as an indicator of the model's excellence. However, there are cases where the implementation site places importance on classification accuracy for a specific class and does not require such high classification accuracy for other classes. In other words, a method of determining a combination of weak classifiers that constitute an ensemble classifier using average classification accuracy cannot take the above-mentioned situation into consideration. To address this issue, in this embodiment, (Evaluation Item 2) is used as an evaluation index.

[0033] (Evaluation item 2) is an index indicating the degree to which an image was correctly identified for each image class. For example, the classification accuracy for each class is a value expressed as a percentage of the number of images that were correctly classified as true positives to the total number of images classified by the weak classifier for images belonging to a specific class. For example, if 100 images are classified into class C1 among the evaluation images and 90 of the 100 images are correctly classified, the classification accuracy is 90%. Note that the classification accuracy is not limited to an index indicating a true positive, and may, of course, be an index indicating, for example, a false positive, a true negative, a false positive, or a combination thereof.

[0034] Another example of an issue that may arise at the site of implementation is memory capacity. In most cases, computer devices such as PCs at the site of implementation have more limited memory capacity than the PCs at the developer. There may be cases where a classifier developed at the developer has too much memory capacity to be installed on the PCs at the site of implementation. In this case, an expert at the developer must reduce the memory capacity of the classifier, recreate it, and then install it again. This re-creation work is extremely tedious. To address this issue, in this embodiment, the memory capacity allowed at the site of implementation is acquired in advance as a requirement condition, which will be described later, and (Evaluation Item 3), which will be described next, is used as an evaluation index so that a weak classifier that satisfies the acquired requirement condition can be selected.

[0035] (Evaluation item 3) is the time required for classification by the weak classifier. The memory capacity may be calculated for each class of evaluation images to be classified, or may be calculated for each class of evaluation images. If the memory capacity required for classification varies for each classification, a statistic (e.g., average value) of the memory capacity required for classification for each class or for all evaluation images is calculated as the memory capacity.

[0036] Another example of an issue that may arise at the site where the system is implemented is the relationship between classification accuracy and execution time. Classification accuracy and execution time are in a trade-off relationship, and when highly accurate classification is required, processing by multiple weak classifiers is required, resulting in long execution times. On the other hand, shortening execution time requires processing using a small number of weak classifiers, which tends to result in lower classification accuracy. Whether classification accuracy or execution time is given priority depends on the site where the system is implemented. For this reason, a system is required that allows workers at the site to arbitrarily set whether to prioritize classification accuracy or execution time. To address this issue, in this embodiment, (Evaluation Item 4), which will be explained next, is used as an evaluation index.

[0037] (Evaluation item 4) is the time required for classification by the weak classifier. The execution time may be calculated for each class of evaluation images to be classified, or may be calculated for each class of evaluation images. If the execution time required for classification varies for each classification, a statistic (e.g., average value) of the time required for classification for each class or for all evaluation images is calculated as the execution time.

[0038] The evaluation unit 132 calculates indices associated with all or some of the above-mentioned evaluation items, and outputs the calculated index values ​​to the selection device 20. The calculated index values ​​are an example of "evaluation information."

[0039] The device control unit 133 comprehensively controls the output device 10. For example, the device control unit 133 transmits the index value calculated by the evaluation unit 132 to the selection device 20 via the communication unit 11. The device control unit 133 stores information indicating the evaluation image received by the communication unit 11 as image information 120 in the storage unit 12. The device control unit 133 stores information indicating the weak classifier received by the communication unit 11 as classifier information 121 in the storage unit 12.

[0040] The selection device 20 is a computer device that selects a combination of weak classifiers that constitute an ensemble classifier. The selection device 20 receives an evaluation result of the classification using the weak classifiers from the output device 10. The selection device 20 also acquires requirements input by a user operation or the like. The requirements are conditions required by the user for classification using the ensemble classifier, and are, for example, information indicating what level of index value is required for evaluation indices such as classification accuracy, execution time, and memory capacity. The selection device 20 selects an optimal combination of weak classifiers that constitutes an ensemble classifier for the user based on the evaluation result and the requirements.

[0041] The selection device 20 includes, for example, a communication unit 21, a storage unit 22, a control unit 23, a display unit 24, and an input unit 25.

[0042] The communication unit 21 is a functional unit that communicates with an external device, and is realized by, for example, a general-purpose communication IC. The communication unit 21 communicates with, for example, the output device 10. The display unit 24 includes, for example, a display device such as a liquid crystal display, and displays an image on the display device according to the control unit 23. The input unit 25 includes, for example, an input device such as a mouse or keyboard, and acquires information input to the input device and outputs the acquired information to the control unit 23.

[0043] The storage unit 22 is realized by, for example, a storage device such as an HDD or a flash memory, or a combination of these. The storage unit 22 stores programs for realizing each component of the selection device 20, variables used when executing the programs, and various types of information. The storage unit 22 stores, for example, evaluation information 220 and composite classifier information 221.

[0044] FIG. 4 is a diagram showing an example of the configuration of the evaluation information 220. The evaluation information 220 is information output from the output device 10 indicating the evaluation results of evaluating the classification of a weak classifier. The evaluation information 220 is generated for each weak classifier. The evaluation information 220 includes items such as an evaluation information ID, an input image, a classifier, classification accuracy, execution time, and memory capacity, for example. The evaluation information ID is identification information that uniquely identifies the evaluation result. The input image is information indicating the evaluation image that was input to the weak classifier during evaluation. The classifier is information indicating the weak classifier that was the subject of evaluation, for example, the classifier information ID of the weak classifier that was the subject of evaluation.

[0045] The classification accuracy is information indicating the classification accuracy of classification by a weak classifier. The classification accuracy is an index value corresponding to the above-mentioned (Evaluation Item 1) and (Evaluation Item 2), etc. The classification accuracy is calculated for each class, and an evaluation index for each class is shown.

[0046] The memory capacity is information indicating the memory capacity required for classification by the weak classifier. The memory capacity is an index value corresponding to the above-mentioned (Evaluation Item 3). In the example in this figure, the memory capacity is calculated for each class, and the execution time for each class is shown.

[0047] The execution time is information indicating the time required for classification by the weak classifier. The execution time is an index value corresponding to the above-mentioned (Evaluation Item 4). In the example of this figure, the execution time is calculated for each class, and the execution time for each class is shown.

[0048] The composite classifier information 221 is information related to an ensemble classifier. The ensemble classifier here is generated by combining weak classifiers selected by the selection device 20. The composite classifier information 221 is generated by combining weak classifiers selected by the selection device 20, that is, for each ensemble classifier. The composite classifier information 221 includes items such as a composite classifier information ID, an input image, a configuration, classification accuracy, an execution time, and a memory capacity. The composite classifier information ID is identification information that uniquely identifies a composite classifier. The input image is information indicating an image input to the composite classifier. The configuration is information indicating the configuration of the composite classifier. The classification accuracy is the classification accuracy when the composite classifier classifies an image specified by the input image items. The execution time is the time required for the composite classifier to classify an image specified by the input image items. The memory capacity is the memory capacity required for the composite classifier to classify an image specified by the input image items.

[0049] Returning to the explanation of FIG. 1, the control unit 23 realizes its functions by executing a program stored in the storage unit 22 using a processing unit such as a CPU or GPU as hardware included in the selection device 20.

[0050] The control unit 23 includes, for example, an evaluation result acquisition unit 230, a requirement condition acquisition unit 231, a first selection unit 232, a composite classification unit 233, a second selection unit 234, and a device control unit 235. The evaluation result acquisition unit 230 acquires the evaluation result of evaluating the classification by the weak classifier, which is output by the output device 10. The evaluation result acquisition unit 230 outputs the acquired evaluation result to the first selection unit 232.

[0051] The requirement condition acquisition unit 231 acquires requirement conditions. The requirement conditions are conditions required by the user for classification using an ensemble classifier, and are, for example, information indicating what level of index value is required for evaluation indexes such as classification accuracy, execution time, and memory capacity. The requirement condition acquisition unit 231 acquires the requirement conditions input by, for example, a user operation, via the input unit 25. The requirement condition acquisition unit 231 outputs the acquired requirement conditions to the first selection unit 232.

[0052] The first selection unit 232 selects candidates for a combination of weak classifiers that constitute an ensemble classifier in accordance with the user's requirements. The first selection unit 232 selects candidates for a combination of weak classifiers using the following method.

[0053] (Selection method 1) Determine the ranking of weak classifiers. (Selection method 2) Determine the weak classifiers that can be used. (Selection Method 3) Determine the lower limit of the number of usable weak classifiers. (Selection Method 4) Candidates for combinations of weak classifiers that compose an ensemble classifier are determined.

[0054] In (Selection Method 1), the first selection unit 232 determines the ranking of the weak classifiers based on the evaluation results. For example, the first selection unit 232 determines the ranking of the weak classifiers in descending order of average classification accuracy across classes. The evaluation images are images that are not included in the learning images that the weak classifiers have been trained on. The higher the classification accuracy in classifying the evaluation images, the higher the classification ability for untrained images, that is, the higher the generalization performance. The first selection unit 232 ranks the weak classifiers, for example, in descending order of generalization performance.

[0055] In (Selection Method 2), the first selection unit 232 determines an upper limit on the number of weak classifiers that can be used based on the rankings determined in (Selection Method 1), the memory capacities of the weak classifiers in the evaluation results, and the upper limit on memory capacity specified in the required conditions. The first selection unit 232 selects weak classifiers, for example, in descending order of ranking, and adds up the memory capacities required for classification in the selected weak classifiers. The first selection unit 232 selects the largest number of weak classifiers within a range in which the added memory capacity does not exceed the upper limit on memory capacity specified in the required conditions. The first selection unit 232 sets the weak classifiers selected in this way as classifiers that can be used in the ensemble classifier.

[0056] In (Selection Method 3), the first selection unit 232 sets the number specified by, for example, a user's input operation as the lower limit of the classifiers that can be used in the ensemble classifier. Here, the lower limit is used as a parameter that is set when selecting an ensemble classifier to be presented to the user. Note that the lower limit of the classifiers that can be used determined here is a value smaller than the number of weak classifiers selected in (Selection Method 2).

[0057] In (Selection Method 4), the first selection unit 232 determines candidate combinations of weak classifiers that constitute an ensemble classifier. The number of weak classifiers selected in (Selection Method 2) is m. The number determined in (Selection Method 3) is n. Here, m and n are natural numbers such that m>n. The first selection unit 232 selects n weak classifiers from the m weak classifiers selected in (Selection Method 2). For example, the first selection unit 232 selects n weak classifiers in descending order of the ranking determined in (Selection Method 1). The first selection unit 232 extracts all possible combinations of the n weak classifiers using at least one weak classifier from the remaining (mn) weak classifiers. The first selection unit 232 determines all the extracted combinations as candidate combinations of weak classifiers that constitute an ensemble classifier. The first selection unit 232 outputs information indicating the candidate combinations of weak classifiers that constitute the ensemble classifier to the composite classification unit 233.

[0058] The composite classification unit 233 causes an ensemble classifier, whose components are each of the combinations of weak classifiers selected by the first selection unit 232, to classify the evaluation image. For example, the composite classification unit 233 inputs image information of the evaluation image into the ensemble classifier and sets the output obtained as the classification result of the ensemble classifier. The composite classification unit 233 evaluates the classification by the ensemble classifier using the classification result of the ensemble classifier. Specifically, for the classification by the ensemble classifier, the composite classification unit 233 calculates the classification accuracy for each class, the average classification accuracy for all classes, memory capacity, execution time, etc. The composite classification unit 233 stores the calculation results in the composite classifier information 221.

[0059] The second selection unit 234 determines a combination of weak classifiers that constitute an ensemble classifier in accordance with the composite classifier information 221 and the required conditions. For example, the second selection unit 234 determines whether the classification accuracy for each class of each ensemble classifier associated with the composite classifier information 221 satisfies the classification accuracy for each class indicated in the required conditions, and extracts an ensemble classifier that satisfies the classification accuracy for each class indicated in the required conditions.

[0060] Furthermore, when the average value of the classification accuracy across all classes is specified in the requirements, the second selection unit 234 determines whether or not the classification accuracy across all classes of each ensemble classifier that satisfies the classification accuracy for each class satisfies the average value of the classification accuracy for each class specified in the requirements, and extracts ensemble classifiers that satisfy the average value of the classification accuracy across all classes specified in the requirements.

[0061] Furthermore, when a lower limit value for the execution time required for classification is specified in the requirements, the second selection unit 234 determines whether the execution time required for classification for each of the ensemble classifiers that satisfy the average classification accuracy for each class and the average classification accuracy for all classes satisfies the execution time specified in the requirements, and extracts the ensemble classifiers that satisfy the execution time specified in the requirements.

[0062] In this way, the second selection unit 234 extracts an ensemble classifier that satisfies the classification accuracy for each class, the average classification accuracy for all classes, and the execution time specified in the required conditions. The second selection unit 234 determines the combination of weak classifiers that make up the extracted ensemble classifier as the combination of weak classifiers that make up the ensemble classifier to be provided to the user.

[0063] The second selection unit 234 displays the combination of weak classifiers that make up the ensemble classifier to be provided to the user on the display unit 24. This allows the user to recognize the combination of weak classifiers that make up the ensemble classifier.

[0064] In this case, the second selection unit 234 may display the classification accuracy, execution time, etc. of each weak classifier that constitutes the ensemble classifier on the display unit 24. Alternatively, the second selection unit 234 may display on the display unit 24 the process of determining a combination of weak classifiers that constitute the ensemble classifier to be provided to the user.

[0065] In this case, the second selection unit 234 may determine the combination of weak classifiers constituting the ensemble classifier while interacting with the user in the process of determining the combination. For example, the second selection unit 234 first extracts ensemble classifiers that satisfy the classification accuracy for each class indicated in the required conditions, and displays the combination of weak classifiers constituting the extracted ensemble classifier together with a confirmation message on the display unit 24. The confirmation message is a message that asks the user to confirm whether or not to further narrow down the combination of weak classifiers constituting the displayed ensemble classifier based on other conditions, i.e., the average classification accuracy across all classes or the execution time.

[0066] Upon viewing the confirmation message, the user instructs whether or not to narrow down the search results by inputting an instruction using a keyboard or the like. The second selection unit 234 acquires information indicating the instruction from the user via the input unit 25 and responds to the instruction. Specifically, when instructed to narrow down the search results, the second selection unit 234 performs narrowing down based on other conditions. By confirming with the user whether or not to perform narrowing down, the user can confirm whether the conditions, such as the average classification accuracy across all classes and execution time, in the required conditions are essential conditions or conditions that can be compromised. This makes it possible to select a combination of weak classifiers that constitute a more appropriate ensemble classifier in line with the user's true requirements.

[0067] The device control unit 235 comprehensively controls the selection device 20. For example, the device control unit 235 stores the evaluation result acquired by the evaluation result acquisition unit 230 as evaluation information 220 in the storage unit 22. The device control unit 235 stores the classification result obtained by the ensemble classifier derived by the composite classification unit 233 as composite classifier information 221 in the storage unit 22. The device control unit 235 also causes the display unit 24 to display the selection result by the second selection unit 234.

[0068] FIG. 6 shows an example of an image displayed on the display unit 24 of the embodiment. The display unit 24 displays an example of a screen for inputting required conditions. As shown in the example of this figure, the required conditions include, for example, items such as classification accuracy for each class, memory capacity, execution time, average classification accuracy across all classes, and the minimum number of weak classifiers to be used. The classification accuracy for each class is the lower limit of the classification accuracy for each class in the ensemble classifier requested by the user. The memory capacity is the upper limit of the memory capacity in the ensemble classifier requested by the user. The execution time is the upper limit of the execution time in the ensemble classifier requested by the user. The average classification accuracy across all classes is the lower limit of the average classification accuracy across all classes in the ensemble classifier requested by the user. The minimum number of weak classifiers to be used is the minimum number of weak classifiers to be used in the ensemble classifier requested by the user.

[0069] Here, the flow of processing performed in the selection system 1 will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a flowchart showing the flow of processing performed by the output device 10 according to the embodiment. Fig. 8 is a flowchart showing the flow of processing performed by the selection device 20 according to the embodiment.

[0070] As shown in FIG. 7, the output device 10 acquires classifier information 121 of the weak classifier (step S1). The output device 10 also acquires image information 120 of the evaluation image (step S2). The output device 10 performs classification (step S3). The output device 10 inputs the image information acquired in step S2 to the weak classifier corresponding to the classifier information 121 acquired in step S1, and sets the classification result output from the weak classifier as the classification result for the input evaluation image. The output device 10 determines whether or not the weak classifier has classified all the evaluation images used for evaluating the weak classifier (step S4). If classification has been performed for all the evaluation images, the output device 10 proceeds to step S5. On the other hand, if classification has not been performed for all the evaluation images, the output device 10 returns to step S2 and repeats the classification of the evaluation images.

[0071] The output device 10 performs evaluation (step S5). Based on the results of the classification of the evaluation images, the output device 10 evaluates the classification of the weak classifiers by calculating the classification accuracy for each class, the average classification accuracy for all classes, the execution time, the memory capacity, and the like. The output device 10 transmits the evaluation results to the selection device 20 (step S6). The output device 10 determines whether or not to perform evaluation for all weak classifiers to be evaluated (step S7), and ends the processing if evaluation has been performed for all weak classifiers. On the other hand, if evaluation has not been performed for all classifiers, the output device 10 returns to step S1 and repeats evaluation of the weak classifiers.

[0072] 7, an example has been described in which the evaluation results are transmitted to the selection device 20 every time a weak classifier is evaluated in step S6, but the present invention is not limited to this. After the output device 10 has evaluated all the weak classifiers to be evaluated, the output device 10 may transmit the evaluation results to the selection device 20 all at once.

[0073] 8, the selection device 20 acquires the evaluation results of the weak classifiers from the output device 10 (step S10). The selection device 20 also acquires user requirements via the input unit 25 or the like (step S11). The selection device 20 determines the ranking of the weak classifiers (step S12). The selection device 20 uses the average classification accuracy across the entire class for each weak classifier in the evaluation results acquired in step S10 to determine the ranking of the weak classifiers, for example, in descending order of average classification accuracy.

[0074] The selection device 20 determines weak classifiers that can be used in the ensemble classifier (step S13). The selection device 20 selects the largest number of weak classifiers in descending order of the ranking determined in step S12, without exceeding the memory capacity specified by the required conditions. The selection device 20 sets the selected weak classifiers as weak classifiers that can be used in the ensemble classifier.

[0075] The selection device 20 determines a lower limit of the number of weak classifiers that can be used in the ensemble classifier (step S14). The selection device 20 sets, for example, a value specified by a user's operation input or a predetermined value as the lower limit of the number of weak classifiers that can be used in the ensemble classifier.

[0076] The selection device 20 determines candidates for combinations of weak classifiers that constitute the ensemble classifier (step S15). The selection device 20 selects n weak classifiers in step S14 from the m weak classifiers selected in step S13. The selection device 20 determines all combinations obtained by adding at least one weak classifier from the remaining (mn) weak classifiers to the selected n weak classifiers as candidates for combinations of weak classifiers that constitute the ensemble classifier.

[0077] The selection device 20 performs ensemble classification (step S16). The selection device 20 inputs an evaluation image to an ensemble classifier whose components are the candidate combinations of weak classifiers determined in step S14, and sets the classification result output from the ensemble classifier as the classification result for the input evaluation image. The selection device 20 performs classification for all evaluation images used to evaluate the ensemble classifier.

[0078] The selection device 20 performs evaluation (step S17). Based on the results of the classification performed in step S15, the selection device 20 evaluates the classification of the ensemble classifier by calculating the classification accuracy for each class, the average classification accuracy for all classes, the execution time, the memory capacity, and the like.

[0079] The selection device 20 determines whether or not evaluation has been performed for ensemble classifiers having as components all of the candidate combinations of weak classifiers determined in step S15 (step S18), and if evaluation has been performed for all of the ensemble classifiers, the process proceeds to step S18. On the other hand, if evaluation has not been performed for all of the ensemble classifiers, the output device 10 returns to step S16 and repeats evaluation of ensemble classifiers using other combinations.

[0080] The selection device 20 determines whether the ensemble classifier satisfies the required conditions based on the evaluation result (steps S19 to S22). In step S19, the selection device 20 determines whether the memory capacity of the ensemble classifier satisfies the required conditions. In step S20, the selection device 20 determines whether the classification accuracy for each class in the ensemble classifier satisfies the required conditions. In step S21, the selection device 20 determines whether the average classification accuracy across all classes in the ensemble classifier satisfies the required conditions. In step S22, the selection device 20 determines whether the execution time of the ensemble classifier satisfies the required conditions.

[0081] If the ensemble classifier satisfies the requirements, the selection device 20 determines a combination of weak classifiers that constitutes the ensemble classifier to be presented to the user (step S23). The selection device 20 determines whether or not the ensemble classifiers including all candidate combinations of weak classifiers determined in step S14 have been checked to see if they satisfy the requirements (step S24), and ends the process if the requirements have been checked for all ensemble classifiers. On the other hand, if the output device 10 has not checked whether or not all ensemble classifiers satisfy the requirements, the process returns to step S19 and checks whether or not an ensemble classifier formed by another combination satisfies the requirements.

[0082] 8, all ensemble classifiers are evaluated in steps S16 to S18, and then it is determined in steps S19 to S24 whether or not all ensemble classifiers satisfy the required conditions. However, the present invention is not limited to this. The selection device 20 may check whether or not an ensemble classifier satisfies the required conditions each time it causes an individual ensemble classifier to perform classification and evaluates the classification.

[0083] Furthermore, in the flow of FIG. 8, if no ensemble classifier that satisfies the required conditions is found in steps S19 to S24, the process may return to step S14, change the lower limit of the number of usable weak classifiers, and then execute steps S15 to S24 again, repeating the process until an ensemble classifier that satisfies the required conditions is found.

[0084] As described above, the selection device 20 of the embodiment includes the evaluation result acquisition unit 230, the requirement acquisition unit 231, and the second selection unit 234 (an example of a selection unit). The evaluation result acquisition unit 230 acquires the evaluation results. The evaluation results are evaluation results obtained based on the classification results obtained by each weak classifier classifying an evaluation image into multiple classes, and are evaluation results obtained by evaluating each of three or more weak classifiers (an example of a classifier). The weak classifiers are classifiers that can be components of an ensemble classifier (an example of a composite classifier). The requirement acquisition unit 231 acquires the requirement conditions. The requirement conditions are requirements imposed on the ensemble classifier and include classification accuracy specified for each class. The second selection unit 234 selects weak classifiers to constitute the ensemble classifier based on the evaluation results and the requirement conditions.

[0085] This allows the selection device 20 of the embodiment to acquire requirements including the classification accuracy specified for each class. This allows the selection of a combination of weak classifiers that constitute an ensemble classifier so as to satisfy the classification accuracy specified for each class. In other words, the selection device 20 can select a combination of weak classifiers that constitute an ensemble classifier according to the classification accuracy for each class.

[0086] The selection device 20 of the embodiment further includes a first selection unit 232 (an example of a selection unit) and a composite classification unit 233 (an example of a selection unit). The first selection unit 232 determines a ranking of the weak classifiers according to their classification accuracy based on the evaluation results. The first selection unit 232 selects multiple weak classifiers in the determined order. The first selection unit 232 extracts candidates for a combination of classifiers from the selected multiple weak classifiers. The composite classification unit 233 performs classification using an ensemble classifier that has the extracted combination candidates as components. The second selection unit 234 selects weak classifiers that constitute the ensemble classifier based on a determination result of whether or not the ensemble classifier satisfies the required conditions.

[0087] As a result, the selection device 20 of the embodiment can combine weak classifiers selected in accordance with the ranking according to the classification accuracy to select a generalizable ensemble classifier as a candidate. Therefore, after narrowing down the ensemble classifiers to combinations of generalizable weak classifiers, the classification accuracy of the ensemble classifiers can be evaluated, and it becomes possible to efficiently select a combination of weak classifiers that meets required conditions.

[0088] Furthermore, in the selection device 20 of the embodiment, the evaluation result acquisition unit 230 acquires the average value of the classification accuracy of the entire class as the evaluation result. The first selection unit 232 determines the ranking based on the classification accuracy of the entire class. As a result, the selection device 20 of the embodiment can narrow down the ensemble classifiers to combinations of weak classifiers that can meet various requirements, and then evaluate the classification accuracy of the ensemble classifiers, thereby making it possible to efficiently select a combination of weak classifiers that meets the requirements.

[0089] Furthermore, in the selection device 20 of the embodiment, the evaluation result acquisition unit 230 acquires, as the evaluation result, the memory capacity used by each weak classifier for classification. The requirement condition acquisition unit 231 acquires, as the requirement condition, an upper limit value of the memory capacity that the ensemble classifier can use for classification. The first selection unit 232 selects multiple weak classifiers in order of rank, within a range that does not exceed the memory capacity indicated in the requirement condition. As a result, the selection device 20 of the embodiment can select as candidates an ensemble classifier that combines weak classifiers with generalizability within the range of memory capacity allowed by the requirement condition. This makes it possible to efficiently select a combination of weak classifiers that satisfies the requirement condition.

[0090] Furthermore, in the selection device 20 of the embodiment, the requirement acquisition unit 231 acquires, as a requirement, a lower limit n of the number of weak classifiers constituting the ensemble classifier. The first selection unit 232 selects n weak classifiers from the m weak classifiers selected according to rank order. The first selection unit 232 determines, as a candidate ensemble classifier, a combination of the n weak classifiers plus at least one of the remaining (mn) weak classifiers. In this way, the selection device 20 of the embodiment can narrow down the candidate ensemble classifiers to combinations of n weak classifiers plus the remaining (mn) weak classifiers, making it possible to select a combination of weak classifiers that satisfies the requirement more efficiently than when all combinations are tried.

[0091] Furthermore, in the selection device 20 of the embodiment, the requirement condition acquisition unit 231 acquires, as a requirement condition, an upper limit value of the execution time required for classification by the ensemble classifier. The second selection unit 234 extracts candidate combinations of weak classifiers and selects, as a combination of classifiers constituting the ensemble classifier, an ensemble classifier formed by the extracted combinations, whose time required for classification by the ensemble classifier is less than the execution time specified in the requirement condition. As a result, in the selection device 20 of the embodiment, when an upper limit value of the execution time is specified as a requirement condition, it is possible to select an ensemble classifier that satisfies the requirement condition based on the results of actual classification. This makes it possible to select a combination of weak classifiers that reliably satisfies the requirement condition.

[0092] In the above-described embodiment, the order of weak classifiers is determined based on the average classification accuracy across all classes in the evaluation results. However, the present invention is not limited to this. For example, the first selection unit 232 may determine the order of weak classifiers based on the classification accuracy for each class in the evaluation results. In this case, for example, the first selection unit 232 identifies a class requiring the highest accuracy based on the classification accuracy for each class specified in the requirement conditions. The requirement condition acquisition unit 231 may determine the order of weak classifiers in descending order of classification accuracy for the identified class based on the evaluation results. This makes it possible to select an ensemble classifier that provides high classification accuracy for the class requiring the highest accuracy in the requirement conditions.

[0093] In the above-described embodiment, a case where the minimum number of weak classifiers usable is specified in the requirements has been described as an example. However, this is not limiting. For example, the first selection unit 232 may determine the minimum number of weak classifiers usable based on the memory capacity required by the requirements. In this case, for example, the first selection unit 232 may use the evaluation results to calculate in advance the average value of the memory capacity used by each weak classifier for classification. The first selection unit 232 may determine the minimum number of weak classifiers usable by dividing a predetermined percentage (e.g., 50%) of the memory capacity required by the requirements by the average value and rounding the result to an integer. In this way, a predetermined percentage of the memory capacity allowed by the requirements can be used as general-purpose weak classifiers, and the remaining memory capacity can be combined with other weak classifiers to form candidates for an ensemble classifier.

[0094] The selection system 1 and the selection device 20 in the above-described embodiment may be implemented in whole or in part by a computer. In this case, a program for implementing the functions may be recorded on a computer-readable recording medium, and the program may be loaded and executed by the computer system. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into the computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client. The program may also be designed to implement some of the functions described above, or may be capable of implementing the functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA.

[0095] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0096] 1. Selection system 10...Output device 20...Selection device 21…Communications Department 22...Storage section 23...Control unit 230...Evaluation result acquisition unit 231…Requirement condition acquisition part 232...First selection unit (selection unit) 233...Composite identification unit (selection unit) 234...Second selection unit (selection unit) 235...Device control section 24…Display section 25...Input section

Claims

1. an evaluation result acquisition unit that acquires an evaluation result obtained by evaluating each of the composite classifiers formed by combining classifiers included in a classifier group consisting of three or more classifiers, based on a classification result obtained by classifying an evaluation image into a plurality of classes by each of the classifiers; a requirement acquisition unit that acquires requirements for the composite classifier, the requirements including a classification accuracy specified for each class; a selection unit that selects classifiers that constitute the composite classifier from the classifier group based on the evaluation result and the required conditions; Equipped with the selection unit determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result; the evaluation result acquisition unit acquires, as the evaluation result, memory capacity used for classification by each of the classifiers; the requirement acquisition unit acquires, as the requirement, an upper limit value of memory capacity that the composite classifier can use for classification; the selection unit selects a plurality of classifiers in accordance with the ranking within a range not exceeding the memory capacity indicated in the required conditions. Selection device.

2. an evaluation result acquisition unit that acquires an evaluation result obtained by evaluating each of the composite classifiers formed by combining classifiers included in a classifier group consisting of three or more classifiers, based on a classification result obtained by classifying an evaluation image into a plurality of classes by each of the classifiers; a requirement acquisition unit that acquires requirements for the composite classifier, the requirements including a classification accuracy specified for each class; a selection unit that selects classifiers that constitute the composite classifier from the classifier group based on the evaluation result and the required conditions; Equipped with the selection unit determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result; the requirement acquisition unit acquires, as the requirement, an upper limit of an execution time required for classification by the composite classifier; the selection unit extracts candidates for a combination of the classifiers constituting the composite classifier, and selects, as a combination of classifiers constituting the composite classifier, a combination of the classifiers for which a time required for classification by the composite classifier formed from the extracted combination is less than an execution time under the required conditions. Selection device.

3. the selection unit selects the plurality of classifiers in the determined order, extracts candidates for a combination of the classifiers consisting of the plurality of selected classifiers, determines whether the composite classifier satisfies the required condition based on a classification result of the composite classifier including the extracted candidates for the combination as components, and selects classifiers constituting the composite classifier from the group of classifiers based on the determination result. The selection device according to claim 1 or 2.

4. the evaluation result acquisition unit acquires an average value of the classification accuracy of the entire class as the evaluation result; The selection unit determines the ranking based on the average value. The selection device according to claim 3 .

5. the requirement acquisition unit acquires, as the requirement, a lower limit n of the number of classifiers constituting the composite classifier, where n is an arbitrary natural number; the selection unit selects m classifiers according to the ranking, where m is a natural number satisfying (m>n), selects n classifiers from the selected m classifiers, and extracts candidates for combinations of the classifiers by adding at least one classifier from the remaining (m-n) classifiers to the selected n classifiers; A selection device according to any one of claims 1 to 4.

6. A selection method by a computer device related to a selection device, comprising: an evaluation result acquisition unit acquires evaluation results of a composite classifier formed by combining classifiers included in a classifier group consisting of three or more classifiers, by evaluating each of the classifiers based on classification results obtained by each of the classifiers classifying an evaluation image into a plurality of classes; a required condition acquisition unit acquires required conditions for the composite classifier, the required conditions including classification accuracy specified for each class; a selection unit selecting classifiers constituting the composite classifier from the group of classifiers based on the evaluation result and the required conditions; the selection unit determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result; the evaluation result acquisition unit acquires, as the evaluation result, memory capacity used for classification by each of the classifiers; the requirement acquisition unit acquires, as the requirement, an upper limit value of memory capacity that the composite classifier can use for classification; the selection unit selects a plurality of classifiers in accordance with the ranking within a range not exceeding the memory capacity indicated in the required conditions. How to select.

7. A selection method by a computer device related to a selection device, comprising: an evaluation result acquisition unit acquires evaluation results of a composite classifier formed by combining classifiers included in a classifier group consisting of three or more classifiers, by evaluating each of the classifiers based on classification results obtained by each of the classifiers classifying an evaluation image into a plurality of classes; a required condition acquisition unit acquires required conditions for the composite classifier, the required conditions including classification accuracy specified for each class; a selection unit selecting classifiers constituting the composite classifier from the group of classifiers based on the evaluation result and the required conditions; the requirement acquisition unit acquires, as the requirement, a lower limit of an execution time required for classification by the composite classifier; the selection unit extracts candidates for a combination of the classifiers constituting the composite classifier, and selects, as a combination of classifiers constituting the composite classifier, a combination of the classifiers for which a time required for classification by the composite classifier formed from the extracted combination is less than an execution time under the required conditions. How to select.

8. A computer device related to the selection device, an evaluation result acquisition means for acquiring evaluation results of each of the three or more classifiers in a composite classifier formed by combining classifiers included in a classifier group consisting of three or more classifiers, based on classification results obtained by classifying an evaluation image into a plurality of classes by each of the classifiers; a requirement acquisition means for acquiring requirements required for the composite classifier, including classification accuracy specified for each class; a selection means for selecting classifiers constituting the composite classifier from the group of classifiers based on the evaluation result and the required conditions; A program that functions as the selecting means determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result; the evaluation result acquisition means acquires, as the evaluation result, the memory capacity used by each of the classifiers for classification; the required condition acquisition means acquires, as the required condition, an upper limit value of memory capacity that the composite classifier can use for classification; the selecting means selects a plurality of classifiers in accordance with the ranking within a range not exceeding the memory capacity indicated in the required conditions; program.

9. A computer device related to the selection device, an evaluation result acquisition means for acquiring evaluation results of each of the three or more classifiers in a composite classifier formed by combining classifiers included in a classifier group consisting of three or more classifiers, based on classification results obtained by classifying an evaluation image into a plurality of classes by each of the classifiers; a requirement acquisition means for acquiring requirements required for the composite classifier, including classification accuracy specified for each class; a selection means for selecting classifiers constituting the composite classifier from the group of classifiers based on the evaluation result and the required conditions; A program that functions as the selecting means determines a ranking of each of the classifiers according to classification accuracy based on the evaluation result; the requirement acquisition means acquires, as the requirement, a lower limit of an execution time required for classification by the composite classifier; the selecting means extracts candidates for combinations of the classifiers constituting the composite classifier, and selects, as a combination of classifiers constituting the composite classifier, a combination of the classifiers for which a time required for classification by the composite classifier formed from the extracted combination is less than an execution time under the required conditions. program.

Images