Information processing device and information processing method

Abstract

To enable detailed classification using a downsized classification model.SOLUTION: An information processing device according to the present application includes a class generation unit that generates higher classes that are higher than a plurality of different classes and correspond to hierarchical clusters based on a plurality of class feature amounts hierarchically clustered on the basis of the similarity between the plurality of class feature amounts which are about data belonging to the respective different classes and are representative values of the respective different classes, and a model generation unit that generates, for each of the higher classes, a classification model which is a machine learning model for classifying classification target data that are to be classified, to any one of a plurality of lower classes belonging to lower levels of the higher classes.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Conventionally, various machine learning models, including deep learning models, are known as classification models that classify data to be classified with high accuracy. Conventionally, various techniques for increasing the size of a machine learning model (hereinafter, sometimes referred to as model size) have been known in order to improve the classification accuracy of a classification model. For example, a technique for complexly scaling up the number of channels, the number of layers, and the resolution of each layer of a convolutional neural network (CNN) used as an image classification model using a single parameter is known. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Mingxing Tan, Quoc V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, [online], May 2019, [Retrieved April 17, 2020], Internet<URL:https: / / arxiv.org / pdf / 1905.11946.pdf> Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned conventional technology, the number of channels, the number of layers, and the resolution of each layer of a convolutional neural network (CNN) used as an image classification model are merely scaled up in a composite manner using a single parameter, and therefore it is not necessarily possible to perform detailed classification using a compact classification model.

[0005] An object of the present application is to provide an information processing device and an information processing method that are capable of performing detailed classification using a compact classification model. [Means for solving the problem]

[0006] The information processing device according to the present application includes a class generation unit that generates higher classes that are higher than the plurality of different classes, the higher classes being classes corresponding to each of hierarchical clusters based on a plurality of class features that are hierarchically clustered based on a similarity between the plurality of class features that are representative values ​​of each of the plurality of different classes, and a model generation unit that generates a classification model for each of the higher classes, which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that belong to the higher class.

[0007] The information processing device according to the present application further includes an acquisition unit that acquires a classification model that is a machine learning model that classifies classification target data to be classified into one of a plurality of lower classes belonging to a higher class that is a class higher than the plurality of different classes, the lower class being a class that corresponds to each of the hierarchical clusters based on a plurality of class features that are hierarchically clustered based on a similarity between the plurality of class features that are representative values ​​of each of the plurality of different classes, and processing target data to be processed; and an estimation unit that estimates a first lower class into which the processing target data is classified using a higher classification model that is the classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest class, selects a lower classification model that is the classification model corresponding to the first lower class from the classification models corresponding to each of the plurality of lower classes belonging to a lower class below the predetermined higher class based on a result of the estimation of the higher classification model, and estimates a second lower class into which the processing target data is classified using the selected lower classification model. Effect of the Invention

[0008] According to one aspect of the embodiment, it may be possible to perform detailed classification using a reduced classification model. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the relationship between the model size of a classification model and the number of classification classes. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Diagram 3] FIG. 3 is a diagram for explaining a hierarchical class generation process according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining the process of generating a hierarchical classification model according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating a training data set of a hierarchical classification model according to the embodiment. [Figure 6] FIG. 6 is a diagram for explaining the class estimation process according to the embodiment. [Figure 7] FIG. 7 is a diagram for explaining a case where an estimation result of a lower class becomes incorrect by adopting only one estimation result by a higher-level classification model. [Figure 8] FIG. 8 is a diagram for explaining a case where a prediction result of a lower class becomes the correct answer by adopting a plurality of prediction results of higher-level classification models. [Figure 9] FIG. 9 is a diagram for explaining a normal multi-class classification model and a multi-class classification model capable of detecting an unknown class. [Figure 10] FIG. 10 is a diagram for explaining a class estimation process according to a modified example. [Figure 11] FIG. 11 is a diagram for explaining an example of a case where the estimation process according to the modified example is stopped. [Figure 12]FIG. 12 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, a detailed description will be given of an information processing device and an information processing method according to the present application (hereinafter, referred to as an "embodiment") with reference to the drawings. Note that the information processing device and the information processing method according to the present application are not limited to the embodiment. In addition, the same components in the following embodiments are given the same reference numerals, and duplicated descriptions are omitted.

[0011] (Embodiment) 1. Introduction Conventionally, it is known that as the number of classification classes by a classification model increases, the complexity of the learning data set used to learn the classification model increases, and the model size of the classification model (e.g., deep learning model) that has learned those features tends to become large. For example, in the case of a behavior recognition model, which is a machine learning model that recognizes the type of behavior of a person included in a video, even for a single action of "grabbing", the type of the behavior (i.e., classification class) of the grab is different for each type of object being grabbed. For example, the behavior recognition model is trained to classify the action of grabbing a person's hand and the action of grabbing a person's foot as different actions. In this way, for example, if the action is between a person and an object, such as the action of "grabbing", the behavior recognition model is trained to classify the action by the type of object that is the target of the person's action. In addition, the types of human actions are not limited to actions between a person and an object, and there are a wide variety of types such as actions between people and people or actions of a person (single person). Therefore, it is generally known that the model size of the behavior recognition model tends to be very large.

[0012] The relationship between the model size of a classification model and the number of classification classes will be described with reference to FIG. 1. FIG. 1 is a diagram showing the relationship between the model size of a classification model and the number of classification classes. As shown in FIG. 1, it is generally known that as the number of classification classes in a classification model increases, the model size of the classification model increases rapidly. In addition, a size limit, which is the upper limit of the model size, is determined according to the machine specifications of an information processing device that implements the classification model and the upper limit of the amount of computer resources. In addition, once the size limit is determined, a classifiable limit, which is the upper limit of the number of classification classes that can be classified by the classification model, is determined according to the size limit.

[0013] As described in FIG. 1, the larger the model size of the classification model, the greater the number of classification classes that can be classified by the classification model. However, data classification processing using a classification model with a large model size (hereinafter, sometimes referred to as a "large classification model") requires higher performance machine specifications and more abundant computer resources. Therefore, for example, it is conceivable to implement a large classification model in cloud computing (hereinafter, sometimes referred to as the cloud), which has more abundant computer resources than on-premise information processing devices, and classify data.

[0014] However, some data to be classified cannot be stored in the cloud from the viewpoint of privacy. In such cases where the cloud cannot be used, the classification model is implemented in an on-premise information processing device to classify the data. However, there is an upper limit to the machine specifications and the amount of computer resources of the on-premise information processing device (particularly the edge device). Even when the cloud is used, there is generally an upper limit to the machine specifications and the amount of computer resources that can be used due to the financial resources of the user, etc.

[0015] Thus, in general, there is an upper limit to the level of available machine specifications and the amount of computer resources. Therefore, when enlarging a classification model to enable more detailed classification, there is an upper limit to the model size of the available classification model (size limit shown in FIG. 1). In addition, since there is an upper limit to the model size of the available classification model, there is also an upper limit to the number of classification classes that can be classified by the classification model (classification limit shown in FIG. 1). Therefore, it has been difficult in some cases to enable detailed classification using an enlarged classification model.

[0016] In contrast, the information processing device according to the present embodiment performs hierarchical clustering on class features, which are average values ​​for each class of features corresponding to data belonging to each of a plurality of different classes obtained from a large-scale classification model generated by a known technique. The information processing device also generates a plurality of upper classes, which are classes corresponding to each of the hierarchical clusters based on the class features hierarchically clustered by the hierarchical clustering and are classes higher than the plurality of different classes. In this way, the information processing device generates a hierarchical class, which is a hierarchical class including a plurality of different classes and a plurality of upper classes that are classes higher than the plurality of different classes. Note that the plurality of different classes corresponds to the lowest class in the hierarchical class. The information processing device also generates a tree diagram, which is a graph representing the process of generating hierarchical clusters by the hierarchical clustering, and obtains classification models in a number corresponding to the number of branch points included in the tree diagram. Here, each of the branch points included in the tree diagram corresponds to each of the upper classes included in the hierarchical class. The information processing device also generates a classification model for each upper class that has been trained to classify the classification target data to be classified into one of a plurality of lower classes that belong to the lower class of the upper class.

[0017] As a result, the information processing device according to this embodiment can increase the number of classification classes while keeping the model size of each of the multiple classification models generated for each higher class small. Therefore, the information processing device can realize a miniaturized classification model that enables detailed classification. Furthermore, since the information processing device can realize a miniaturized classification model that enables detailed classification, it can make it possible to perform detailed classification using a miniaturized classification model. For example, the information processing device can make it possible to perform detailed classification of behaviors using a miniaturized behavior recognition model.

[0018] Further, the information processing device according to the present embodiment acquires a plurality of classification models generated for each upper class and processing target data to be processed. Next, the information processing device selects a top classification model corresponding to the top class from among the plurality of classification models. Next, the information processing device uses the selected top classification model to estimate a first top class (hereinafter, may be referred to as a first top class) in which the processing target data is classified from among the plurality of upper classes belonging to the lower level of the highest class. Next, the information processing device selects a first top classification model corresponding to the first top class from among the classification models corresponding to each of the plurality of upper classes belonging to the lower level of the highest class based on the estimation result of the top classification model. Next, the information processing device uses the selected first top classification model to estimate a second top class (hereinafter, may be referred to as a second top class) in which the processing target data is classified from among the plurality of upper classes belonging to the lower level of the first top class. Next, the information processing device selects a second higher classification model corresponding to the second higher class from among the classification models corresponding to each of the multiple higher classes subordinate to the first higher class based on the estimation result of the first higher classification model (hereinafter, this may be referred to as a "selection process"). Next, the information processing device uses the selected second higher classification model to estimate a class into which the data to be processed is classified from among the multiple classes subordinate to the second higher class (hereinafter, this may be referred to as an "estimation process"). In this way, the information processing device repeatedly executes the selection process and the estimation process until the class into which the data to be processed is classified matches one of the multiple different classes that are the lowest class.

[0019] As a result, the information processing device according to the present embodiment can perform detailed classification using a compact classification model. For example, the information processing device can perform detailed classification of behaviors using a compact behavior recognition model.

[0020] In this specification, the expressions "upper class" and "lower class" indicate a relative relationship in a hierarchical class. For example, a "higher class" in this specification refers to a class that is higher than multiple different classes that are the lowest class included in the hierarchical class. Also, a "lower class" in this specification refers to a class that belongs to a lower level than a predetermined higher class included in the hierarchical class. Therefore, among the multiple classes included in the hierarchical class, classes other than the highest class and the lowest class (multiple different classes) may be higher classes or lower classes.

[0021] Furthermore, the classification model in this specification may be, for example, a classification model that classifies images (also referred to as an image classification model), a classification model that classifies videos (also referred to as a video classification model), a classification model that classifies types of behavior of people included in videos (also referred to as a behavior recognition model), or a classification model that classifies sentences (also referred to as a sentence classification model or text classification model).

[0022] 2. Configuration of Information Processing Device An example of the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0023] (Communication unit 110) The communication unit 110 is realized by a NIC (Network Interface Card), an antenna, etc. The communication unit 110 is connected to various networks by wire or wirelessly, and transmits and receives information to and from other information processing devices other than the information processing device 100, for example.

[0024] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Specifically, the storage unit 120 stores various data. For example, the storage unit 120 stores information about a learning data set used for learning a classification model. The storage unit 120 also stores various programs. For example, the storage unit 120 stores information about the classification model generated for each upper class by the model generation unit 133 and information capable of identifying the upper class in association with each other.

[0025] (Control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), etc., executing various programs stored in a storage device inside the information processing device 100 using a RAM as a working area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0026] The control unit 130 has an acquisition unit 131, a class generation unit 132, a model generation unit 133, and an estimation unit 134 as functional units, and may realize or execute the information processing action described below. The internal configuration of the control unit 130 is not limited to the configuration shown in Fig. 2, and may be other configurations as long as they perform the information processing described below. Furthermore, each functional unit indicates a function of the control unit 130, and does not necessarily have to be physically distinct.

[0027] (Acquisition part 131) The acquisition unit 131 may acquire various information. Specifically, the acquisition unit 131 may acquire features related to data belonging to a plurality of different classes. The acquisition unit 131 may acquire features corresponding to the data belonging to a plurality of different classes as features related to the data belonging to a plurality of different classes. For example, the acquisition unit 131 may acquire features corresponding to the data belonging to each of the plurality of different classes extracted by feature extraction provided in the large-scale classification model from another information processing device implementing a large-scale classification model generated by a known technique via the communication unit 110. For example, the acquisition unit 131 may acquire features corresponding to the data belonging to each of the plurality of different classes (hereinafter, may be referred to as "plurality of different classes") which are all classes that can be classified by the large-scale classification model. In other words, the data belonging to a plurality of different classes is data that is classified into a plurality of different classes using the large-scale classification model.

[0028] For example, if the classification model is an image classification model, the large classification model generated by known techniques may be AlexNet (Krizhevsky et al., 2012), ZFNet (Matthew et al., 2013), VGGNet (Simonyan et al., 2014), GoogLeNet (Szegedy et al., 2014), ResNet (Residual Network) (Kaiming He et al., 2015), DenseNet (Gao Huang et al., 2016), MobileNet (Andrew G. Howard et al., 2017), SENet (Squeeze-and-Excitation Networks) (Jie Hu et al., 2018), EfficientNet (Tan et al., 2019), or EfficientNetV2 (Tan et al., 2021), etc.

[0029] (Class generation unit 132) The class generation unit 132 may calculate class features that are average values, for each of the multiple different classes, of features corresponding to data belonging to each of the multiple different classes acquired by the acquisition unit 131. The class generation unit 132 may calculate class features that are average values ​​for each of the multiple different classes of features of data belonging to each of the multiple different classes. Here, the class features are representative values ​​of the features of data belonging to each of the multiple different classes.

[0030] A hierarchical class generation process according to the embodiment will be described with reference to FIG. 3. FIG. 3 is a diagram for describing a hierarchical class generation process according to the embodiment. The horizontal and vertical axes shown in the leftmost, second and third leftmost diagrams of FIG. 3 roughly indicate a feature space of class features. Also, the circles shown in the leftmost, second and third leftmost diagrams of FIG. 3 indicate class features of each of a plurality of different classes. In the leftmost, second and third leftmost diagrams of FIG. 3, the class generation unit 132 generates hierarchical clusters by performing hierarchical clustering on the class features based on the similarity of the class features. In the rightmost diagram of FIG. 3, the class generation unit 132 generates a tree diagram, which is a graph that represents the process of generating hierarchical clusters. For example, the class generation unit 132 may generate a tree diagram that is a tree structure of a binary tree. Note that in FIG. 3, only 12 class features are illustrated for simplicity, but in reality, 13 or more class features may exist.

[0031] In the leftmost diagram of FIG. 3, the class generation unit 132 generates a plurality of clusters having each of a plurality of class features as an element. Hereinafter, a cluster having each of a plurality of class features as an element may be referred to as a lowest cluster. In the leftmost diagram of FIG. 3, the class features and the lowest cluster are indicated by the same circle. In addition, when the class generation unit 132 generates a plurality of lowest clusters, the class generation unit 132 generates a plurality of lowest classes that are classes corresponding to each of the plurality of lowest clusters. Here, each of the plurality of lowest classes corresponds to each of the plurality of different classes described above. In the rightmost diagram of FIG. 3, each of the plurality of white circles shown at the bottom layer of the tree diagram GH1 indicates each of the plurality of lowest classes. In addition, in the rightmost diagram of FIG. 3, for simplicity, the illustration of four lowest classes is omitted and only eight lowest classes are illustrated among the 12 lowest classes. In addition, in reality, 13 or more lowest classes may exist.

[0032] Next, the class generating unit 132 calculates the similarity between the class features for every pair of class features. For example, the class generating unit 132 may calculate the similarity between the class features. Next, the class generating unit 132 combines the lowest-level clusters in order starting from the two lowest-level clusters having as elements two class features with the highest similarity between the class features. For example, the class generating unit 132 generates a plurality of first higher-level clusters C11 to C16 by combining two lowest-level clusters having as elements two class features whose similarity between the class features exceeds a first similarity. The first higher-level cluster is a cluster whose elements are a set of two lowest-level clusters whose elements are two class features whose similarity between the class features exceeds a first similarity. In addition, when the class generating unit 132 generates a plurality of first higher-level clusters C11 to C16, the class generating unit 132 generates a plurality of first higher-level classes L11 to L16 which are classes corresponding to each of the plurality of first higher-level clusters C11 to C16. In the rightmost diagram of Fig. 3, each of the multiple circles shown in the second lowest layer of the tree diagram GH1 represents each of the multiple first upper classes L11 to L14. For simplicity, the circles corresponding to the first upper classes L15 and L16 are omitted in the rightmost diagram of Fig. 3. In reality, seven or more first upper clusters and first upper classes may exist.

[0033] Note that calculating the similarity between class features for every pair of class features corresponds to calculating the distance between every pair of lowest-level clusters. The class generation unit 132 may calculate the distance between every pair of lowest-level clusters. For example, the class generation unit 132 may calculate the Euclidean distance or Manhattan distance between the lowest-level clusters. Next, the class generation unit 132 may generate a plurality of first higher order clusters C11 to C16 by combining two lowest-level clusters whose distance between the lowest-level clusters is less than a first distance.

[0034] In the second and third diagrams from the left in FIG. 3, the class generation unit 132 calculates the similarity between all pairs of first top clusters. For example, the class generation unit 132 may calculate the similarity between the first top clusters based on the average value of the two class features included in the first top clusters. Next, the class generation unit 132 generates multiple second top clusters C21 to C23 by combining two first top clusters whose similarity between the first top clusters exceeds a second threshold. The second top clusters are clusters whose elements are pairs of two first top clusters whose similarity between the first top clusters exceeds the second threshold. In FIG. 3, the class generating unit 132 generates a second higher-level cluster C21 having a pair of first higher-level clusters C11 and C12 as elements, a second higher-level cluster C22 having a pair of first higher-level clusters C13 and C14 as elements, and a second higher-level cluster C23 having a pair of first higher-level clusters C15 and C16 as elements. When the class generating unit 132 generates a plurality of second higher-level clusters C21 to C23, the class generating unit 132 generates a plurality of second higher-level classes L21 to L23 which are classes corresponding to the plurality of second higher-level clusters C21 to C23, respectively. In the rightmost diagram of FIG. 3, a plurality of circles shown in the third layer from the bottom of the tree diagram GH1 respectively represent a plurality of second higher-level classes L21 to L22. For simplicity, the circle corresponding to the second higher-level class L23 is omitted in the rightmost diagram of FIG. 3. In reality, four or more second higher-level clusters and classes may exist.

[0035] Note that calculating the similarity between all pairs of first top clusters corresponds to calculating the distance between all pairs of first top clusters. The class generation unit 132 may calculate the distance between the first top clusters for all pairs of first top clusters. For example, the class generation unit 132 may calculate the Euclidean distance or Manhattan distance between the first top clusters. Next, the class generation unit 132 may generate a plurality of second top clusters C21 to C23 by combining two first top clusters whose distance between them is less than the second distance.

[0036] As described above, the class generation unit 132 generates higher classes that correspond to each of the hierarchical clusters based on the class features hierarchically clustered based on the similarity of the class features, which are features related to data belonging to each of the multiple different classes and are representative values ​​of each of the multiple different classes, and are classes higher than the multiple different classes. Specifically, the class generation unit 132 generates the higher class by using the average value of the features of the data belonging to each of the multiple different classes as the representative value. Note that the class generation unit 132 may generate the higher class by using the median or mode of the features of the data belonging to each of the multiple different classes as the representative value.

[0037] In addition, the class generation unit 132 generates higher-level classes by generating a plurality of first higher-level classes which are classes corresponding to each of a plurality of first higher-level clusters which are clusters generated by combining each of a plurality of lowest-level clusters which are clusters having each of the plurality of class features as elements based on the similarity between the plurality of class features, and a plurality of second higher-level classes which are classes corresponding to each of a plurality of second higher-level clusters which are clusters generated by combining each of the plurality of first higher-level clusters based on the similarity between the plurality of first higher-level clusters.

[0038] Furthermore, the class generation unit 132 may perform hierarchical clustering on the class features until the number of newly generated top clusters becomes several to 10 or less. In this way, the class generation unit 132 hierarchically clusters a plurality of class features until the number of newly generated top clusters becomes below a predetermined threshold. For example, the class generation unit 132 may perform hierarchical clustering on the class features until the number of newly generated top clusters becomes one. When the class generation unit 132 performs hierarchical clustering on the class features until the number of newly generated top clusters becomes one, the class generation unit 132 may set the top class corresponding to the last generated cluster as the top class.

[0039] (Model generation unit 133) The model generation unit 133 generates a classification model, which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that belong to a lower class, for each upper class. A process of generating a hierarchical classification model according to an embodiment will be described with reference to FIG. 4. FIG. 4 is a diagram for describing a process of generating a hierarchical classification model according to an embodiment. The model generation unit 133 generates classification models in a number corresponding to the number of branch points included in the tree diagram generated by the class generation unit 132. Specifically, the model generation unit 133 may acquire classification models in a number corresponding to the number of branch points included in the tree diagram generated by the class generation unit 132. For example, the model generation unit 133 may acquire a classification model that is smaller than a large classification model. For example, the model generation unit 133 may acquire a classification model such as MobileNet (Andrew G. Howard et al., 2017) or ShuffleNet (Xiangyu Zhang et al., 2018). In the diagram at the left end of Fig. 4, the model generation unit 133 may acquire classification models corresponding to branch points P11, P12 (not shown), P13, and P21 included in the tree diagram GH2 generated by the class generation unit 132. For example, the model generation unit 133 may acquire four classification models from the storage unit 120 that stores information related to classification models. Note that the diagram at the left end of Fig. 4 shows the tree diagram GH2 generated by the class generation unit 132, which is composed of layer B, which is the lowest layer, and layer A, which is the layer one layer above layer B. Note that the tree diagram generated by the class generation unit 132 may be composed of three or more layers.

[0040] As described above, each of the branch points included in the tree diagram corresponds to each of the upper classes included in the hierarchical classes. In Fig. 4, branch point P21 corresponds to the top class, and each of branch points P11 to P13 corresponds to each of the three upper classes 1 to 3. Fig. 4 shows a hierarchical class including six lowest classes 4 to 9, three upper classes 1 to 3, and one highest class generated by class generation unit 132. The number of the lowest classes generated by class generation unit 132 may be seven or more. The number of the upper classes generated by class generation unit 132 may be four or more.

[0041] Furthermore, when the model generation unit 133 acquires classification models in a number corresponding to the number of branch points included in the tree diagram, the model generation unit 133 generates classification models corresponding to each of the upper classes corresponding to each of the branch points included in the tree diagram. In the central diagram of Fig. 4, the model generation unit 133 may generate classification model A_1 corresponding to the top class corresponding to branch point P21 included in the tree diagram GH2, and classification models B_1 to B_3 corresponding to each of upper classes 1 to 3 corresponding to branch points P11 to P13 included in the tree diagram GH2.

[0042] Specifically, the model generation unit 133 generates a classification model A_1 trained based on a first learning data set that is a set of data sampled from each of three upper classes 1 to 3 that belong to a lower level of the highest class and a label corresponding to each of the three upper classes 1 to 3. For example, when data included in the first learning data set is input to the classification model A_1 as input information, the model generation unit 133 generates a classification model A_1 trained to output a label corresponding to the data as output information. For example, when data included in the first learning data set is input to the classification model A_1 as input information, the model generation unit 133 may generate a classification model A_1 trained to estimate a probability value of data being classified into each of the three upper classes 1 to 3, and output a label corresponding to the upper class with the largest estimated probability value as output information.

[0043] Furthermore, the model generation unit 133 generates a classification model B_1 trained based on a second learning data set that is a set of data sampled from each of the two lowest classes 4 and 5 that belong to the lower level of the higher class 1 and labels corresponding to each of the two lowest classes 4 and 5. For example, when data included in the second learning data set is input to the classification model B_1 as input information, the model generation unit 133 generates a classification model B_1 trained to output a label corresponding to the data as output information. For example, when data included in the second learning data set is input to the classification model B_1 as input information, the model generation unit 133 may generate a classification model B_1 trained to estimate a probability value of data being classified into each of the two lowest classes 4 and 5, and output a label corresponding to the lowest class with the largest estimated probability value as output information.

[0044] Furthermore, the model generation unit 133 generates a classification model B_2 trained based on a third learning data set that is a set of data sampled from each of the two lowest classes 6 and 7 that belong to the lower level of the higher class 2 and labels corresponding to each of the two lowest classes 6 and 7. For example, when data included in the third learning data set is input to the classification model B_2 as input information, the model generation unit 133 generates a classification model B_2 trained to output a label corresponding to the data as output information. For example, when data included in the third learning data set is input to the classification model B_2 as input information, the model generation unit 133 may generate a classification model B_2 trained to estimate a probability value of data being classified into each of the two lowest classes 6 and 7, and output a label corresponding to the lowest class with the largest estimated probability value as output information.

[0045] Furthermore, the model generation unit 133 generates a classification model B_3 trained based on a fourth learning data set that is a set of data sampled from each of the two lowest classes 8 and 9 that belong to the lower level of the upper class 3 and labels corresponding to each of the two lowest classes 8 and 9. For example, when data included in the fourth learning data set is input to the classification model B_3 as input information, the model generation unit 133 generates a classification model B_3 trained to output a label corresponding to the data as output information. For example, when data included in the fourth learning data set is input to the classification model B_3 as input information, the model generation unit 133 may generate a classification model B_3 trained to estimate a probability value of the data being classified into each of the two lowest classes 8 and 9, and output a label corresponding to the lowest class with the largest estimated probability value as output information.

[0046] As described above, when data sampled from each of a plurality of lower classes is input to the machine learning model as input information, the model generation unit 133 generates a classification model for each upper class, which is a machine learning model trained to output a lower label, which is a label corresponding to the lower class to which the data belongs, as output information. For example, when data sampled from each of a plurality of lower classes is input to the machine learning model as input information, the model generation unit 133 generates a classification model for each upper class, which is a machine learning model trained to estimate a probability value for classifying data into each of the lower classes and output a lower label, which is a label corresponding to the lower class with the largest estimated probability value, as output information. In this way, the model generation unit 133 generates a first hierarchical classification model, which is a collection of a plurality of classification models having a hierarchical nature corresponding to the hierarchical structure of the hierarchical classes.

[0047] A learning data set of a hierarchical classification model according to an embodiment will be described with reference to FIG. 5. FIG. 5 is a diagram for describing a learning data set of a hierarchical classification model according to an embodiment. The model generation unit 133 generates a classification model, which is a machine learning model trained based on a learning data set that is a pair of data sampled from each of a plurality of lower classes so that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs, for each upper class. In FIG. 5, the model generation unit 133 generates a second learning data set that is a pair of data sampled from each of the lowest classes 4 and 5 so that the number of data sampled from each of the lowest classes 4 and 5 described in FIG. 4 is as equal as possible, and each of the lowest labels 4 and 5 corresponding to each of the lowest classes 4 and 5, as a learning data set of the classification model B_1 corresponding to the upper class 1 described in FIG. 4.

[0048] (Estimation part 134) A class estimation process according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram for explaining a class estimation process according to the embodiment. The acquisition unit 131 acquires a first hierarchical classification model generated by the model generation unit 133 and processing target data to be processed. In Fig. 6, the acquisition unit 131 acquires the top classification model A_1 included in the first hierarchical classification model generated by the model generation unit 133, the first top classification models B_1 to B_3 belonging to the lower level of the top classification model A_1, the second top classification models C_1 to C_20 belonging to the lower level of the first top classification models B_1 to B_3, ..., and the Nth (N is a natural number equal to or greater than 3) top classification models X_1 to X_100 belonging to the lower level of the (N-1)th top classification model (not shown), and input data to be processed.

[0049] The estimation unit 134 selects the top classification model from the first hierarchical classification models acquired by the acquisition unit 131. In FIG. 6, the estimation unit 134 may select the top classification model A_1 from the first hierarchical classification models acquired by the acquisition unit 131. Next, the estimation unit 134 uses the selected top classification model to estimate a first top class (hereinafter, may be referred to as the first top class) in which the data to be processed is classified among a plurality of top classes below the top class. In FIG. 6, the estimation unit 134 may use the selected top classification model A_1 to estimate a first top class in which the input data is classified among three top classes below the top class. For example, the estimation unit 134 may input the input data to the top classification model A_1 and obtain the first top class corresponding to the label output from the top classification model A_1 as an estimation result.

[0050] Next, the estimation unit 134 selects a first higher-level classification model corresponding to the first higher-level class from among the classification models corresponding to each of the multiple higher-level classes subordinate to the highest-level class, based on the estimation result of the highest-level classification model. In Fig. 6, the estimation unit 134 may select the first higher-level classification model B_3 corresponding to the first higher-level class from among the first higher-level classification models B_1 to B_3 corresponding to each of the multiple higher-level classes subordinate to the highest-level class.

[0051] Next, the estimation unit 134 uses the selected first higher-level classification model to estimate a second higher-level class (hereinafter, may be referred to as a second higher-level class) into which the data to be processed is classified, among a plurality of higher-level classes below the first higher-level class. In FIG. 6, the estimation unit 134 may use the selected first higher-level classification model B_3 to estimate a second higher-level class into which the input data is classified, among 20 higher-level classes below the first higher-level class. For example, the estimation unit 134 may input the input data to the first higher-level classification model B_3, and obtain the second higher-level class corresponding to the label output from the first higher-level classification model B_3 as an estimation result.

[0052] Next, the estimation unit 134 selects a second higher class classification model corresponding to the second higher class from among the classification models corresponding to each of the multiple higher classes subordinate to the first higher class based on the estimation result of the first higher class classification model. In Fig. 6, the estimation unit 134 may select a second higher class classification model C_11 corresponding to the second higher class from among the second higher class classification models C_1 to C_20 corresponding to each of the multiple higher classes subordinate to the first higher class.

[0053] Furthermore, the estimation unit 134 selects an Nth higher classification model corresponding to the Nth higher class from among the classification models corresponding to each of the multiple higher classes subordinate to the (N-1)th higher class based on the estimation result of the (N-1)th higher classification model. In Fig. 6, the estimation unit 134 may select an Nth higher classification model X_50 corresponding to the Nth higher class from among the Nth higher classification models X_1 to X_100 corresponding to each of the multiple higher classes subordinate to the (N-1)th higher class.

[0054] Next, the estimation unit 134 uses the selected Nth highest classification model to estimate the lowest class into which the data to be processed is classified, among the multiple lowest classes that belong to the lower level of the Nth highest class. In FIG. 6, the estimation unit 134 may use the selected Nth highest classification model X_50 to estimate the lowest class into which the data to be processed is classified, among the 200 lowest classes that belong to the lower level of the Nth highest class. For example, the estimation unit 134 may input the input data to the Nth highest classification model X_50, and obtain the lowest class corresponding to the label output from the Nth highest classification model X_50 (corresponding to the output data shown in FIG. 6) as the estimation result. In this way, the estimation unit 134 repeatedly executes the selection process and the estimation process until the class into which the data to be processed is classified matches one of the multiple different classes that are the lowest class.

[0055] As described above, the estimation unit 134 estimates a first lower class into which the data to be processed is classified using a higher classification model that is a classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest class, selects a lower classification model that is a classification model corresponding to the first lower class from among classification models corresponding to each of a plurality of lower classes below the predetermined higher class based on the estimation result of the higher classification model, and estimates a second lower class into which the data to be processed is classified using the selected lower classification model. Furthermore, the estimation unit 134 repeatedly executes a process of selecting a lower classification model from among classification models corresponding to each of a plurality of lower classes based on the estimation result of the higher classification model, and estimating a second lower class using the selected lower classification model, until the data to be processed is classified into one of a plurality of different classes.

[0056] 3. Modifications The processing according to the above-described embodiment may be implemented in various different forms other than the above embodiment.

[0057] In the above-described embodiment, a case has been described in which the model generation unit 133 generates a first hierarchical classification model that is a collection of classification models that classify data to be processed into two or more classes (hereinafter, may be referred to as "normal multi-class classification model"). In contrast, in the modified example, a case will be described in which the model generation unit 133 generates a second hierarchical classification model that is a collection of classification models that classify data to be processed into three or more classes including an unknown class (hereinafter, may be referred to as "multi-class classification model capable of detecting an unknown class") instead of a two-class classification model.

[0058] In the above-described embodiment, the estimation unit 134 adopts only one estimation result by a higher-level classification model until the data to be processed is classified into one of the lowest classes, selects one classification model from among the classification models corresponding to each of the lower classes based on the one estimation result, and repeatedly executes the process of estimating the lower class using the selected one classification model. In contrast, in the modified example, the estimation unit 134 adopts two or more estimation results by a higher-level classification model until the data to be processed is classified into one of the lowest classes, selects two or more classification models from among the classification models corresponding to each of the lower classes based on the two or more estimation results, and repeatedly executes the process of estimating each of the lower classes using each of the selected two or more classification models. In the modified example, the estimation unit 134 determines to stop the process of selecting the classification model and subsequent processes when the estimation results of each of the selected two or more classification models satisfy a predetermined condition.

[0059] First, a case where the estimation result of the lower class becomes incorrect by adopting only one estimation result by a higher-level classification model will be described with reference to FIG. 7. FIG. 7 is a diagram for describing a case where the estimation result of the lower class becomes incorrect by adopting only one estimation result by a higher-level classification model. In FIG. 7, the correct classification destination of the input data input to the classification model is the lowest class 9. At this time, when the input data is input to the highest-level classification model A_1, the estimation unit 134 estimates the probability value of the data being classified into each of the higher classes 1 to 3, and outputs the label corresponding to the higher class 1 having the maximum estimated probability value (0.5 in FIG. 7) as output information. Next, when the input data is input to the classification model B_1 corresponding to the higher class 1, the estimation unit 134 estimates the probability value of the data being classified into each of the lowest classes 4 and 5, and outputs the label corresponding to the lowest class 5 having the maximum estimated probability value (0.6 in FIG. 7) as output information. Thus, in FIG. 7, the estimation result of the lower class becomes incorrect by adopting only one estimation result by a higher-level classification model.

[0060] Next, a case where the estimation result of the lower class becomes the correct answer by adopting a plurality of estimation results by the upper classification model will be described with reference to FIG. 8. FIG. 8 is a diagram for describing a case where the estimation result of the lower class becomes the correct answer by adopting a plurality of estimation results by the upper classification model. In FIG. 8, as in FIG. 7, the correct classification destination of the input data input to the classification model is the lowest class 9. At this time, when the input data is input to the highest classification model A_1, the estimation unit 134 estimates the probability value that the data is classified into each of the upper classes 1 to 3, and selects the classification model B_1 and the classification model B_3 corresponding to the two upper classes 1 and 3, respectively, in descending order of the estimated probability value. Next, the estimation unit 134 uses the two selected classification models B_1 and B_3 to estimate the probability value that the input data is classified into each of the lowest classes belonging to the lower levels of the upper classes 1 and 3 corresponding to the two selected classification models B_1 and B_3. For example, when input data is input to classification model B_1, the estimation unit 134 estimates the probability value that the data will be classified into the lowest class 4 or 5, and outputs the label corresponding to the lowest class 5, which has the maximum estimated probability value (0.6 in FIG. 8), as output information. When input data is input to classification model B_3, the estimation unit 134 estimates the probability value that the data will be classified into the lowest class 8 or 9, and outputs the label corresponding to the lowest class 9, which has the maximum estimated probability value (0.9 in FIG. 8), as output information.

[0061] As described above, the estimation unit 134 estimates the probability value that the data to be processed will be classified into each of a plurality of lower classes using a higher-level classification model, selects a plurality of classification models corresponding to each of a predetermined number of lower classes (two or more) in descending order of probability value from among the classification models corresponding to each of the plurality of lower classes based on the estimation result of the higher-level classification model, and estimates the probability value that the data to be processed will be classified into each of the classes lower than the lower classes corresponding to each of the selected classification models using the selected plurality of classification models. This allows the information processing device 100 to prevent the estimation result of the lower classes from being incorrect by adopting only one estimation result by the higher-level classification model.

[0062] Also, in FIG. 8, the estimation unit 134 may calculate the product of the probability values ​​of classification of each class in the classification paths (1) and (2) from the highest class to the two lowest classes 5 and 9. For example, the estimation unit 134 may calculate 0.5×0.6=0.3, which is the product of two probability values ​​0.5 and 0.6 corresponding to the classification path (1). Also, the estimation unit 134 may calculate 0.4×0.9=0.36, which is the product of two probability values ​​0.4 and 0.9 corresponding to the classification path (2). Next, based on a comparison between 0.3 and 0.36, which are the product values ​​of the probability values ​​in each of the classification paths (1) and (2), the estimation unit 134 may adopt the classification path (2) with the largest value of the product of the probability values ​​of 0.36, and estimate that the input data will be classified into the lowest class 9. In this way, in FIG. 8, the estimation result of the lower class becomes the correct answer by adopting multiple estimation results by the higher-level classification models.

[0063] As described above, when the data to be processed is classified into two or more classes among a plurality of different classes, the estimation unit 134 estimates that the data to be processed will be classified into the class corresponding to the classification path having the largest value of the product of the probability values, based on a comparison of the values ​​of the product of the probability values ​​of classification into each class in the classification path from the highest class to the class corresponding to each of the two or more classes among the plurality of different classes.

[0064] Next, a normal multi-class classification model and a multi-class classification model capable of detecting an unknown class will be described with reference to FIG. 9. FIG. 9 is a diagram for describing a normal multi-class classification model and a multi-class classification model capable of detecting an unknown class. The left side of FIG. 9 shows a normal multi-class classification model. The normal multi-class classification model estimates a probability value that input data is classified into each of a plurality of classes learned in advance, regardless of the type of input data. The normal multi-class classification model shown on the left side of FIG. 9 (hereinafter, referred to as a "first multi-class classification model") is a classification model that has been trained to classify data to be classified into either a dog class or a cat class. In this case, when input data that should originally be classified into a bird class is input to the first multi-class classification model, the first multi-class classification model estimates the probability that the input data is classified into the dog class to be 70% and the probability that the input data is classified into the cat class to be 30%.

[0065] The right side of FIG. 9 shows a multi-class classification model capable of detecting an unknown class. The multi-class classification model capable of detecting an unknown class can classify input data that is not classified into a plurality of previously learned classes into an unknown class. The multi-class classification model capable of detecting an unknown class shown on the right side of FIG. 9 (hereinafter, referred to as a "second multi-class classification model") is a classification model that has been trained to classify data to be classified into one of a dog class, a cat class, or an unknown class. In this case, when input data to be classified into a bird class is input to the second multi-class classification model, the second multi-class classification model estimates that the probability that the input data is classified into a dog class is 10%, the probability that the input data is classified into a cat class is 15%, and the probability that the input data is classified into an unknown class is 75%. In this way, the multi-class classification model capable of detecting an unknown class can classify data to be processed that is to be classified into a class other than a plurality of previously learned classes into an unknown class.

[0066] Therefore, in the modified example, the model generation unit 133 generates a classification model, which is a machine learning model that classifies data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, for each upper class. The model generation unit 133 generates a classification model, which is a machine learning model that classifies data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, for each upper class. For example, the model generation unit 133 may acquire a multi-class classification model capable of detecting an unknown class as a classification model in a number corresponding to the number of branch points included in the tree diagram generated by the class generation unit 132. Then, the model generation unit 133 may generate a classification model in the same manner as described in FIG. 4 and FIG. 5. For example, when data sampled from each of the plurality of lower classes is input to the machine learning model as input information, the model generation unit 133 generates a classification model, which is a machine learning model that has been trained to estimate a probability value at which data is classified into each of the plurality of lower classes, and output the estimated probability value as output information, for each upper class. In this way, the model generation unit 133 generates a second hierarchical classification model that is a collection of multiple classification models having a hierarchy corresponding to the hierarchical structure of the hierarchical classes. The model generation unit 133 also generates a second hierarchical classification model that is a collection of multi-class classification models capable of detecting unknown classes. This allows the information processing device 100 to stop processing when the processing target data is classified into an unknown class by a higher-level classification model, for example, and therefore prevents the estimation result of a lower class from being incorrect due to an error in the estimation result of the higher-level classification model.

[0067] Next, a class estimation process according to a modified example will be described with reference to FIG. 10. FIG. 10 is a diagram for explaining a class estimation process according to a modified example. In FIG. 10, the description of the contents overlapping with FIG. 6 will be omitted. The acquisition unit 131 acquires a classification model, which is a machine learning model that classifies data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, generated for each upper class, and data to be processed. In FIG. 10, the acquisition unit 131 acquires a second hierarchical classification model generated by the model generation unit 133 and data to be processed. In Figure 10, the acquisition unit 131 acquires the top-level classification model A_1 included in the second hierarchical classification model generated by the model generation unit 133, the first top-level classification models B_1 to B_3 subordinate to the top-level classification model A_1, the second top-level classification models C_1 to C_20 subordinate to the first top-level classification models B_1 to B_3, ..., and the Nth (N is a natural number greater than or equal to 3) top-level classification models X_1 to X_100 subordinate to the (N-1)th top-level classification model (not shown), as well as the input data to be processed.

[0068] In FIG. 10, the estimation unit 134 may use the selected top-level classification model A_1 to estimate the probability values ​​of the input data being classified into three upper classes below the top-level class and into an unknown class. Based on the estimation result of the top-level classification model, the estimation unit 134 determines whether to stop the process after the process of selecting a plurality of classification models corresponding to each of a predetermined number of lower classes, which are selected from the classification models corresponding to each of the plurality of lower classes in order from the one with the highest probability value. Here, the predetermined number of two or more may be a number smaller than the number of the plurality of lower classes. Hereinafter, the process of determining whether to stop the process may be referred to as a process continuation determination or a continuation determination process. For example, when the classification target data is classified into an unknown class, when the difference in the probability values ​​of the processing target data being classified into each of the plurality of lower classes or the unknown class exceeds a first threshold value, or when the value of the product of the probability values ​​of the processing target data being classified into each class in the classification path from the top-level class to the classification into a class corresponding to each of two or more classes among the plurality of different classes falls below a second threshold value, the estimation unit 134 determines to stop the process after the process of selecting a plurality of classification models corresponding to each of the predetermined number of lower classes. In FIG. 10, the estimation unit 134 determines to continue the processing as a result of the processing continuation determination.

[0069] Furthermore, when the estimation unit 134 decides to continue the process, it selects a plurality of classification models corresponding to each of a predetermined number of lower classes, two or more, from among the classification models corresponding to each of the plurality of lower classes, in descending order of probability value, based on the estimation result of the higher-level classification model. Hereinafter, the process of selecting a plurality of classification models may be referred to as a multiple selection process. In FIG. 10, the estimation unit 134 may select two first higher-level classification models B_2 and B_3 corresponding to each of the two first higher-level classes, in descending order of probability value, from among the first higher-level classification models B_1 to B_3 corresponding to each of the plurality of first higher-level classes below the highest-level class, based on the estimation result of the highest-level classification model. Next, the estimation unit 134 may estimate the probability value of the input data being classified into each of the plurality of higher-level classes and the unknown class below each of the two first higher-level classes, using the two selected first higher-level classification models B_2 and B_3, respectively. The estimation unit 134 repeatedly executes the continuation determination process, the multiple selection process, and the estimation process until the class into which the processing target data is classified matches any one of a plurality of different classes that is the lowest class.

[0070] Next, an example of a case where the estimation process according to the modified example is stopped will be described with reference to FIG. 11. FIG. 11 is a diagram for describing an example of a case where the estimation process according to the modified example is stopped. In FIG. 11, the estimation unit 134 determines to stop the process after the process of selecting a plurality of classification models corresponding to each of a predetermined number of lower classes because the difference in the probability value that the processing target data is classified into each of a plurality of lower classes or an unknown class exceeds a first threshold. Note that, when the estimation unit 134 determines to stop the process before the class into which the processing target data is classified matches any one of a plurality of different classes that is the lowest class, the estimation unit 134 may relax the condition of the continuation determination process and execute the process again from the beginning. For example, the estimation unit 134 may increase the predetermined number when selecting a plurality of classification models corresponding to each of a predetermined number of lower classes, two or more, in order from the one with the highest probability value, from among the classification models corresponding to each of the plurality of lower classes. Furthermore, when the classification target data is classified into an unknown class, the estimation unit 134 may increase the first threshold when stopping the process when the difference in the probability value that the processing target data is classified into each of a plurality of lower classes or an unknown class exceeds the first threshold. In addition, the estimation unit 134 may reduce the second threshold value used to stop processing when the product of the probability values ​​of classification into each class in the classification path from the top class to the classes corresponding to each of two or more of the multiple different classes falls below the second threshold value.

[0071] [4. Effects] As described above, the information processing device 100 according to the embodiment includes a class generation unit 132 and a model generation unit 133. The class generation unit 132 generates upper classes that are classes higher than the multiple different classes and correspond to each of the hierarchical clusters based on the multiple class features that are hierarchically clustered based on the similarity of the multiple class features that are feature amounts related to data respectively belonging to multiple different classes and are representative values ​​of the multiple different classes. The model generation unit 133 generates a classification model, which is a machine learning model that classifies the data to be classified into one of multiple lower classes that belong to the upper class, for each upper class.

[0072] Thereby, the information processing device 100 can increase the number of classification classes while keeping the model size of each of the multiple classification models generated for each upper class small. Therefore, the information processing device 100 can realize the miniaturization of the classification model that enables detailed classification. Furthermore, since the information processing device 100 can realize the miniaturization of the classification model that enables detailed classification, it is possible to perform detailed classification using the miniaturized classification model. For example, the information processing device 100 can perform detailed classification of behavior using a miniaturized behavior recognition model. Furthermore, since the information processing device 100 can perform detailed classification using a miniaturized classification model, it is possible to contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilience, innovate and innovate."

[0073] Moreover, the class generating unit 132 generates a higher class by using the average value of the feature amounts of data belonging to each of a plurality of different classes as a representative value.

[0074] This enables the information processing device 100 to appropriately generate a higher class based on the average values ​​of the feature amounts of data belonging to a plurality of different classes.

[0075] Furthermore, the class generating unit 132 hierarchically clusters a plurality of class features until the number of newly generated clusters falls below a predetermined threshold.

[0076] This enables the information processing device 100 to generate a desired number of higher classes.

[0077] In addition, the class generation unit 132 generates higher-level classes by generating a plurality of first higher-level classes which are classes corresponding to each of a plurality of first higher-level clusters which are clusters generated by combining each of a plurality of lowest-level clusters which are clusters having each of the plurality of class features as elements based on the similarity between the plurality of class features, and a plurality of second higher-level classes which are classes corresponding to each of a plurality of second higher-level clusters which are clusters generated by combining each of the plurality of first higher-level clusters based on the similarity between the plurality of first higher-level clusters.

[0078] This allows the information processing device 100 to appropriately generate the first higher class based on the similarity of the class features. Also, the information processing device 100 can appropriately generate the second higher class based on the similarity of the first higher cluster.

[0079] In addition, the class generation unit 132 generates a tree diagram, which is a graph expressing the process of generating hierarchical clusters. The model generation unit 133 generates classification models in a number corresponding to the number of branch points included in the tree diagram.

[0080] This enables the information processing device 100 to generate a classification model corresponding to the number of generated upper classes.

[0081] In addition, the model generation unit 133 generates a classification model for each higher class, which is a machine learning model trained based on a learning dataset that is a pair of data sampled from each of the multiple lower classes so that the number of data sampled from each of the multiple lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs.

[0082] This enables the information processing device 100 to prevent bias in the data of lower classes included in the learning data set, thereby enabling generation of a highly accurate classification model.

[0083] Furthermore, when data is input as input information to the machine learning model, the model generation unit 133 generates, for each upper class, a classification model that is a machine learning model trained to output lower labels as output information.

[0084] This enables the information processing device 100 to appropriately generate a classification model.

[0085] In addition, the model generation unit 133 generates, for each upper class, a classification model that is a machine learning model that classifies the data to be classified into one of a plurality of lower classes or an unknown class that is not one of the plurality of lower classes.

[0086] As a result, the information processing device 100 can stop processing, for example, when the data to be processed is classified into an unknown class by a higher-level classification model, thereby preventing the estimation result of a lower class from being incorrect due to an error in the estimation result of the higher-level classification model.

[0087] The information processing device 100 also includes an acquisition unit 131 and an estimation unit 134. The acquisition unit 131 acquires a classification model, which is a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes belonging to a higher class that is a class higher than the plurality of different classes, and which corresponds to each of hierarchical clusters based on a plurality of class features that are feature amounts related to data belonging to a plurality of different classes and are representative values ​​of each of the plurality of different classes, based on similarities of the plurality of class features, and acquires data to be processed that is a processing target. The estimation unit 134 estimates a first lower class into which the data to be processed is classified using a higher classification model that is a classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest class, selects a lower classification model that is a classification model corresponding to the first lower class from among classification models corresponding to each of a plurality of lower classes belonging to a lower class below the predetermined higher class, based on the estimation result of the higher classification model, and estimates a second lower class into which the data to be processed is classified using the selected lower classification model.

[0088] This enables the information processing device 100 to perform detailed classification using a miniaturized classification model. For example, the information processing device 100 can perform detailed classification of behavior using a miniaturized behavior recognition model. In addition, since the information processing device 100 can perform detailed classification using a miniaturized classification model, it can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs) "Build resilient infrastructure and promote inclusive and sustainable industrialization."

[0089] Furthermore, the estimation unit 134 repeatedly executes the process of selecting a lower classification model from among the classification models corresponding to each of the multiple lower classes based on the estimation result of the higher classification model, and estimating a second lower class using the selected lower classification model, until the processing target data is classified into one of multiple different classes.

[0090] This enables the information processing device 100 to perform detailed classification using a compact classification model.

[0091] In addition, the estimation unit 134 estimates a probability value that the data to be processed will be classified into each of a plurality of lower classes using a higher-level classification model, and based on the estimation result of the higher-level classification model, selects a plurality of classification models corresponding to each of a predetermined number of two or more lower classes in order from the classification models corresponding to each of the plurality of lower classes in descending order of probability value from among the classification models corresponding to each of the plurality of lower classes, and estimates a probability value that the data to be processed will be classified into each of the classes further below the lower classes corresponding to each of the selected plurality of classification models.

[0092] In this way, the information processing device 100 can prevent the estimation result of a lower class from being incorrect by adopting only one estimation result based on a higher-level classification model.

[0093] Furthermore, when the data to be processed is classified into two or more of a plurality of different classes, the estimation unit 134 estimates that the data to be processed will be classified into the class corresponding to the classification path having the largest value of the product of the probability values, based on a comparison of the values ​​of the product of the probability values ​​of classification into each class in the classification path from the highest class to the class corresponding to each of the two or more classes among the plurality of different classes.

[0094] This allows the information processing device 100 to obtain appropriate classification results even when estimation results based on a plurality of classification models are adopted.

[0095] The acquisition unit 131 acquires the classification model, which is a machine learning model that classifies the classification target data into one of a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, generated for each upper class, and the processing target data. The estimation unit 134 determines whether to stop processing after the processing of selecting, from the classification models corresponding to each of the plurality of lower classes, a plurality of classification models corresponding to each of a predetermined number of lower classes, based on the estimation result of the upper classification model.

[0096] As a result, the information processing device 100 can stop processing, for example, when the data to be processed is classified into an unknown class, thereby preventing the estimation result of a lower class from being incorrect due to an error in the estimation result of a higher-level classification model.

[0097] Furthermore, the estimation unit 134 determines to discontinue processing subsequent to the process of selecting a plurality of classification models corresponding to each of a predetermined number of lower classes when the data to be classified is classified into an unknown class, when a difference in the probability values ​​that the data to be processed will be classified into each of the plurality of lower classes or the unknown class exceeds a first threshold value, or when a value of the product of the probability values ​​that the data to be processed will be classified into each of the classes along the classification path from the highest class to a class corresponding to each of two or more classes among the plurality of different classes falls below a second threshold value.

[0098] As a result, the information processing device 100 can stop processing if an error in the estimation result of a higher-level classification model is likely to occur, thereby preventing the estimation result of a lower class from being incorrect due to an error in the estimation result of a higher-level classification model.

[0099] [5. Hardware Configuration] Moreover, the information processing device 100 according to the embodiment described above is realized by a computer 1000 having a configuration as shown in Fig. 12, for example. Fig. 12 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0100] The CPU 1100 operates and controls each unit based on a program stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, programs that depend on the hardware of the computer 1000, and the like.

[0101] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0102] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0103] The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0104] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0105] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0106] [6. Other] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0107] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0108] For example, in the above-mentioned embodiment, the case where the information processing device 100 has the acquisition unit 131, the class generation unit 132, the model generation unit 133, and the estimation unit 134 as functional units has been described, but each unit can be configured to be distributed in a separate device. For example, the information processing device 100 can have the acquisition unit 131 and the estimation unit 134 as functional units as an information processing device that estimates a class into which the processing target data is classified. In addition, an information processing device other than the information processing device 100 (hereinafter, referred to as a "generation device") can have the acquisition unit 131, the class generation unit 132, and the model generation unit 133 as functional units as an information processing device that generates a classification model. At this time, the information processing device 100 and the generation device may be connected to various networks by wire or wirelessly, and may transmit and receive information to each other. For example, the information processing device 100 may receive information about the classification model generated by the generation device from the generation device. In addition, the information processing device 100 may estimate a class into which the processing target data is classified using the classification model received from the generation device.

[0109] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory. [Explanation of symbols]

[0110] 100 Information processing device 110 Communications Department 120 Storage section 130 Control section 131 Acquisition Department 132 Class Generation Unit 133 Model Generation Unit 134 Estimation Department

Claims

1. a class generation unit that generates a higher class that is a higher class than the plurality of different classes, the higher class corresponding to each of the hierarchical clusters based on the plurality of class features hierarchically clustered based on a similarity between the plurality of class features relating to data respectively belonging to a plurality of different classes and being representative values ​​of the plurality of different classes; a model generation unit that generates a classification model, which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that are lower than the upper class, for each of the upper classes; Equipped with The model generation unit generating, for each of the upper classes, the classification model, which is the machine learning model trained based on a training data set that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs; Information processing device.

2. The class generation unit generating the higher class by using an average value of the feature amounts of the data belonging to each of the plurality of different classes as the representative value; The information processing device according to claim 1 .

3. The class generation unit hierarchically clustering the plurality of class features until the number of newly generated clusters falls below a predetermined threshold. The information processing device according to claim 1 .

4. The class generation unit generating a plurality of first higher order classes which are classes corresponding to a plurality of first higher order clusters which are clusters generated by combining each of a plurality of lowest-level clusters which are clusters having each of the plurality of class features as elements based on the similarities of the plurality of class features, and generating a plurality of second higher order classes which are classes corresponding to a plurality of second higher order clusters which are clusters generated by combining each of the plurality of first higher order clusters based on the similarities of the plurality of first higher order clusters, thereby generating the higher order classes; The information processing device according to claim 1 .

5. The class generation unit generating a tree diagram, which is a graph representing the process of generating the hierarchical clusters; The model generation unit generating the classification models in a number corresponding to the number of branch points included in the tree diagram; The information processing device according to claim 1 .

6. The model generation unit generating, for each of the upper classes, the classification model, which is the machine learning model trained to output the lower label as output information when the data is input as input information to the machine learning model; The information processing device according to claim 1 .

7. The model generation unit generating, for each of the upper classes, the classification model, which is the machine learning model that classifies the data to be classified into one of the plurality of lower classes or an unknown class that is not one of the plurality of lower classes; The information processing device according to claim 1 .

8. a classification model which is a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a higher class that is a higher class than the plurality of different classes, the lower class being a class corresponding to each of the hierarchical clusters based on a plurality of class features hierarchically clustered based on a similarity between the plurality of class features that are representative values ​​of the plurality of different classes, the classification model being a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a lower class that is a higher class than the plurality of different classes, the classification model being trained based on a learning dataset that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs, and an acquisition unit that acquires the classification model which is the machine learning model generated for each of the higher classes, and processing target data to be processed; an estimation unit that estimates a first lower class into which the data to be processed is classified using a higher-level classification model, which is the classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest-level class, selects a lower-level classification model, which is the classification model corresponding to the first lower class, from among the classification models corresponding to each of a plurality of lower classes below the predetermined higher-level class based on a result of the estimation of the higher-level classification model, and estimates a second lower class into which the data to be processed is classified using the selected lower-level classification model; An information processing device comprising:

9. The estimation unit is repeatedly performing a process of selecting the lower classification model from among the classification models corresponding to each of the plurality of lower classes based on an estimation result of the higher classification model, and estimating the second lower class using the selected lower classification model, until the processing target data is classified into one of the plurality of different classes. The information processing device according to claim 8.

10. The estimation unit is using the higher-level classification model, estimating a probability value that the data to be processed will be classified into each of the plurality of lower classes; selecting a plurality of the classification models corresponding to each of two or more predetermined number of lower classes in descending order of probability value from among the classification models corresponding to each of the plurality of lower classes based on an estimation result of the higher-level classification model; and estimating a probability value that the data to be processed will be classified into each of classes further below the lower classes corresponding to each of the selected plurality of classification models. The information processing device according to claim 8.

11. The estimation unit is When the data to be processed is classified into two or more of the plurality of different classes, it is estimated that the data to be processed will be classified into the class corresponding to the classification path having the largest value of the product of the probability values ​​based on a comparison of the product of the probability values ​​of the data to be classified into each class in a classification path from the highest class to a class corresponding to each of the two or more classes among the plurality of different classes. The information processing device according to claim 10.

12. The acquisition unit is The classification model is a machine learning model that classifies the data to be classified into one of the plurality of lower classes or an unknown class that is not one of the plurality of lower classes, and the classification model generated for each of the upper classes and the data to be processed are acquired, The estimation unit is determining whether to discontinue processing subsequent to a processing of selecting a plurality of classification models corresponding to each of the predetermined number of lower classes from among the classification models corresponding to each of the plurality of lower classes based on an estimation result of the higher-level classification model; The information processing device according to claim 10.

13. The estimation unit is when the classification target data is classified into the unknown class, when a difference in probability values ​​for the processing target data being classified into each of the plurality of lower classes or the unknown class exceeds a first threshold value, or when a value of a product of probability values ​​for the processing target data being classified into each class in a classification path from the highest class to a class corresponding to each of two or more classes among the plurality of different classes falls below a second threshold value, determining to discontinue processing subsequent to the processing of selecting a plurality of the classification models corresponding to each of the predetermined number of lower classes. The information processing device according to claim 12.

14. An information processing method implemented by a program executed by an information processing device, comprising: a class generating step of generating higher classes that are higher than the plurality of different classes, the higher classes corresponding to the hierarchical clusters based on the plurality of class features hierarchically clustered based on a similarity between the plurality of class features relating to data respectively belonging to a plurality of different classes and that are representative values ​​of the plurality of different classes; a model generation step of generating, for each upper class, a classification model which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that belong to the lower class; Including, The model generation step includes: generating, for each of the upper classes, the classification model, which is the machine learning model trained based on a training data set that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs; Information processing methods.

15. An information processing method implemented by a program executed by an information processing device, comprising: a classification model which is a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a higher class that is a higher class than the plurality of different classes, the lower class being a class corresponding to each of the hierarchical clusters based on a plurality of class features hierarchically clustered based on a similarity between the plurality of class features that are representative values ​​of the plurality of different classes, the classification model being a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a lower class that is a higher class than the plurality of different classes, the classification model being trained based on a learning dataset that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs, and an acquisition step of acquiring the classification model which is the machine learning model generated for each of the higher classes, and the processing target data to be processed; an estimation step of estimating a first lower class into which the data to be processed is classified using a higher-level classification model which is the classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest-level class, selecting a lower-level classification model which is the classification model corresponding to the first lower class from among the classification models corresponding to each of a plurality of lower classes below the predetermined higher-level class based on a result of the estimation of the higher-level classification model, and estimating a second lower class into which the data to be processed is classified using the selected lower-level classification model; An information processing method comprising:

Images