Information processing apparatus and information processing method

Abstract

To efficiently predict each of a plurality of classes for classifying target data to be classified among a plurality of classes forming a hierarchical structure.SOLUTION: An information processing apparatus according to the present application includes an acquisition unit configured to acquire learning data including data of a set of input data and a plurality of input label data indicating each of a plurality of classes for classifying the input data among a plurality of classes forming a hierarchical structure, and a generation unit configured to generate a machine learning model including a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchies in the hierarchical structure, each of the plurality of feature extraction mechanisms extracting a feature value indicating a feature of the input data from the input data, and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchies, each of the plurality of prediction mechanisms being trained to predict the input label data corresponding to each of the plurality of hierarchies based on the feature value extracted by each of the plurality of feature extraction mechanisms.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Conventionally, there is known a technique for predicting each of a plurality of classes that classify target data to be classified, among a plurality of classes forming a hierarchical structure. For example, there is known a technique for predicting each of a plurality of classes that classify target data using a machine learning model in which the output layer of a convolutional neural network (CNN) is replaced with a fully connected linear layer, the number of which is the same as the number of layers in the hierarchical structure (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Riccardo La Grassa, Ignazio Gallo, Nicola Landro, "Learn Class Hierarchy using Convolutional Neural Networks", May 2020, [Retrieved May 21, 2020], Internet<URL:https: / / link.springer.com / article / 10.1007 / s10489-020-02103-6> Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for a technology that enables efficient prediction of each of the multiple classes that form a hierarchical structure and that classify target data to be classified.

[0005] The present application aims to provide an information processing device and an information processing method that can efficiently predict each of multiple classes that classify target data to be classified, out of multiple classes that form a hierarchical structure. [Means for solving the problem]

[0006] The information processing device according to the present application includes an acquisition unit that acquires training data including sets of data of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified out of a plurality of classes forming a hierarchical structure, and a generation unit that generates a machine learning model including a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchies in the hierarchical structure, the plurality of feature extraction mechanisms extracting, from the input data, feature amounts indicating characteristics of the input data, and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchies, the plurality of prediction mechanisms being trained to predict the input label data corresponding to each of the plurality of hierarchies based on the feature amounts extracted by each of the plurality of feature extraction mechanisms.

[0007] The information processing device according to the present application comprises: a machine learning model that includes, based on training data including sets of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified out of a plurality of classes forming a hierarchical structure, a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchies in the hierarchical structure, the plurality of feature extraction mechanisms extracting, from the input data, feature amounts indicating characteristics of the input data; and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchies, the plurality of prediction mechanisms trained to predict the input label data corresponding to each of the plurality of hierarchies based on the feature amounts extracted by each of the plurality of feature extraction mechanisms; an acquisition unit that acquires target data that is data to be processed; and a prediction unit that inputs the target data to the machine learning model and predicts each of a plurality of target label data indicating each of a plurality of classes into which the target data is classified out of a plurality of classes forming the hierarchical structure. [Effects of the Invention]

[0008] According to one aspect of the embodiment, it is possible to efficiently predict each of a plurality of classes that classify target data to be classified, out of a plurality of classes forming a hierarchical structure. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a plurality of classes forming a hierarchical structure. [Figure 2] FIG. 2 is a diagram showing an example of a plurality of classes forming a hierarchical structure. [Figure 3] FIG. 3 is a diagram illustrating an example of a machine learning model according to the comparative technique. [Figure 4] FIG. 4 is a diagram illustrating an example of a machine learning model according to the comparative technique. [Figure 5] FIG. 5 is a diagram illustrating an example of the configuration of an information processing device according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating a process for generating a machine learning model according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating a prediction process using a machine learning model according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the number of pieces of training data related to the comparison technique. [Figure 9] FIG. 9 is a diagram illustrating an example of the number of pieces of training data according to the embodiment. [Figure 10] FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a detailed description will be given of an information processing device and an information processing method according to the present application (hereinafter referred to as an "embodiment") with reference to the drawings. Note that the information processing device and the information processing method according to the present application are not limited to the embodiment. Furthermore, the same components in the following embodiments are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0011] (Embodiment) 1. Introduction Hierarchical relationships naturally and frequently exist in the real world. Here, having a hierarchical relationship can be rephrased as having a hierarchy or forming a hierarchical structure. A hierarchical structure is formed by multiple nodes belonging to multiple levels. Specifically, a hierarchical structure is a structure in which multiple nodes located one level below a certain level are arranged in a branching state from one node belonging to a certain level. Nodes belonging to each level in a hierarchical structure have a parent-child relationship. More specifically, a parent node has multiple child nodes. These child nodes also have multiple child nodes with themselves as parent nodes. A child node never has multiple parent nodes. A hierarchical structure is also called a tree structure. A tree structure is an expression that compares nodes to the branches and leaves of a tree. In a tree structure (hierarchical structure), the node belonging to the highest level that is higher than all other nodes (i.e., has no parent node) is called the root node.

[0012] For example, there are cases where multiple classes that classify target data to be classified form a hierarchical structure. Here, each of the multiple classes that classify target data to be classified corresponds to the above-mentioned node. For example, a hierarchical structure is formed by multiple classes belonging to multiple levels. For example, a hierarchical structure is a structure in which multiple classes located one level below a certain level branch off from a class belonging to that level. Furthermore, classes belonging to each level in the hierarchical structure have a parent-child relationship. More specifically, a parent class has multiple child classes. Furthermore, these child classes have multiple child classes as their parent class. Furthermore, a child class does not have multiple parent classes. Note that a class belonging to the highest level that is higher than all other classes (i.e., has no parent class) is called a root class. Below, an example of multiple classes forming a hierarchical structure is described using Figures 1 and 2.

[0013] FIG. 1 is a diagram illustrating an example of multiple classes forming a hierarchical structure. FIG. 1 shows a hierarchical structure HL1 formed by multiple classes that classify organisms that exist in nature. The hierarchical structure HL1 shown in FIG. 1 is formed by multiple classes located in each of a first hierarchy H11, a second hierarchy H12, and a third hierarchy H13. The hierarchical structure HL1 shown in FIG. 1 is a structure in which classes C11 to C13 corresponding to animals, microorganisms, and plants, respectively, located in the first hierarchy H11, which is one hierarchy below the top hierarchy, are arranged in a branched state from a root class corresponding to organisms located in the top hierarchy. Furthermore, the hierarchical structure HL1 is a structure in which classes C21 and C22 corresponding to Cucurbitaceae and Rosaceae, respectively, located in the second hierarchy H12, which is one hierarchy below the first hierarchy H11, are arranged in a branched state from class C13 corresponding to plants located in the first hierarchy H11. Although not shown in the figures, a plurality of classes located in the second hierarchy H12 are arranged in a branched state from each of class C11 corresponding to animals and class C12 corresponding to microorganisms located in the first hierarchy H11. Furthermore, in the hierarchical structure HL1, classes C31 and C32 corresponding to watermelon and melon, respectively, located in the third hierarchy H13, one hierarchy level below the second hierarchy H12, are arranged in a branched state from class C21 corresponding to Cucurbitaceae located in the second hierarchy H12. Furthermore, in the hierarchical structure HL1, classes C33 and C34 corresponding to apple and strawberry, respectively, located in the third hierarchy H13 are arranged in a branched state from class C22 corresponding to Rosaceae located in the second hierarchy H12. Although not shown in the figures, a plurality of classes located in the third hierarchy H13 are arranged in a branched state from each of the plurality of classes located in the second hierarchy H12 that branch off from class C11 corresponding to animals. Although not shown in the figure, a plurality of classes located in the second hierarchy H12 branching off from class C12 corresponding to microorganisms are also arranged in a branched state from each of the plurality of classes located in the third hierarchy H13. Note that the position of a given class in each hierarchy can be rephrased as the given class belonging to each hierarchy.

[0014] Furthermore, classes belonging to each level in the hierarchical structure HL1 have a parent-child relationship. For example, the root class corresponding to living organisms belonging to the highest level has itself as its parent class and classes C11 to C13 corresponding to animals, microorganisms, and plants belonging to the first level H11, which is one level below the top level, as its child classes. Furthermore, class C13 corresponding to plants belonging to the first level H11 has itself as its parent class and classes C21 and C22 corresponding to the Cucurbitaceae and Rosaceae families belonging to the second level H12, which is one level below the first level H11, as its child classes. Although not shown in the figure, class C11 corresponding to animals belonging to the first level H11 also has itself as its parent class and multiple classes belonging to the second level H12 as its child classes. Furthermore, although not shown in the figure, class C12 corresponding to microorganisms belonging to the first level H11 also has itself as its parent class and multiple classes belonging to the second level H12 as its child classes. Furthermore, class C21 corresponding to the Cucurbitaceae family in the second hierarchy H12 has itself as its parent class, and has classes C31 and C32 corresponding to watermelon and melon, respectively, in the third hierarchy H13, which is one hierarchy level below the second hierarchy H12, as its child classes. Class C22 corresponding to the Rosaceae family in the second hierarchy H12 has itself as its parent class, and has classes C33 and C34 corresponding to apple and strawberry, respectively, in the third hierarchy H13, as its child classes. Although not shown, each of the classes in the second hierarchy H12 that branch off from class C11 corresponding to animals also has multiple classes in the third hierarchy H13 as its child classes. Although not shown, each of the classes in the second hierarchy H12 that branch off from class C12 corresponding to microorganisms also has multiple classes in the third hierarchy H13 as its child classes.

[0015] In FIG. 1, target data to be classified is classified into one of a plurality of classes belonging to each level in the hierarchical structure HL1. Specifically, the target data is classified into one of classes C11 to C13 corresponding to animals, microorganisms, and plants, respectively, belonging to the first level H11. The target data is also classified into one of a plurality of classes belonging to the second level H12, which is branched off from the class into which the target data was classified in the first level H11. For example, if the target data is classified into class C13 corresponding to plants in the first level H11, the target data is classified into one of class C21 corresponding to Cucurbitaceae or class C22 corresponding to Rosaceae, which are branched off from class C13 corresponding to plants in the second level H12. The target data is also classified into one of a plurality of classes belonging to the third level H13, which is branched off from the class into which the target data was classified in the second level H12. For example, if the target data is classified into class C22 corresponding to Rosaceae in the second level H12, the target data is classified into class C33 corresponding to apple or class C34 corresponding to strawberry, which are branched off from class C22 corresponding to Rosaceae. For example, if the target data is data containing the characteristics of an apple, the target data is classified into class C33 corresponding to apples.

[0016] Generally, apples are known to be plants of the Rosaceae family. Therefore, data containing characteristics of apples is presumed to be data containing characteristics of the Rosaceae family and characteristics of plants. For example, image data of apples is an example of data containing characteristics of apples. The image data of apples is presumed to be image data containing characteristics of apples, image data containing characteristics of the Rosaceae family, and image data containing characteristics of plants. Therefore, the target data, which is image data of apples, is presumed to be classified into class C13 corresponding to plants in the first layer H11 of the multiple classes forming the hierarchical structure HL1. Furthermore, the target data, which is image data of apples, is presumed to be classified into class C22 corresponding to the Rosaceae family in the second layer H12 of the multiple classes forming the hierarchical structure HL1. Furthermore, the target data, which is image data of apples, is presumed to be classified into class C33 corresponding to apples in the third layer H13 of the multiple classes forming the hierarchical structure HL1.

[0017] FIG. 2 is a diagram illustrating an example of multiple classes forming a hierarchical structure. FIG. 2 shows a hierarchical structure HL2 formed by multiple classes that classify human behavior. The hierarchical structure HL2 shown in FIG. 2 is formed by multiple classes located in each of a first hierarchy H21, a second hierarchy H22, and a third hierarchy H23. The hierarchical structure HL2 shown in FIG. 2 is a structure in which classes C14 to C16 corresponding to human-only behavior, person-to-person behavior, and person-to-object behavior, located in the first hierarchy H21, one hierarchy below the top hierarchy, are arranged in a branching manner from a root class corresponding to behavior located in the top hierarchy. Furthermore, the hierarchical structure HL2 is a structure in which classes C23 and C24 corresponding to object-picking-up behavior and object-throwing behavior, located in the second hierarchy H22, one hierarchy below the first hierarchy H21, are arranged in a branching manner from class C16 corresponding to object-to-person behavior located in the first hierarchy H21. Although not shown, multiple classes located in the second hierarchy H22 are arranged branching off from class C14, which corresponds to the behavior of people alone, and class C15, which corresponds to the behavior of people interacting with other people, located in the first hierarchy H21. Furthermore, the hierarchical structure HL2 is structured such that classes C35 and C36, which correspond to the behavior of picking up trash and the behavior of picking up a ball, located in the third hierarchy H23, one hierarchy below the second hierarchy H22, are arranged branching off from class C23, which corresponds to the behavior of picking up an object located in the second hierarchy H22. Furthermore, the hierarchical structure HL2 is structured such that classes C37 and C38, which correspond to the behavior of throwing trash and the behavior of throwing a ball, located in the third hierarchy H23, are arranged branching off from class C24, which corresponds to the behavior of throwing an object located in the second hierarchy H22. Although not shown, multiple classes located in the third hierarchy H23 are arranged branching off from each of the multiple classes located in the second hierarchy H22 that branch off from class C14, which corresponds to the behavior of people alone. Also, although not shown in the figure, multiple classes located on the third hierarchical level H23 are arranged in a branched state from each of the multiple classes located on the second hierarchical level H22 that branch off from class C15, which corresponds to actions between people.

[0018] Furthermore, classes belonging to each level in the hierarchical structure HL2 have a parent-child relationship. For example, the root class corresponding to behavior belonging to the highest level has itself as its parent class, and has classes C14 to C16 corresponding to behaviors of people only, person-to-person behaviors, and person-to-object behaviors belonging to the first level H21, which is one level below the highest level, as its child classes. Furthermore, class C16 corresponding to behaviors of people and objects belonging to the first level H21 has itself as its parent class, and has classes C23 and C24 corresponding to behaviors of picking up and throwing objects belonging to the second level H22, which is one level below the first level H21, as its child classes. Although not shown in the figure, class C14 corresponding to behaviors of people only belonging to the first level H21 also has itself as its parent class, and multiple classes belonging to the second level H22 as its child classes. Furthermore, although not shown in the figure, class C15 corresponding to behaviors of people belonging to the first level H21 has itself as its parent class, and multiple classes belonging to the second level H22 as its child classes. Furthermore, class C23, which corresponds to the action of picking up an object and belongs to the second hierarchy H22, has itself as its parent class and has classes C35 and C36, which correspond to the action of picking up trash and the action of picking up a ball, respectively, which belong to the third hierarchy H23, which is one hierarchy level below the second hierarchy H22, as its child classes. Class C24, which corresponds to the action of throwing an object and belongs to the second hierarchy H22, has itself as its parent class and classes C37 and C38, which correspond to the action of throwing trash and the action of throwing a ball, respectively, which belong to the third hierarchy H23. Although not shown, each of the multiple classes belonging to the second hierarchy H22 that branch off from class C14, which corresponds to the action of only people, also has multiple classes belonging to the third hierarchy H23 as its child classes. Furthermore, although not shown, each of the multiple classes belonging to the second hierarchy H22 that branch off from class C15, which corresponds to the action of people interacting with other people, also has multiple classes belonging to the third hierarchy H23 as its child classes.

[0019] In FIG. 2, target data to be classified is classified into one of multiple classes belonging to each level in the hierarchical structure HL2. Specifically, the target data is classified into one of classes C14 to C16 belonging to the first level H21, which correspond to actions of people only, actions between people, and actions between people and objects. The target data is also classified into one of multiple classes belonging to the second level H22, which branch off from the classes into which the target data is classified in the first level H21. For example, if the target data is classified into class C16 corresponding to actions between people and objects in the first level H21, the target data is then classified into class C23 corresponding to actions of picking up an object or class C24 corresponding to actions of throwing an object, which belong to the second level H22, which branch off from class C16 corresponding to actions between people and objects. The target data is also classified into one of multiple classes belonging to the third level H23, which branch off from the classes into which the target data is classified in the second level H22. For example, if the target data is classified into class C24 corresponding to the behavior of throwing an object in the second hierarchical level H22, the target data is classified into either class C37 corresponding to the behavior of throwing trash or class C38 corresponding to the behavior of throwing a ball, which belong to the third hierarchical level H23 branching off from class C24 corresponding to the behavior of throwing an object. For example, if the target data is data that includes the characteristics of the behavior of throwing trash, the target data is classified into class C37 corresponding to the behavior of throwing trash.

[0020] Note that although Figures 1 and 2 describe a case where the hierarchical structure includes three hierarchies: a first hierarchical level, a second hierarchical level, and a third hierarchical level, the hierarchical structure according to the embodiment may also be a hierarchical structure including four or more hierarchical levels.

[0021] As described in Figures 1 and 2, hierarchical relationships naturally and frequently exist in the real world. For example, there are cases where multiple classes that classify target data to be classified form a hierarchical structure. However, it has not been said that there exists a technology that can efficiently predict each of the multiple classes that classify target data to be classified, among the multiple classes that form the hierarchical structure. Below, an example of a machine learning model related to a comparative technology is described using Figures 3 and 4.

[0022] FIG. 3 is a diagram illustrating an example of a machine learning model according to a comparative technique. Model #1 illustrated in FIG. 3 is a machine learning model trained to output, when input data is input, a single piece of label data (also referred to as multi-label data) including a plurality of label data corresponding to each of a plurality of classes that classify the input data from among a plurality of classes forming a hierarchical structure HL1 (see FIG. 1). In FIG. 3, when image data G1 of an apple is input to model #1 as target data, model #1 outputs a single piece of label data including label data indicating class C13 corresponding to plants, label data indicating class C22 corresponding to Rosaceae, and label data indicating class C33 corresponding to apples. Because the comparative technique illustrated in FIG. 3 predicts a single piece of label data including a plurality of label data indicating each of a plurality of classes that classify the target data, it cannot be said to predict each of a plurality of classes that classify the target data to be classified from among a plurality of classes forming a hierarchical structure.

[0023] FIG. 4 is a diagram illustrating an example of a machine learning model according to a comparative technique. Model #11 shown in FIG. 4 is a machine learning model trained to output, when input data is input, label data indicating either class C11 corresponding to animals or class C13 corresponding to plants, among the multiple classes forming the hierarchical structure HL1 (see FIG. 1). Model #12 shown in FIG. 4 is a machine learning model trained to output, when input data is input, label data indicating either class C21 corresponding to Cucurbitaceae or class C22 corresponding to Rosaceae, among the multiple classes forming the hierarchical structure HL1. Model #13 shown in FIG. 4 is a machine learning model trained to output, when input data is input, label data indicating either class C33 corresponding to apples or class C34 corresponding to strawberries, among the multiple classes forming the hierarchical structure HL1. In FIG. 4, image data G1 of an image of an apple is input to each of models #11 to #13 as target data. In FIG. 4, model #11 outputs label data indicating class C13 corresponding to plants. Model #12 outputs label data indicating class C22 corresponding to Rosaceae. Model #13 outputs label data indicating class C33, which corresponds to apples.

[0024] The comparative technique shown in Figure 4 requires separate training of multiple machine learning models corresponding to each of the multiple classes into which the target data to be classified is classified, resulting in high training costs. Furthermore, the comparative technique shown in Figure 4 requires separate prediction of label data indicating each of the multiple classes into which the target data to be classified is classified, resulting in high prediction costs. For this reason, the comparative technique shown in Figure 4 cannot be said to be able to efficiently predict each of the multiple classes into which the target data to be classified is classified, among the multiple classes forming a hierarchical structure.

[0025] 2. Configuration of Information Processing Device An example of the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0026] (Communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC). The communication unit 110 is connected to various networks via wired or wireless connections, and transmits and receives information to and from, for example, terminal devices used by users.

[0027] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Specifically, the storage unit 120 stores various data. For example, the storage unit 120 stores training data used for training a machine learning model. The storage unit 120 also stores various programs. For example, the storage unit 120 stores an information processing program according to the embodiment. The storage unit 120 may also store information related to the various data acquired by the acquisition unit 131. The storage unit 120 may also store information related to the machine learning model generated by the generation unit 132. The storage unit 120 may also store information related to the prediction result predicted by the prediction unit 133.

[0028] (control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 100 using RAM as a work area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0029] The control unit 130 has an acquisition unit 131, a generation unit 132, a prediction unit 133, and a provision unit 134 as functional units, and may realize or execute the information processing actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 5, and may be any other configuration that performs the information processing described below. Furthermore, each functional unit indicates a function of the control unit 130, and does not necessarily have to be physically distinct.

[0030] (Acquisition part 131) The acquiring unit 131 may acquire various types of data. Specifically, the acquiring unit 131 acquires training data including data pairs each consisting of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified, among a plurality of classes forming a hierarchical structure. More specifically, the acquiring unit 131 may acquire training data including a plurality of pairs each consisting of input data and a plurality of input label data. For example, the acquiring unit 131 may acquire the training data by referring to the storage unit 120. Note that the training data may be data created by a person, or may be data automatically generated by a computer that generates training data.

[0031] Here, the input label data is label data indicating a class into which input data is classified in each of multiple hierarchical levels in the hierarchical structure. The same number of input label data is prepared as the number of hierarchical levels in the hierarchical structure. For example, in the example shown in FIG. 1, input data that is image data of apples is classified into class C13 corresponding to plants in the first hierarchical level H11 among the multiple classes forming the hierarchical structure HL1. In this case, the input label data corresponding to the first hierarchical level H11 is label data indicating class C13 corresponding to plants. Furthermore, the input data that is image data of apples is classified into class C22 corresponding to Rosaceae in the second hierarchical level H12 among the multiple classes forming the hierarchical structure HL1. In this case, the input label data corresponding to the second hierarchical level H12 is label data indicating class C22 corresponding to Rosaceae. Furthermore, the input data that is image data of apples is classified into class C33 corresponding to apples in the third hierarchical level H13 among the multiple classes forming the hierarchical structure HL1. In this case, the input label data corresponding to the third hierarchical level H13 is label data indicating class C33 corresponding to apples. That is, among the multiple classes forming the hierarchical structure HL1, the multiple input label data indicating each of the multiple classes into which the input data, which is image data of apples, is classified are label data indicating class C13 corresponding to plants, label data indicating class C22 corresponding to the Rosaceae family, and label data indicating class C33 corresponding to apples.

[0032] (Generation unit 132) The generation unit 132 may generate various types of information. Specifically, the generation unit 132 may generate a machine learning model trained based on the training data acquired by the acquisition unit 131. More specifically, when input data included in the training data acquired by the acquisition unit 131 is input, the generation unit 132 may generate a machine learning model trained to output each of a plurality of input label data included in the training data.

[0033] FIG. 6 is a diagram illustrating a process for generating a machine learning model according to an embodiment. In FIG. 6, a case will be described in which a hierarchical structure includes a first layer, a second layer, ..., and an Nth layer (N is a natural number equal to or greater than 3). In FIG. 6, the generation unit 132 may generate a machine learning model M1 including a first feature extraction mechanism F1 to an Nth feature extraction mechanism FN corresponding to the first layer to the Nth layer in the hierarchical structure, respectively, and each extracting a feature quantity indicating a feature of the input data IN1 from the input data IN1. Here, the feature quantity may be, for example, a vector. In other words, the generation unit 132 may generate a machine learning model M1 including a first feature extraction mechanism F1 corresponding to the first layer in the hierarchical structure, a second feature extraction mechanism F2 corresponding to the second layer, ..., and an Nth feature extraction mechanism FN corresponding to the Nth layer. Furthermore, the generation unit 132 may generate a machine learning model M1 that is trained so that when input data IN1 is input to each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN, each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN outputs a feature that indicates the characteristics of the input data IN1.

[0034] Furthermore, the generation unit 132 may generate a machine learning model M1 including a feature extraction mechanism corresponding to a layer one layer lower than the first layer, trained to extract a second feature from the input data IN1, based on the input data IN1 and a first feature extracted by the feature extraction mechanism corresponding to a layer included in the first layer to the Nth layer. For example, the generation unit 132 may generate a machine learning model M1 including a second feature extraction mechanism F2 corresponding to a second layer one layer lower than the first layer, trained to extract a second feature from the input data IN1, based on the first feature extracted by the first feature extraction mechanism F1 corresponding to the first layer and the input data IN1. For example, the generation unit 132 may generate a machine learning model M1 trained so that when the first feature and the input data IN1 are input to the second feature extraction mechanism F2, the second feature extraction mechanism F2 outputs a second feature indicating a feature of the input data IN1. Similarly, the generation unit 132 may generate a machine learning model M1 including a k-th feature extraction mechanism Fk corresponding to the k-th hierarchical level, which is one hierarchical level below the (k-1) hierarchical level, trained to extract the k-th feature from the input data IN1, based on the (k-1)th feature extracted by the (k-1)th feature extraction mechanism F(k-1) corresponding to the (k-1) hierarchical level (k is a natural number from 2 to N) and the input data IN1. For example, the generation unit 132 may generate a machine learning model M1 trained so that when the (k-1)th feature and the input data IN1 are input to the k-th feature extraction mechanism Fk, the k-th feature extraction mechanism Fk outputs the k-th feature indicating the feature of the input data IN1.

[0035] Here, each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN may be realized by any known technology capable of extracting, from the input data, a feature amount indicating the characteristics of the input data. For example, each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN may be an encoder that extracts, from the input data, a feature amount indicating the characteristics of the input data.

[0036] For example, when the input data is image data, each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN shown in Fig. 6 may be an image encoder that extracts features (hereinafter, sometimes referred to as "image features") indicating characteristics of the image data (hereinafter, sometimes referred to as "image"). For example, the image encoder may include a convolutional neural network (CNN). The image encoder may then extract the image features from the image using the CNN. For example, the image encoder may include Residual Network (ResNet) (Kaiming He et al., 2015), AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan et al., 2014), GoogLeNet (Szegedy et al., 2014), Squeeze-and-Excitation Networks (SENet) (Jie Hu et al., 2018), EfficientNet (Tan et al., 2019), or ZFNet (Matthew et al., 2013), which are developed for object recognition. The image encoder may extract image features from an image using ResNet, AlexNet, VGGNet, GoogLeNet, SENet, EfficientNet, or ZFNet. For example, the image encoder may include Faster R-CNN (Shaoqing Ren et al., 2015), YOLO (You Look Only Once) (Joseph Redmon et al., 2015), or SSD (Single Shot MultiBox Detector) (Wei Liu., 2015), which are developed for object detection. The image encoder may extract image features from the image using Faster R-CNN, YOLO, or SSD. For example, the image encoder may include an image encoder for CLIP (Alec Radford et al., 2021). The image encoder may extract image features from the image using the CLIP image encoder.

[0037] 6, the generation unit 132 may generate a machine learning model M1 in which the first prediction mechanism P1 to the Nth prediction mechanism PN correspond to the first to Nth layers in the hierarchical structure, respectively, and the first prediction mechanism P1 to the Nth prediction mechanism PN are trained to predict input label data corresponding to the first to Nth layers, respectively, based on feature amounts extracted by the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN. In other words, the generation unit 132 may generate a machine learning model M1 including a first prediction mechanism P1 corresponding to the first layer in the hierarchical structure, a second prediction mechanism P2 corresponding to the second layer, ..., and an Nth prediction mechanism PN corresponding to the Nth layer. Furthermore, the generation unit 132 may generate a machine learning model M1 that is trained so that when features extracted by the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN are input to the first prediction mechanism P1 to the Nth prediction mechanism PN, the first prediction mechanism P1 to the Nth prediction mechanism PN each output input label data corresponding to the first layer to the Nth layer.

[0038] Each of the first prediction mechanism P1 to the Nth prediction mechanism PN is designed according to each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN and the type of training data. For example, each of the first prediction mechanism P1 to the Nth prediction mechanism PN is designed based on multilayer perceptron (MLP) technology. For example, each of the first prediction mechanism P1 to the Nth prediction mechanism PN may be a neural network consisting of a linear layer (fully-connected layer) and an activation function (such as a softmax function). Each of the first prediction mechanism P1 to the Nth prediction mechanism PN may employ, for example, an SVM (support vector machine) as a classification layer.

[0039] More specifically, the generation unit 132 generates first loss values ​​L corresponding to the first to Nth layers based on the first prediction result data R11 to Nth prediction result data R1N predicted by the first prediction mechanism P1 to Nth prediction mechanism PN, respectively. 階層1 ~Nth loss value L 階層NFor example, when a first feature extracted by the first feature extraction mechanism F1 is input to the first prediction mechanism P1, the generation unit 132 acquires first prediction result data R11 output by the first prediction mechanism P1. Furthermore, the generation unit 132 calculates a first loss value L indicating the error between the first predicted value and the first correct value based on the first predicted value corresponding to the first prediction result data R11 and the first correct value corresponding to the first input label data corresponding to the first layer, which is the first correct data A1. 階層1 Furthermore, when the second feature quantity extracted by the second feature extraction mechanism F2 is input to the second prediction mechanism P2, the generation unit 132 acquires the second prediction result data R12 output by the second prediction mechanism P2. Furthermore, the generation unit 132 calculates a second loss value L indicating the error between the second predicted value and the second correct value based on the second predicted value corresponding to the second prediction result data R12 and the second correct value corresponding to the second input label data corresponding to the second layer, which is the second correct data A2. 階層2 Similarly, when the k-th feature quantity extracted by the k-th feature extraction mechanism Fk is input to the k-th prediction mechanism Pk, the generation unit 132 acquires the k-th prediction result data R1k output by the k-th prediction mechanism Pk. Furthermore, the generation unit 132 calculates a k-th loss value L indicating the error between the k-th predicted value and the k-th correct value based on the k-th predicted value corresponding to the k-th prediction result data R1k and the k-th correct value corresponding to the k-th input label data corresponding to the k-th layer, which is the k-th correct data Ak. 階層k may be calculated.

[0040] Furthermore, the generation unit 132 generates the first loss value L 階層1 , the second loss value L 階層2 , …, Nth loss value L 階層N and the first loss value L 階層1 , the second loss value L 階層2 , …, Nth loss value L 階層N αL is the value obtained by multiplying each of the weights α, β, ..., γ corresponding to 階層1 , βL 階層2 , …, γL 階層N The total loss value L total For example, the generation unit 132 may calculate the total loss value L based on the following equation (1): totalmay be calculated.

[0041]

number

[0042] Furthermore, the generation unit 132 calculates the total loss value L total For example, the generation unit 132 may generate the machine learning model M1 by learning the machine learning model M1 based on the total loss value L total The machine learning model M1 may be generated by learning the parameters (connection coefficients) of the machine learning model M1 so as to reduce .

[0043] As described above, the generation unit 132 generates a machine learning model (machine learning model M1 in FIG. 6) including: a plurality of feature extraction mechanisms (first feature extraction mechanism F1 to Nth feature extraction mechanism FN in FIG. 6) corresponding to each of a plurality of layers in the hierarchical structure (first layer to Nth layer in FIG. 6), which extract feature quantities indicating features of input data from input data (input data IN1 in FIG. 6); and a plurality of prediction mechanisms (first prediction mechanism P1 to Nth prediction mechanism PN in FIG. 6) corresponding to each of the plurality of layers, which are trained to predict input label data (first input label data to Nth input label data in FIG. 6) corresponding to each of the plurality of layers, based on the feature quantities extracted by each of the plurality of feature extraction mechanisms.

[0044] The machine learning model also includes a first feature extraction mechanism (the (k-1) feature extraction mechanism F(k-1) in FIG. 6) that is a feature extraction mechanism corresponding to one layer (the (k-1)th layer in FIG. 6) included in multiple layers (the first layer to the Nth layer in FIG. 6), a second feature extraction mechanism (the k-th feature extraction mechanism Fk in FIG. 6) that is a feature extraction mechanism corresponding to the layer one layer below the first layer (the k-th layer in FIG. 6), and a second prediction mechanism (the k-th prediction mechanism in FIG. 6) that is a prediction mechanism corresponding to the layer one layer below. The second feature extraction mechanism (the k-th feature extraction mechanism Fk in FIG. 6) extracts a second feature (the k-th feature in FIG. 6) from the input data based on the first feature (the (k-1)th feature in FIG. 6) extracted by the first feature extraction mechanism (the (k-1)th feature extraction mechanism F(k-1) in FIG. 6) and the input data (input data IN1 in FIG. 6), and the second prediction mechanism (the k-th prediction mechanism in FIG. 6) is trained to predict input label data (the k-th input label data in FIG. 6) corresponding to the next lower hierarchical level (the k-th hierarchical level in FIG. 6) based on the second feature.

[0045] The generation unit 132 also generates a plurality of loss values ​​(first loss value L in FIG. 6 ) corresponding to each of a plurality of layers (first layer to Nth layer in FIG. 6 ) based on a plurality of prediction results (first prediction result data R11 to Nth prediction result data R1N in FIG. 6 ) predicted by each of a plurality of prediction mechanisms (first prediction mechanism P1 to Nth prediction mechanism PN in FIG. 6 ). 階層1 ~Nth loss value L 階層N ) and calculates the total loss value (in FIG. 6, total loss value L total ) and generates a machine learning model by training the machine learning model (machine learning model M1 in Figure 6) based on the overall loss value.

[0046] (Prediction section 133) The prediction unit 133 may predict various types of information. Specifically, the prediction unit 133 may use the machine learning model generated by the generation unit 132 to predict each of a plurality of target label data indicating each of a plurality of classes into which the target data is classified. More specifically, the prediction unit 133 may input the target data into the machine learning model generated by the generation unit 132, and predict each of a plurality of target label data indicating each of a plurality of classes into which the target data is classified, among a plurality of classes forming a hierarchical structure. Note that a user may use target label data of a desired hierarchical level from the plurality of target label data predicted by the prediction unit 133.

[0047] FIG. 7 is a diagram illustrating a prediction process using a machine learning model according to an embodiment. The machine learning model M1 shown in FIG. 7 is the same as the machine learning model M1 shown in FIG. 6, but differs from FIG. 6 in that it is a trained machine learning model. Similar to FIG. 6, FIG. 7 illustrates a case in which the hierarchical structure includes a first layer, a second layer, ..., and an Nth layer (N is a natural number equal to or greater than 3). In FIG. 7, the acquisition unit 131 acquires the machine learning model M1 generated by the generation unit 132 and target data IN2, which is data to be processed. The prediction unit 133 may input the target data IN2 to the machine learning model M1 and predict each of the first to Nth target label data indicating each of the multiple classes into which the target data IN2 is classified, out of the multiple classes forming the hierarchical structure.

[0048] More specifically, the prediction unit 133 may input the target data IN2 to each of the first feature extraction mechanisms F1 to N-th feature extraction mechanisms FN included in the machine learning model M1. When the target data IN2 is input to each of the first feature extraction mechanisms F1 to N-th feature extraction mechanisms FN, the first feature extraction mechanisms F1 to N-th feature extraction mechanisms FN output the first'th feature amount to the N'th feature amount, respectively. The k-th feature extraction mechanism Fk extracts the k'th feature amount from the target data IN2 based on the (k-1)'th feature amount extracted by the (k-1)'th feature extraction mechanism F(k-1) and the target data IN2. In other words, when the (k-1)'th feature amount and the target data IN2 are input to the k-th feature extraction mechanism Fk, the k-th feature extraction mechanism Fk outputs the k'th feature amount. Furthermore, when the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN output the first' to N'th feature amounts, respectively, the first prediction mechanism P1 to the Nth prediction mechanism PN receive the first' to N'th feature amounts. Furthermore, when the first' to N'th feature amounts are input to the first prediction mechanism P1 to the Nth prediction mechanism PN, the first prediction mechanism P1 to the Nth prediction mechanism PN receive the first prediction result data R21 to the Nth prediction result data R2N, respectively. Furthermore, the prediction unit 133 may obtain the first prediction result data R21 to the Nth prediction result data R2N as the first target label data to the Nth target label data, respectively.

[0049] As described above, based on learning data including sets of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified out of a plurality of classes forming the hierarchical structure, the acquisition unit 131 acquires a machine learning model (machine learning model M1 in FIG. 7) including a plurality of feature extraction mechanisms (first feature extraction mechanism F1 to Nth feature extraction mechanism FN in FIG. 7) corresponding to each of a plurality of layers in the hierarchical structure (first layer to Nth layer in FIG. 7), each of which extracts features indicating the characteristics of the input data from the input data, and a plurality of prediction mechanisms (first prediction mechanism P1 to Nth prediction mechanism PN in FIG. 7) corresponding to each of the plurality of layers, each of which is trained to predict input label data corresponding to each of the plurality of layers based on the features extracted by each of the plurality of feature extraction mechanisms, and target data (target data IN2 in FIG. 7), which is the data to be processed. The prediction unit 133 inputs the target data into a machine learning model and predicts each of a plurality of target label data (in FIG. 7, the first target label data to the Nth target label data) indicating each of a plurality of classes into which the target data is classified, among a plurality of classes forming a hierarchical structure.

[0050] (Provider 134) The providing unit 134 may provide various information. Specifically, the providing unit 134 may provide the target label data predicted by the prediction unit 133 to the user. For example, the providing unit 134 may transmit the target label data to the user's terminal device. For example, the providing unit 134 may provide target label data of a hierarchical level desired by the user from among a plurality of target label data. In FIG. 7, the providing unit 134 may provide the user with second prediction result data R22 corresponding to the second hierarchical level desired by the user as second label data.

[0051] [3. Number of training data] FIG. 8 shows an example of the number of training data sets used in the comparison technique. The left side of FIG. 8 shows that as the number of classification classes increases, the distribution of the number of training data sets often becomes long-tailed. In contrast, the predictive accuracy of a machine learning model often depends on the number of training data sets, resulting in poor predictive accuracy for minority classes with a small number of training data sets. The right side of FIG. 8 shows an undersampling technique that reduces the number of training data sets for the majority class with a large number of training data sets to match the number of training data sets for the minority class in order to resolve the imbalance in the number of training data sets shown on the left side of FIG. 8. Undersampling techniques result in significant loss of information for the majority class with a large number of training data sets.

[0052] FIG. 9 is a diagram illustrating an example of the number of pieces of training data according to an embodiment. In FIG. 9, training data for predicting a class corresponding to strawberry and training data for predicting a class corresponding to apple are combined and used as training data for predicting a class corresponding to Rosaceae. Similarly, training data for predicting a class corresponding to melon and training data for predicting a class corresponding to watermelon are combined and used as training data for predicting a class corresponding to Cucurbitaceae. As shown in FIG. 9, the training data according to the embodiment is combined with training data for predicting each of multiple lower classes belonging to a higher class and used as training data for predicting the higher class. This allows the information processing device 100 to increase the number of pieces of training data for predicting the higher class, even when, for example, there is little training data for predicting the higher class.

[0053] In the above-described embodiment, the input data and target data are image data, but the input data and target data are not limited to image data. Furthermore, the input data and target data are not limited to still image data. For example, the input data and target data may be video data. Furthermore, the input data and target data may be text data or audio data.

[0054] For example, if the input data is text data, each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN shown in FIGS. 6 and 7 may be a text encoder that extracts features (hereinafter, sometimes referred to as "text features") indicating characteristics of the text data (hereinafter, sometimes referred to as "text"). For example, the text encoder may include a recurrent neural network (RNN). The text encoder may then extract the text features from the text using the RNN. Furthermore, for example, the text encoder may include a gated recurrent unit (GRU) or a long short-term memory (LSTM). The text encoder may then extract the text features from the text using the GRU or LSTM. For example, the text encoder may include a Transformer (Ashish Vaswani et al., 2017), a Transformer-based Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-Training 3 (GPT-3), or a Text-to-Text Transfer Transformer (T5). The text encoder may extract text features from the text using the Transformer, BERT, GPT-3, or T5. For example, the text encoder may include a CLIP text encoder. The text encoder may extract text features from the text using the CLIP text encoder.

[0055] Furthermore, for example, when the input data is speech data, each of the first feature extraction mechanism F1 to the Nth feature extraction mechanism FN shown in Figures 6 and 7 may be a speech encoder that extracts features (hereinafter, sometimes referred to as "speech features") indicating characteristics of speech data (hereinafter, sometimes referred to as "speech"). For example, the speech encoder may include wav2vec (Alexei Baevski et al., 2020). The speech encoder may then use wav2vec to extract speech features from the speech.

[0056] [4. Effects] As described above, the information processing device 100 according to the embodiment includes an acquisition unit 131 and a generation unit 132. The acquisition unit 131 acquires training data including pairs of input data and input label data indicating each of multiple classes into which the input data is classified, among multiple classes forming a hierarchical structure. The generation unit 132 generates a machine learning model including: multiple feature extraction mechanisms corresponding to multiple hierarchical levels in the hierarchical structure, each extracting feature amounts indicating characteristics of the input data from the input data; and multiple prediction mechanisms corresponding to the multiple hierarchical levels, each trained to predict input label data corresponding to each of the multiple hierarchical levels, based on the feature amounts extracted by each of the multiple feature extraction mechanisms.

[0057] As a result, the information processing device 100 can use the generated machine learning model to extract, from the target data to be classified, each of a plurality of feature quantities corresponding to each of a plurality of levels in a hierarchical structure. That is, the information processing device 100 can extract, from the target data, each of a plurality of feature quantities reflecting the hierarchical relationships in the hierarchical structure. Furthermore, the information processing device 100 can efficiently predict, from among the multiple classes forming the hierarchical structure, each of a plurality of classes into which the target data to be classified is classified, based on each of the multiple feature quantities reflecting the hierarchical relationships in the hierarchical structure. Therefore, the information processing device 100 can efficiently predict, from among the multiple classes forming the hierarchical structure, each of a plurality of classes into which the target data to be classified is classified. Furthermore, the information processing device 100 can efficiently predict, from among the multiple classes forming the hierarchical structure, each of a plurality of classes into which the target data to be classified is classified, thereby contributing to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and promote sustainable development."

[0058] In addition, the generation unit 132 calculates each of a plurality of loss values ​​corresponding to each of a plurality of hierarchies based on each of a plurality of prediction results predicted by each of a plurality of prediction mechanisms, calculates an overall loss value which is the sum of values ​​obtained by multiplying each of the plurality of loss values ​​by a weight corresponding to each of the plurality of loss values, and generates a machine learning model by learning the machine learning model based on the overall loss value.

[0059] As a result, the information processing device 100 does not need to individually train each of a plurality of machine learning models corresponding to each of a plurality of classes into which the target data to be classified is to be classified, thereby reducing the training cost of the machine learning models.Furthermore, the information processing device 100 does not need to individually predict label data indicating each of a plurality of classes into which the target data to be classified is to be classified, thereby reducing the inference cost of the machine learning models.Therefore, the information processing device 100 can efficiently predict each of a plurality of classes into which the target data to be classified is to be classified, out of a plurality of classes forming a hierarchical structure.

[0060] The information processing device 100 further includes a prediction unit 133. Based on training data including pairs of input data and input label data indicating each of a plurality of classes into which the input data is classified among a plurality of classes forming a hierarchical structure, the acquisition unit 131 acquires a machine learning model including: a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchical layers in the hierarchical structure, each extracting feature amounts indicating characteristics of the input data from the input data; and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchical layers, each trained to predict the input label data corresponding to each of the plurality of hierarchical layers based on the feature amounts extracted by each of the feature extraction mechanisms; and target data, which is data to be processed. The prediction unit 133 inputs the target data to the machine learning model and predicts each of a plurality of target label data indicating each of a plurality of classes into which the target data is classified among a plurality of classes forming the hierarchical structure.

[0061] As a result, the information processing device 100 can use the acquired machine learning model to extract, from the target data to be classified, each of a plurality of feature quantities corresponding to each of a plurality of hierarchical levels in the hierarchical structure. That is, the information processing device 100 can extract, from the target data, each of a plurality of feature quantities reflecting the hierarchical relationships in the hierarchical structure. Furthermore, the information processing device 100 can efficiently predict, from among the multiple classes forming the hierarchical structure, each of a plurality of classes into which the target data to be classified is classified, based on each of the multiple feature quantities reflecting the hierarchical relationships in the hierarchical structure. Therefore, the information processing device 100 can efficiently predict, from among the multiple classes forming the hierarchical structure, each of a plurality of classes into which the target data to be classified is classified. Furthermore, because the information processing device 100 can efficiently predict, from among the multiple classes forming the hierarchical structure, each of the multiple classes into which the target data to be classified is classified, this can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0062] The machine learning model also includes a first feature extraction mechanism that is a feature extraction mechanism corresponding to one of the multiple hierarchies, a second feature extraction mechanism that is a feature extraction mechanism corresponding to the hierarchical level one level lower than the first hierarchical level, and a second prediction mechanism that is a prediction mechanism corresponding to the hierarchical level one level lower. The second feature extraction mechanism extracts second features from the input data based on the first features extracted by the first feature extraction mechanism and the input data, and the second prediction mechanism is trained to predict input label data corresponding to the hierarchical level one level lower based on the second features.

[0063] In this way, the information processing device 100 adds features corresponding to a higher layer in the hierarchical structure to the input of a feature extraction mechanism corresponding to a layer one level lower than the higher layer. This allows the feature extraction mechanism corresponding to a lower layer to extract features specialized for identifying a class corresponding to that layer. That is, the information processing device 100 can extract features for a lower layer based on the extraction results of features for a higher layer in the hierarchical structure. This allows the prediction mechanism for each layer in the hierarchical structure to predict a class in that layer based on features specialized for predicting a class in that layer. That is, the information processing device 100 can enable the prediction mechanism for each layer in the hierarchical structure to have a function specialized for predicting a class in that layer. Therefore, the information processing device 100 can efficiently predict each of the multiple classes that classify the target data to be classified, among the multiple classes forming the hierarchical structure.

[0064] [5. Hardware Configuration] The information processing device 100 according to the embodiment described above is realized by, for example, a computer 1000 configured as shown in Fig. 10. Fig. 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0065] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0066] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0067] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0068] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0069] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0070] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have undergone various modifications and improvements based on the knowledge of those skilled in the art.

[0071] [6. Other] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0072] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0073] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content. [Explanation of symbols]

[0074] 100 Information processing device 110 Communications Department 120 Storage section 130 control section 131 Acquisition Department 132 Generation part 133 Prediction Department 134 Provision Department

Claims

1. an acquisition unit that acquires learning data including a set of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified, among a plurality of classes forming a hierarchical structure; a generation unit that generates a machine learning model including: a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchical layers in the hierarchical structure, the plurality of feature extraction mechanisms extracting, from the input data, feature quantities that indicate characteristics of the input data; and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchical layers, the plurality of prediction mechanisms being trained to predict the input label data corresponding to each of the plurality of hierarchical layers based on the feature quantities extracted by each of the plurality of feature extraction mechanisms; An information processing device comprising:

2. The generation unit Based on each of a plurality of prediction results predicted by each of the plurality of prediction mechanisms, a plurality of loss values ​​corresponding to each of the plurality of layers are calculated, an overall loss value is calculated as the sum of values ​​obtained by multiplying each of the plurality of loss values ​​by a weight corresponding to each of the plurality of loss values, and the machine learning model is trained based on the overall loss value, thereby generating the machine learning model. The information processing device according to claim 1 .

3. an acquisition unit that acquires, based on learning data including sets of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified out of a plurality of classes forming a hierarchical structure, a machine learning model including: a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchical layers in the hierarchical structure, the plurality of feature extraction mechanisms extracting, from the input data, feature amounts indicating characteristics of the input data; and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchical layers, the plurality of prediction mechanisms being trained to predict the input label data corresponding to each of the plurality of hierarchical layers based on the feature amounts extracted by each of the plurality of feature extraction mechanisms; and target data that is data to be processed; a prediction unit that inputs the target data into the machine learning model and predicts each of a plurality of target label data indicating each of a plurality of classes into which the target data is classified, among a plurality of classes that form the hierarchical structure; An information processing device comprising:

4. the machine learning model includes a first feature extraction mechanism that is a feature extraction mechanism corresponding to one of the multiple hierarchies, a second feature extraction mechanism that is a feature extraction mechanism corresponding to a hierarchy one level lower than the first hierarchy, and a second prediction mechanism that is a prediction mechanism corresponding to the hierarchy one level lower; The second feature extraction mechanism extracts a second feature from the input data based on the first feature extracted by the first feature extraction mechanism and the input data, and the second prediction mechanism is trained to predict the input label data corresponding to the next lower hierarchical level based on the second feature.

4. The information processing device according to claim 1.

5. An information processing method realized by a program executed by an information processing device, an acquisition step of acquiring learning data including a set of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified, among a plurality of classes forming a hierarchical structure; a generation process for generating a machine learning model including: a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchical layers in the hierarchical structure, each of which extracts a feature quantity indicating a feature of the input data from the input data; and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchical layers, each of which is trained to predict the input label data corresponding to each of the plurality of hierarchical layers based on the feature quantity extracted by each of the plurality of feature extraction mechanisms; An information processing method including:

6. An information processing method realized by a program executed by an information processing device, an acquisition process for acquiring, based on training data including a set of input data and a plurality of input label data indicating each of a plurality of classes into which the input data is classified out of a plurality of classes forming a hierarchical structure, a machine learning model including: a plurality of feature extraction mechanisms corresponding to each of a plurality of hierarchical layers in the hierarchical structure, the plurality of feature extraction mechanisms extracting, from the input data, feature amounts indicating characteristics of the input data; and a plurality of prediction mechanisms corresponding to each of the plurality of hierarchical layers, the plurality of prediction mechanisms trained to predict the input label data corresponding to each of the plurality of hierarchical layers based on the feature amounts extracted by each of the plurality of feature extraction mechanisms; and target data which is data to be processed; a prediction step of inputting the target data into the machine learning model and predicting each of a plurality of target label data indicating each of a plurality of classes into which the target data is classified, among a plurality of classes forming the hierarchical structure; An information processing method including:

Images