Information processing device, information processing method, and recording medium

Abstract

Extracts features easily and accurately, even without training data. [Solution] The information processing device comprises: a first selection unit for selecting data to be analyzed; a second selection unit for selecting hint data related to features of interest; a model construction unit for constructing a machine learning model that extracts a first feature contained in the data to be analyzed and a second feature contained in the hint data based on the data to be analyzed selected by the first selection unit and the hint data selected by the second selection unit; a first index calculation unit for calculating a first index for evaluating the first feature; a second index calculation unit for calculating a second index for evaluating the second feature; and a model learning unit for updating the weights of the machine learning model based on the first and second indexes.

Description

Technical Field

[0001] One embodiment of the present invention relates to an information processing apparatus, an information processing method, and a recording medium.

Background Art

[0002] Machine learning models for extracting feature amounts included in input data have been proposed. Existing feature amount extraction processing includes a learning stage in which a plurality of teacher data with known feature amounts are input to the model and the weights of the model are updated repeatedly, and an inference stage in which data to be inferred is input to the learned model to extract feature amounts.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the learning stage of existing feature amount extraction processing, teacher data is often required. If appropriate teacher data cannot be prepared, the extraction accuracy of feature amounts in the inference stage cannot be improved.

[0005] For example, when constructing a model for defect extraction in the manufacturing process of semiconductor devices, it is not easy to prepare a large number of images including various defects that may occur in the manufacturing process as teacher data.

[0006] Therefore, one embodiment of the present invention provides an information processing apparatus, an information processing method, and a recording medium that can extract feature amounts simply and accurately without teacher data. [Means for solving the problem]

[0007] To solve the above problems, according to one embodiment of the present invention, a first selection unit for selecting data to be analyzed, A second selection unit selects hint data related to the features to be of interest, A model building unit constructs a machine learning model that extracts a first feature from the data to be analyzed and a second feature from the hint data based on the data to be analyzed selected by the first selection unit and the hint data selected by the second selection unit, A first indicator calculation unit calculates a first indicator for evaluating the first feature, A second indicator calculation unit calculates a second indicator for evaluating the aforementioned second feature, An information processing device is provided, comprising a model learning unit that updates the weights of the machine learning model based on the first and second indicators. [Brief explanation of the drawing]

[0008] [Figure 1] A functional block diagram of the learning phase of the information processing device according to the first embodiment. [Figure 2] A functional block diagram of the inference stage of the information processing device according to the first embodiment. [Figure 3] A flowchart showing the processing operation of the information processing device according to the first embodiment during the learning phase. [Figure 4] This diagram shows the structure of Vision Transformer, an example of a machine learning model built by the Model Building Department. [Figure 5] A diagram showing the internal structure of a transformer encoder. [Figure 6] Functional block diagram of the learning phase of the information processing device according to the first modified example. [Figure 7] Functional block diagram of the learning phase of the information processing device according to the second modified example. [Figure 8] Functional block diagram of the learning phase of the information processing device according to the third modified example. [Figure 9] Functional block diagram of the learning stage of the information processing apparatus according to the fourth modification example. [Figure 10] Functional block diagram of the learning stage of the information processing apparatus according to the fifth modification example. [Figure 11] Functional block diagram of the learning stage of the information processing apparatus according to the sixth modification example. [Figure 12] Functional block diagram of the learning stage of the information processing apparatus according to the seventh modification example. [Figure 13] Functional block diagram of the learning stage of the information processing apparatus according to the eighth modification example. [Figure 14] Functional block diagram of the learning stage of the information processing apparatus according to the second embodiment. [Figure 15] Flowchart showing the processing operations in the learning stage of the information processing apparatus according to the second embodiment.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of an information processing apparatus, an information processing method, and a recording medium will be described with reference to the drawings. Hereinafter, the description will focus on the main components of the information processing apparatus, but there may be components and functions that are not shown or described in the information processing apparatus. The following description does not exclude components and functions that are not shown or described.

[0010] (First Embodiment) FIG. 1 is a functional block diagram of the learning stage of the information processing apparatus 1 according to the first embodiment. At least some of the functional blocks of the information processing apparatus 1 shown in FIG. 1 are configured by software or hardware. More specifically, the information processing apparatus 1 according to the first embodiment is provided, for example, in the form of a program executable by a computer, and by executing this program by the computer, the processing shown in the functional block diagram of FIG. 1 is performed. Alternatively, at least some of the functional blocks of the information processing apparatus 1 shown in FIG. 1 can also be configured by hardware.

[0011] As shown in FIG. 1, in the learning stage, the information processing apparatus 1 according to the first embodiment includes an analysis target data selection unit 2, a hint data selection unit 3, a model construction unit 4, an analysis feature separation determination unit 5, a hint extraction determination unit 6, and a model learning unit 7.

[0012] The analysis target data selection unit 2 selects two or more input data from the analysis target data set input from the outside. The analysis target data set includes a plurality of analysis target data. In this specification, the input data for which features are to be extracted is called analysis target data, and the analysis target data selection unit 2 may be called the first selection unit. In this specification, an example in which the input data is image data will be mainly described, but the input data is not necessarily limited to image data.

[0013] The hint data selection unit 3 selects hint data related to the feature amount to be noted from the hint data set input from the outside. The hint data set includes a plurality of hint data. Each hint data is not the feature amount to be noted itself, but data including a hint that can analogize the feature amount to be noted. For example, a feature amount obtained by adding some noise to the feature amount to be noted may be used as hint data. Alternatively, a feature amount similar to the feature amount to be noted may be used as hint data. In this specification, the hint data selection unit 3 may be called the second selection unit.

[0014] No correct label is attached to the analysis target data selected by the analysis target data selection unit 2 and the hint data selected by the hint data selection unit 3. The correct label is identification information corresponding to the feature amount. In the learning stage, the information processing apparatus 1 according to the first embodiment performs learning of the machine learning model without using teacher data with correct labels attached.

[0015] The model building unit 4 constructs a machine learning model that extracts the target features included in the target data and the hint features included in the hint data, based on the target data selected by the data selection unit 2 and the hint data selected by the hint data selection unit 3. The model building unit 4 constructs an existing machine learning model, such as a CNN (Convolutional Neural Network) or a ViT (Vision Transformer). The specific type of machine learning model constructed by the model building unit 4 is arbitrary and is not limited to a specific type of model such as a CNN. In this specification, the target features extracted by the model building unit 4 are sometimes referred to as first features, and the hint features as second features.

[0016] The feature separation determination unit 5 calculates a first index for evaluating the feature quantities to be analyzed extracted by the model building unit 4. The first index is, for example, the value of the loss function (loss value) that represents the degree of separation of the feature quantities to be analyzed. A smaller loss value indicates that the feature quantities to be analyzed are remarkably separated. Remarkably separated means that the differences between the feature quantities to be analyzed are clearly distinguishable. In this specification, the feature separation determination unit 5 may be referred to as the first index calculation unit. The feature separation determination unit 5 also assigns labels to each feature quantity to be analyzed.

[0017] The hint extraction decision unit 6 calculates a second indicator for evaluating the hint features extracted by the model building unit 4. The second indicator is, for example, the value of the loss function (loss value) that represents the degree to which hint features are extracted. A smaller loss value indicates higher accuracy in hint feature extraction. The hint extraction decision unit 6 assigns a label to each individual hint feature.

[0018] The model learning unit 7 updates the weights of the machine learning model based on the first and second indicators. In the first embodiment, the information processing device 1 repeatedly updates the weights of the machine learning model in the model learning unit 7 for each batch containing the dataset to be analyzed and the hint dataset.

[0019] Figure 2 is a functional block diagram of the inference stage of the information processing device 1 according to the first embodiment. As shown in Figure 2, the information processing device 1 according to the first embodiment includes a dataset acquisition unit 11, a trained machine learning model 12, and a feature output unit 13 during the inference stage. At least a portion of the functional blocks shown in Figure 2 are composed of software or hardware.

[0020] The dataset acquisition unit 11 acquires a dataset containing multiple datasets to be analyzed. The acquired dataset is input to the trained machine learning model 12. The machine learning model 12 extracts the features of each dataset included in the input dataset. The feature output unit 13 outputs the features of each individual dataset included in the dataset.

[0021] Thus, in the inference stage of the information processing device 1 according to the first embodiment, no hint data is used. Instead, the data to be analyzed is input into the trained machine learning model 12 to extract the features to be analyzed.

[0022] Figure 3 is a flowchart showing the processing operation of the information processing device 1 during the learning phase according to the first embodiment. The data to be analyzed selection unit 2 selects two or more data to be analyzed from the input data to be analyzed (step S1). Before or after step S1, the hint data selection unit 3 selects two or more hint data from the input hint data (step S2). As described above, the data to be analyzed selected in step S1 and the hint data selected in step S2 do not have correct labels assigned to them.

[0023] Next, the model building unit 4 inputs the data to be analyzed selected in step S1 and the hint data selected in step S2 into the machine learning model under training to extract the data to be analyzed and the hint features (step S3).

[0024] Next, the feature separation determination unit 5 calculates a value (loss value) of the loss function (first indicator) that represents the degree of separation of the feature quantities to be analyzed (step S4). Before or after step S4, the hint extraction determination unit 6 calculates a value (loss value) of the loss function (second indicator) that represents the degree of extraction of the hint feature quantities (step S5).

[0025] Next, the model learning unit 7 calculates the gradient of the weights at each stage of the machine learning model based on the two loss values ​​(first indicator and second indicator) calculated in steps S4 and S5, and updates the weights at each stage based on the calculated gradient (step S6). The machine learning model has a hierarchical structure, and the weights can be updated at each hierarchical stage. In step S6, the weights at each stage of the machine learning model are updated based on the loss values ​​calculated in steps S4 and S5.

[0026] During the learning phase, steps S1 to S6 are repeated for each of the multiple batches.

[0027] Figure 4 shows the structure of a vision transformer (ViT) 21, which is an example of a machine learning model constructed by the model building unit 4. As shown in Figure 4, the vision transformer 21 comprises a patch division unit 22, a patch embedding unit 23, a class token coupling unit 24, a position information embedding unit 25, a transformer encoder 26, and an MLP (Multi Layer Perception) head 27.

[0028] The patch division unit 22 divides the input image data into multiple patch images, flattens each patch image, and converts it into a one-dimensional vector. This one-dimensional vector is called a token.

[0029] The patch embedding unit 23 inputs the transformed tokens into a fully connected layer for classification. The class token merging unit 24 combines class tokens with the classified tokens. The location information embedding unit 25 combines location information with each token. The tokens with combined location information are input into the transformer encoder 26. The transformer encoder 26 extracts feature quantities from the input image data.

[0030] Figure 5 shows the internal configuration of the transformer encoder 26. As shown in Figure 5, the transformer encoder 26 has a first normalization unit 31, a multi-head self-attention unit 32, a second normalization unit 33, and an MLP unit 34.

[0031] The first normalization unit 31 performs normalization using the mean and standard deviation of the token itself, which is combined with the location information in the location information embedding unit 25.

[0032] The multi-head self-attention unit 32 calculates self-attention (self-relevance) in parallel using multiple heads and extracts features by determining the attention (relevance) between each patch image. As described later, self-attention can be calculated using any of the multiple heads. The extracted features differ depending on which head is used. An adder 30a is placed on the output side of the multi-head self-attention unit 32. This adder 30a performs residual connection by adding the output of the multi-head self-attention unit 32 and the input of the first normalization unit 31. Residual connection is provided to prevent the loss of processing effects for each set by skipping some sets and propagating the input of the first normalization unit 31 to other sets when multiple sets of the first normalization unit 31 and the multi-head self-attention unit 32 are connected in cascaded order.

[0033] The second normalization unit 33 normalizes the feature quantities extracted by the multi-head self-attention unit 32.

[0034] The MLP unit 34 has a fully connected layer, an activation function, and dropout, and classifies the feature quantities normalized by the second normalization unit 33. An adder 30b is placed on the output side of the MLP unit 34. This adder 30b performs residual connection, adding the output of the MLP unit 34 and the input of the second normalization unit 33. Residual connection is provided when multiple pairs of second normalization unit 33 and MLP unit 34 are connected in cascaded order, in order to skip some pairs and propagate the input of the second normalization unit 33 to other pairs so that the processing effect of each pair is not lost.

[0035] The MLP head 27 outputs the feature quantities extracted by the transformer encoder 26.

[0036] The functional block diagram of the learning stage of the information processing device 1 according to the first embodiment shown in Figure 1 can be modified in various ways. Representative modifications of the functional block diagram of the learning stage will be described below.

[0037] Figure 6 is a functional block diagram of the learning stage of the information processing device 1a according to the first modified example. The information processing device 1a according to the first modified example shown in Figure 6 includes an automatic hint data extraction unit 8 in addition to the configuration of the information processing device 1 shown in Figure 1. The automatic hint data extraction unit 8 selects two or more data to be analyzed from the data set to be analyzed input to the data to be analyzed selection unit 2. In this specification, the automatic hint data extraction unit 8 may be referred to as the hint extraction unit.

[0038] More specifically, the hint data automatic extraction unit 8 generates a histogram representing the feature distribution of two or more data points selected from the data set to be analyzed. This histogram is, for example, the entropy of the multiple data points to be analyzed. Entropy, also known as average information, is an indicator that represents the disorder or uncertainty of information. The more disordered or uncertain the information, the higher the entropy. Based on the histogram representing the feature distribution of the multiple data points to be analyzed, the hint data automatic extraction unit 8 automatically extracts two or more data points that correspond to outliers with low frequency.

[0039] The automatic hint data extraction unit 8 automatically extracts two or more data points for analysis, which are then sent to the hint data selection unit 3. The hint data selection unit 3 selects the two or more data points for analysis automatically extracted by the automatic hint data extraction unit 8 as hint data.

[0040] Thus, in the information processing device 1a according to the first modified example, the processing of the hint data selection unit 3 can be simplified by providing an automatic hint data extraction unit 8. The automatic hint data extraction unit 8 automatically extracts two or more data points to be analyzed that correspond to outliers in the histogram representing the feature distribution of the data set to be analyzed, making it easy to extract hint data related to the data to be analyzed. Hint data is necessary to quickly and accurately extract the features to be analyzed from the data to be analyzed, and careful consideration is required when selecting hint data. However, since the outliers in the histogram mentioned above have a high correlation with the data to be analyzed, automatically extracting hint data from the outliers in the histogram makes it possible to quickly and accurately extract the features to be analyzed from the data to be analyzed.

[0041] Figure 7 is a functional block diagram of the learning stage of the information processing device 1b according to the second modified example. The information processing device 1b according to the second modified example shown in Figure 7 includes an automatic hint data expansion unit 9 in addition to the configuration of the information processing device 1 shown in Figure 1. The automatic hint data expansion unit 9 selects two or more data to be analyzed whose features are similar to existing hint data. Existing hint data used is, for example, data generated by the user and stored in a memory unit not shown. Alternatively, as described in the first modified example, data automatically extracted from outliers of a histogram may be used as existing hint data. The automatic hint data expansion unit 9 may also select data to be analyzed that has features extracted by a trained machine learning model 12, which is provided separately from the machine learning model in the learning stage of the model building unit 4.

[0042] The two or more data points selected by the automatic hint data expansion unit 9 are sent to the hint data selection unit 3. The hint data selection unit 3 selects the two or more data points selected by the automatic hint data expansion unit 9 as hint data.

[0043] The hint data automatic expansion unit 9 selects two or more data points for analysis from the analysis target dataset for each batch.

[0044] Thus, in the information processing device 1b according to the second modified example, by providing an automatic hint data expansion unit 9, two or more data to be analyzed that have similar characteristics to existing hint data can be easily selected as new hint data. By providing the automatic hint data expansion unit 9, the number of hint data can be increased without inconveniencing the user.

[0045] Figure 8 is a functional block diagram of the learning stage of the information processing device 1c according to the third modified example. The information processing device 1c according to the third modified example shown in Figure 8 includes an optimal hint data selection unit 14 in addition to the configuration of the information processing device 1 shown in Figure 1. The optimal hint data selection unit 14 selects N data points (N is an integer between 2 and M) whose features approximate each other from among the M data points (M is any integer greater than or equal to 2) selected by the data selection unit 2 for each batch. The optimal hint data selection unit 14 may also select N data points whose features approximate each other using a trained machine learning model 12, which is provided separately from the machine learning model in the learning stage of the model building unit 4.

[0046] The N data points to be analyzed selected by the optimal hint data selection unit 14 are sent to the hint data selection unit 3. The hint data selection unit 3 selects the N data points to be analyzed selected by the optimal hint data selection unit 14 as hint data.

[0047] Thus, in the information processing device 1c according to the third modified example, the data selection unit 2 selects N data points for analysis that have similar feature quantities from among the M data points for analysis selected by the data selection unit 2 for each batch, thereby simplifying the processing of the hint data selection unit 3.

[0048] Figure 9 is a functional block diagram of the learning stage of the information processing device 1d according to the fourth modified example. The information processing device 1d according to the fourth modified example shown in Figure 9 includes an automatic hint generation unit 15 in addition to the configuration of the information processing device 1 shown in Figure 1. The automatic hint generation unit 15 generates two or more new hint data. For example, the automatic hint generation unit 15 generates new hint data based on information handwritten or entered by the user. Alternatively, the automatic hint generation unit 15 may generate new hint data based on previously selected hint data. Alternatively, the automatic hint generation unit 15 may generate new hint data using a data generator (not shown) that adds random noise to existing data to be analyzed. In this specification, the automatic hint generation unit 15 may be referred to as the hint generation unit.

[0049] The hint automatic generation unit 15 generates two or more hint data, which are then sent to the hint data selection unit 3. The hint data selection unit 3 selects two or more hint data generated by the hint automatic generation unit 15.

[0050] Thus, in the information processing device 1d according to the fourth modified example, the effort of inputting a hint dataset from an external source is eliminated in order to generate new hint data.

[0051] The functions of the information processing devices 1 to 1d described in the first to fourth modified examples above can be combined in any way. That is, at least two of the following may be added to the information processing device 1 shown in Figure 1: the automatic hint data extraction unit 8 in Figure 6, the automatic hint data expansion unit 9 in Figure 7, the optimal hint data selection unit 14 in Figure 8, or the automatic hint generation unit 15 in Figure 9.

[0052] The machine learning model constructed by the model building unit 4 of the information processing devices 1 to 1d shown in Figures 1, 6 to 9 above extracts the target features from the data to be analyzed, as well as hint features from the hint data, using updatable common weights. In contrast to this, a second weight for extracting hint features may be provided separately from the first weight for extracting the target features.

[0053] Figure 10 is a functional block diagram of the learning stage of the information processing device 1e according to the fifth modified example. The information processing device 1e according to the fifth modified example shown in Figure 10 includes a model building unit 4a that builds a machine learning model with a different configuration from that shown in Figure 1. The machine learning model according to the fifth modified example has a second weight for extracting hint features, separate from the first weight for extracting the features to be analyzed.

[0054] The model learning unit 7 updates the first weight for extracting the feature to be analyzed based on the loss value of the loss function calculated by the analysis feature separation judgment unit 5, and updates the second weight for extracting hint features based on the loss value of the loss function calculated by the hint extraction judgment unit 6.

[0055] The model building unit 4a extracts the target features using the first weights updated by the model learning unit 7, and extracts hint features using the second weights updated by the model learning unit 7. Note that the model building unit 4a in Figure 10 can also be applied to the model building units 4 in Figures 1, 6 to 9 described above.

[0056] Thus, in the information processing device 1e according to the fifth modified example, a second weight for extracting hint features is provided separately from the first weight for extracting the feature to be analyzed. Therefore, the feature to be analyzed and the hint features can be extracted with greater accuracy than if they were extracted using a common weight.

[0057] Figure 11 is a functional block diagram of the learning stage of the information processing device 1f according to the sixth modified example. The information processing device 1f according to the sixth modified example shown in Figure 11 includes a model update processing unit 16 in addition to the configuration of the information processing device 1 in Figure 1. The model update processing unit 16 has a classifier 17, an intermediate value update unit 18, and a feature quantity update unit 19.

[0058] The classifier 17 updates the machine learning model by aggregating the M hint data (where M is an integer greater than or equal to 2) selected by the hint data selection unit 3 into N hint data (where N is greater than or equal to 2 and less than or equal to M) as needed, and classifying them accordingly.

[0059] The intermediate value update unit 18 updates some intermediate values ​​for determining the target feature and hint features based on the N hint data classified by the classifier 17.

[0060] The feature update unit 19 extracts the feature to be analyzed and the hint feature based on some of the intermediate values ​​updated by the intermediate value update unit 18.

[0061] Thus, in the information processing device 1f according to the sixth modified example, depending on the type of hint data, a portion of the intermediate values ​​of the machine learning model can be updated, and then the features to be analyzed and the hint features can be extracted.

[0062] Figure 12 is a functional block diagram of the learning stage of the information processing device 1g according to the seventh modified example. The information processing device 1g according to the seventh modified example shown in Figure 12 includes a head selection unit 20 in addition to the configuration of the information processing device 1 in Figure 1.

[0063] While the machine learning models constructed by the model building units 4 of the information processing devices 1a to 1f shown in Figures 1, 6 to 11 above can be of any form, the model building unit 4 of the information processing device 1g shown in Figure 12 is based on the premise of constructing a vision transformer 21 having the internal configuration shown in Figures 4 and 5.

[0064] As shown in Figure 5, the vision transformer 21 has a multi-head self-attention unit 32. The multi-head self-attention unit 32 has a plurality of arbitrarily selectable heads.

[0065] The information processing device 1g according to the seventh modified example shown in Figure 12 includes a head selection unit 20. The head selection unit 20 aggregates and classifies the M hint data (where M is an integer of 2 or more) selected by the hint data selection unit 3 into N hint data (where N is 2 or more and less than or equal to M) as needed.

[0066] In the vision transformer 21 constructed by the model building unit 4 in Figure 12, N hint data classified by the head selection unit 20 is input to N of the multiple heads of the multi-head self-attention unit 32 shown in Figure 5, while the other heads are disabled.

[0067] Thus, in the seventh modified example of the information processing device 1g, depending on the type of hint data, the head to be enabled from among the multiple heads of the multi-head self-attention unit 32 can be switched, and feature quantities corresponding to the hint data can be extracted.

[0068] Figure 13 is a functional block diagram of the learning stage of the information processing device 1h according to the eighth modified example. The information processing device 1h according to the eighth modified example shown in Figure 13 includes an interaction unit 40 in addition to the configuration of the information processing device 1 in Figure 1. The interaction unit 40 interacts the hint data selected by the hint data selection unit 3 with the data to be analyzed selected by the data to be analyzed 2 and inputs it to the model construction unit 4. It also interacts the data to be analyzed and the hint features extracted by the machine learning model and inputs them to the analysis feature separation judgment unit 5 and the hint extraction judgment unit 6.

[0069] More specifically, the interaction section 40 comprises first to fourth interaction sections 41 to 44.

[0070] The first interaction unit 41 inputs the data to be analyzed, which is created by interacting the data to be analyzed selected by the data to be analyzed in

[0071] The second interaction unit 42 inputs hint data to the model building unit 4, which is created by interacting the hint data selected by the hint data selection unit 3 with the data to be analyzed selected by the data to be analyzed selection unit 2.

[0072] The third interaction unit 43 inputs the analysis target features, obtained by interacting the analysis target features extracted by the machine learning model with the hint features extracted by the machine learning model, into the analysis feature separation judgment unit 5.

[0073] The fourth interaction unit 44 inputs hint features obtained by interacting hint features extracted by the machine learning model with analysis target features extracted by the machine learning model to the hint extraction decision unit 6.

[0074] Thus, in the information processing device 1h according to the eighth modified example, the data to be analyzed and the hint data are made to interact and input to the model building unit 4, and the values ​​of the loss function of the data to be analyzed and the loss function of the hint features are calculated by making the data to be analyzed and the hint features extracted by the machine learning model interact, so that the data to be analyzed can be extracted while taking the hint data into consideration.

[0075] Thus, in the first embodiment, since the machine learning model is trained using hint data related to the features of interest, the machine learning model can be trained efficiently without training data that has been assigned correct labels.

[0076] (Second embodiment) Figure 14 is a functional block diagram of the learning stage of the information processing device 1i according to the second embodiment. The information processing device 1i according to the second embodiment shown in Figure 14 includes a hint adjustment unit 35 in addition to the configuration of the information processing devices 1 to 1h in Figure 1. The hint adjustment unit 35 adjusts the hint data selected by the hint data selection unit 3 or at least one of the weights for calculating the loss function (second indicator) calculated by the hint extraction decision unit 6, based on at least one of the relationship between the loss function (first indicator) calculated by the analysis feature separation decision unit 5 and the loss function (second indicator) calculated by the hint extraction decision unit 6, or the relationship between the distance between the analysis target features and the distance between the hint features.

[0077] For example, the hint adjustment unit 35 may perform a first process in which, if the value obtained by dividing the loss value calculated by the hint extraction determination unit 6 by the loss value calculated by the analysis feature separation determination unit 5 exceeds a first threshold, it reduces the weight of the loss value calculated by the hint extraction determination unit 6, and if it is below the first threshold, it increases the weight of the loss value calculated by the hint extraction determination unit 6.

[0078] Alternatively, the hint adjustment unit 35 may perform a second process in which, if the value obtained by dividing the distance between hint features by the distance between the features to be analyzed exceeds the second threshold, it reduces the weight of the loss value calculated by the hint extraction judgment unit 6, and if it is below the second threshold, it increases the weight of the loss value calculated by the hint extraction judgment unit 6.

[0079] Alternatively, if the hint data is image data, the hint adjustment unit 35 may perform a third process in which, if the value obtained by dividing the loss value calculated by the hint extraction determination unit 6 by the loss value calculated by the analysis feature separation determination unit 5 exceeds the third threshold, the contrast of the hint data to be selected next by the hint data selection unit 3 is increased, and if it is below the third threshold, the contrast of the hint data to be selected next by the hint data selection unit 3 is decreased.

[0080] Alternatively, if the hint data is image data, the hint adjustment unit 35 may perform a fourth process in which, if the value obtained by dividing the distance between hint features by the distance between the features to be analyzed exceeds the fourth threshold, the contrast of the hint data to be selected next by the hint data selection unit 3 is increased, and if it is below the fourth threshold, the contrast of the hint data to be selected next by the hint data selection unit 3 is decreased.

[0081] The hint adjustment unit 35 performs, for example, at least one of the first to fourth processes described above. For example, the hint data selection unit 3 selects the hint data adjusted by the hint adjustment unit 35. Also, for example, the hint extraction decision unit 6 adjusts the hint features using the weights adjusted by the hint adjustment unit 35.

[0082] Figure 15 is a flowchart showing the processing operations of the information processing device 1i during the learning phase according to the second embodiment. In Figure 15, the same steps as in Figure 3 are given the same step numbers, and the following explanation will focus on processes that differ from those in Figure 3.

[0083] The hint adjustment unit 35 adjusts at least one of the weights used to calculate the loss function (second indicator) calculated by the hint data selection unit 3 or the loss function (second indicator) calculated by the hint extraction judgment unit 6, based on the loss function (first indicator) calculated in step S4, the loss function (second indicator) calculated in step S5, the distances between the analysis target features extracted in step S3, and the distances between the hint features (step S7).

[0084] Thus, in the second embodiment, at least one of the following is performed based on the extracted features and calculated loss values: adjusting the hint data or adjusting the weights for calculating the loss values ​​of the hint features. This allows for easy and accurate optimization of both the hint data and hint features.

[0085] At least a portion of the information processing devices 1 to 1i described in each of the embodiments above may be configured as hardware or as software. In the case of software configuration, a program that implements at least a portion of the functions of the information processing device 1 may be stored on a recording medium such as a flexible disk or CD-ROM, loaded into a computer, and executed. The recording medium is not limited to removable ones such as magnetic disks or optical disks, but may also be a fixed recording medium such as a hard disk drive or memory.

[0086] Furthermore, programs that implement at least some of the functions of information processing devices 1 to 1i may be distributed via communication lines such as the Internet (including wireless communication). In addition, the same programs may be encrypted, modulated, or compressed and distributed via wired or wireless lines such as the Internet, or stored on a recording medium.

[0087] [Note] [Item 1] A first selection unit for selecting data to be analyzed, A second selection unit selects hint data related to the features to be of interest, A model building unit constructs a machine learning model that extracts a first feature from the data to be analyzed and a second feature from the hint data based on the data to be analyzed selected by the first selection unit and the hint data selected by the second selection unit, A first indicator calculation unit calculates a first indicator for evaluating the first feature, A second indicator calculation unit calculates a second indicator for evaluating the aforementioned second feature, The system comprises a model learning unit that updates the weights of the machine learning model based on the first and second indicators, Information processing device. [Item 2] The system includes a hint adjustment unit that adjusts at least one of the hint data selected by the second selection unit or the weights for calculating the second indicator, based on at least one of the relationship between the first indicator and the second indicator or the relationship between the distance between the first features and the distance between the second features. The information processing device described in item 1. [Item 3] The second selection unit selects the hint data adjusted by the hint adjustment unit. The information processing device described in item 2. [Item 4] The second indicator calculation unit adjusts the second indicator using the weight adjusted by the hint adjustment unit. The information processing device described in item 2. [Item 5] The hint adjustment unit reduces the weight of the second indicator if the value obtained by dividing the second indicator by the first indicator exceeds the first threshold, and increases the weight of the second indicator if the value is less than or equal to the first threshold. The information processing device described in item 2. [Item 6] The hint adjustment unit reduces the weight of the second indicator if the value obtained by dividing the distance between the second features by the distance between the first features exceeds the second threshold, and increases the weight of the second indicator if the value is below the second threshold. The information processing device described in item 2. [Item 7] The aforementioned hint data is image data. The hint adjustment unit increases the contrast of the hint data to be selected next by the second selection unit if the value obtained by dividing the second index by the first index exceeds the third threshold, and decreases the contrast of the hint data to be selected next by the second selection unit if the value is below the third threshold. The information processing device described in item 2. [Item 8] The aforementioned hint data is image data. The hint adjustment unit increases the contrast of the hint data to be selected next by the second selection unit if the value obtained by dividing the distance between the second features by the distance between the first features exceeds the second threshold, and decreases the contrast of the hint data to be selected next by the second selection unit if the value is below the third threshold. The information processing device described in item 2. [Item 9] The first selection unit selects two or more of the data to be analyzed from the multiple data to be analyzed that have been input. The second selection unit selects two or more hint data from among the multiple hint data that are input separately from the multiple data to be analyzed. An information processing device as described in any one of items 1 through 8. [Item 10] The first selection unit includes a hint extraction unit that extracts two or more of the data to be analyzed based on the feature distribution of multiple data to be analyzed input to the first selection unit, The second selection unit selects the two or more data to be analyzed extracted by the hint extraction unit as two or more hint data. An information processing device as described in any one of items 1 through 9. [Item 11] The hint extraction unit extracts two or more hint data from outliers in the feature distribution of the multiple data to be analyzed. The information processing device described in item 10. [Item 12] The first selection unit further comprises a hint extension unit that extracts two or more data to be analyzed from among a plurality of data to be analyzed whose features are similar to the hint data, The second selection unit selects the two or more data to be analyzed extracted by the hint extension unit as two or more hint data. An information processing device as described in any one of items 1 through 9. [Item 13] The process by which the first selection unit selects two or more of the data to be analyzed, and the process by which the second selection unit selects two or more of the hint data, are repeated for each batch. For each batch, the system further includes an optimal hint data selection unit that selects two or more data sets for analysis that have similar characteristics from among the multiple data sets for analysis input to the first selection unit, The second selection unit selects, for each batch, the two or more data to be analyzed selected by the optimal hint data selection unit as two or more hint data. An information processing device as described in any one of items 1 through 9. [Item 14] The system includes a hint generation unit that generates the aforementioned hint data, The second selection unit selects the hint data generated by the hint generation unit. An information processing device as described in any one of items 1 through 9. [Item 15] The model learning unit updates the first weight for extracting the first feature based on the first indicator, and updates the second weight for extracting the second feature based on the second indicator. The model building unit extracts the first feature using the first weight updated by the model learning unit, and extracts the second feature using the second weight updated by the model learning unit. An information processing device as described in any one of items 1 through 14. [Item 16] The system includes a model update processing unit that updates the machine learning model by classifying the M hint data (where M is an integer greater than or equal to 2) selected by the second selection unit into N hint data (where N is greater than or equal to 2 and less than or equal to M), The aforementioned model update processing unit, An intermediate value update unit updates some intermediate values ​​for determining the first feature and the second feature based on the N hint data, The system includes a feature update unit that updates some of the intermediate values ​​in the intermediate value update unit and then extracts the first and second feature quantities. An information processing device as described in any one of items 1 through 14. [Item 17] The system includes a classifier that classifies the M hint data (where M is an integer greater than or equal to 2) selected by the second selection unit into N hint data (where N is greater than or equal to 2 and less than or equal to M), The aforementioned model building unit constructs a vision transformer having a multi-head self-attention unit, Of the N or more heads in the multi-head self-attention unit, the N heads are input with the N hint data classified by the classifier, and the other heads are disabled. An information processing device as described in any one of items 1 through 14. [Item 18] The system includes an interaction unit that interacts the data to be analyzed selected by the first selection unit with the hint data selected by the second selection unit and inputs it into the model building unit, and an interaction unit that interacts the first and second features extracted by the machine learning model and inputs them into the first and second indicator calculation units. An information processing device as described in any one of items 1 through 17. [Item 19] The first indicator is the value of the first loss function, which represents the degree of separation between the first features. The aforementioned second indicator is the value of the second loss function, which represents the degree to which the aforementioned second feature is extracted. An information processing device as described in any one of items 1 through 18. [Item 20] The data to be analyzed selected by the first selection unit does not have a label assigned to it. The hint data selected by the second selection unit is not labeled. The first indicator calculation unit assigns a label corresponding to the first feature, The second indicator calculation unit assigns a label corresponding to the second feature. An information processing device as described in any one of items 1 through 19. [Item 21] On the computer, The procedure for selecting the data to be analyzed, The procedure for selecting hint data related to the features to focus on, A procedure for constructing a machine learning model that extracts a first feature contained in the data to be analyzed and a second feature contained in the hint data, based on the selected data to be analyzed and the selected hint data, The procedure for calculating the first metric to evaluate the aforementioned first feature, The procedure for calculating a second metric to evaluate the aforementioned second feature, The procedure for updating the weights of the machine learning model based on the first and second indicators is performed. Information processing methods. [Item 22] On the computer, The procedure for selecting the data to be analyzed, The procedure for selecting hint data related to the features to focus on, A procedure for constructing a machine learning model that extracts a first feature contained in the data to be analyzed and a second feature contained in the hint data, based on the selected data to be analyzed and the selected hint data, The procedure for calculating the first metric to evaluate the aforementioned first feature, The procedure for calculating a second metric to evaluate the aforementioned second feature, A computer-readable recording medium containing a program for recording a procedure for updating the weights of the machine learning model based on the first and second indicators.

[0088] The aspects of this disclosure are not limited to the individual embodiments described above, but include various modifications that a person skilled in the art could conceive, and the effects of this disclosure are not limited to those described above. In other words, various additions, modifications, and partial deletions are possible, as long as they do not depart from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents. [Explanation of symbols]

[0089] 1, 1a, 1b, 1c, 1d, 1e, 1f, 1g, 1h, 1i Information processing device, 2 Analysis target data selection unit, 3 Hint data selection unit, 4, 4a Model construction unit, 5 Analysis feature separation judgment unit, 6 Hint extraction judgment unit, 7 Model learning unit, 8 Hint data automatic extraction unit, 9 Hint data automatic expansion unit, 11 Dataset acquisition unit, 12 Machine learning model, 13 Feature output unit, 14 Optimal hint data selection unit, 15 Hint automatic generation unit, 16 Model update processing unit, 17 Classifier, 18 Intermediate value update unit, 19 Feature update unit, 20 Head selection unit, 21 Vision transformer, 22 Patch splitting unit, 23 Patch embedding unit, 24 Class token coupling unit, 25 Position information embedding unit, 26 Transformer encoder, 27 MLP head, 30a, 30b Adder, 31 First normalization unit, 32 Multi-head self-attention unit, 33 Second normalization unit, 34 MLP unit, 35 Hint adjustment unit, 40 Interaction unit, 41 First interaction unit, 42 Second interaction unit, 43 Third interaction unit, 44 Fourth interaction unit

Claims

1. A first selection unit for selecting data to be analyzed, A second selection unit selects hint data related to the features to be of interest, A model building unit constructs a machine learning model that extracts a first feature from the data to be analyzed and a second feature from the hint data based on the data to be analyzed selected by the first selection unit and the hint data selected by the second selection unit. A first index calculation unit calculates a first index for evaluating the first feature, A second indicator calculation unit calculates a second indicator for evaluating the aforementioned second feature, The model includes a model learning unit that updates the weights of the machine learning model based on the first and second indicators, Information processing device.

2. The system includes a hint adjustment unit that adjusts at least one of the hint data selected by the second selection unit or the weights for calculating the second indicator, based on at least one of the relationship between the first indicator and the second indicator or the relationship between the distance between the first features and the distance between the second features. The information processing apparatus according to claim 1.

3. The second selection unit selects the hint data adjusted by the hint adjustment unit. The information processing apparatus according to claim 2.

4. The second indicator calculation unit adjusts the second indicator using the weight adjusted by the hint adjustment unit. The information processing apparatus according to claim 2.

5. The hint adjustment unit reduces the weight of the second indicator if the value obtained by dividing the second indicator by the first indicator exceeds a first threshold, and increases the weight of the second indicator if the value is less than or equal to the first threshold. The information processing apparatus according to claim 2.

6. The hint adjustment unit reduces the weight of the second indicator if the value obtained by dividing the distance between the second features by the distance between the first features exceeds the second threshold, and increases the weight of the second indicator if the value is below the second threshold. The information processing apparatus according to claim 2.

7. The aforementioned hint data is image data. The hint adjustment unit increases the contrast of the hint data to be selected next by the second selection unit if the value obtained by dividing the second index by the first index exceeds the third threshold, and decreases the contrast of the hint data to be selected next by the second selection unit if the value is below the third threshold. The information processing apparatus according to claim 2.

8. The aforementioned hint data is image data. The hint adjustment unit increases the contrast of the hint data to be selected next by the second selection unit if the value obtained by dividing the distance between the second features by the distance between the first features exceeds the fourth threshold, and decreases the contrast of the hint data to be selected next by the second selection unit if the value is below the fourth threshold. The information processing apparatus according to claim 2.

9. The first selection unit selects two or more of the data to be analyzed from the multiple data to be analyzed that have been input. The second selection unit selects two or more hint data from among the multiple hint data that are input separately from the multiple data to be analyzed. The information processing apparatus according to claim 1.

10. The first selection unit includes a hint extraction unit that extracts two or more of the data to be analyzed based on the feature distribution of the multiple data to be analyzed input to the first selection unit, The second selection unit selects the two or more data to be analyzed extracted by the hint extraction unit as two or more hint data. The information processing apparatus according to claim 1.

11. The hint extraction unit extracts two or more hint data from outliers in the feature distribution of the multiple data to be analyzed. The information processing apparatus according to claim 10.

12. The first selection unit further comprises a hint extension unit that extracts two or more data to be analyzed from among a plurality of data to be analyzed whose features are similar to those of the hint data, The second selection unit selects the two or more data to be analyzed extracted by the hint extension unit as two or more hint data. The information processing apparatus according to claim 1.

13. The process by which the first selection unit selects two or more of the data to be analyzed, and the process by which the second selection unit selects two or more of the hint data, are repeated for each batch. For each batch, the system further includes an optimal hint data selection unit that selects two or more data sets for analysis that have similar characteristics from among the multiple data sets for analysis input to the first selection unit. The second selection unit selects two or more of the two or more data to be analyzed selected by the optimal hint data selection unit as hint data for each batch. The information processing apparatus according to claim 1.

14. The system includes a hint generation unit that generates the aforementioned hint data, The second selection unit selects the hint data generated by the hint generation unit. The information processing apparatus according to claim 1.

15. The model learning unit updates the first weight for extracting the first feature based on the first indicator, and updates the second weight for extracting the second feature based on the second indicator. The model building unit extracts the first feature using the first weight updated by the model learning unit, and extracts the second feature using the second weight updated by the model learning unit. The information processing apparatus according to claim 1.

16. The system includes a model update processing unit that updates the machine learning model by classifying the M hint data (where M is an integer of 2 or more) selected by the second selection unit into N hint data (where N is 2 or more and less than or equal to M), The aforementioned model update processing unit, An intermediate value update unit updates some intermediate values ​​for determining the first and second feature quantities based on the N hint data, The system includes a feature update unit that updates some of the intermediate values ​​in the intermediate value update unit and then extracts the first and second feature quantities. The information processing apparatus according to claim 1.

17. The system includes a classifier that classifies the M hint data (where M is an integer greater than or equal to 2) selected by the second selection unit into N hint data (where N is greater than or equal to 2 and less than or equal to M), The aforementioned model building unit constructs a vision transformer equipped with a multi-head self-attention unit, Of the N or more heads in the multi-head self-attention unit, the N heads are input with the N hint data classified by the classifier, and the other heads are disabled. The information processing apparatus according to claim 1.

18. The system includes an interaction unit that interacts the data to be analyzed selected by the first selection unit with the hint data selected by the second selection unit and inputs it into the model building unit, and an interaction unit that interacts the first and second features extracted by the machine learning model and inputs them into the first and second indicator calculation units. The information processing apparatus according to claim 1.

19. The first indicator is the value of the first loss function, which represents the degree of separation between the first features. The second indicator is the value of the second loss function, which represents the degree to which the second feature is extracted. The information processing apparatus according to claim 1.

20. The data to be analyzed selected by the first selection unit does not have a label assigned to it. The hint data selected by the second selection unit is not labeled. The first indicator calculation unit assigns a label corresponding to the first feature, The second indicator calculation unit assigns a label corresponding to the second feature. The information processing apparatus according to claim 1.

21. On the computer, The procedure for selecting the data to be analyzed, The procedure for selecting hint data related to the features to focus on, A procedure for constructing a machine learning model that extracts a first feature contained in the data to be analyzed and a second feature contained in the hint data, based on the selected data to be analyzed and the selected hint data, A procedure for calculating a first index to evaluate the aforementioned first feature, The procedure for calculating a second index to evaluate the aforementioned second feature, The procedure for updating the weights of the machine learning model based on the first and second indicators is performed. Information processing methods.

22. On the computer, The procedure for selecting the data to be analyzed, The procedure for selecting hint data related to the features to focus on, A procedure for constructing a machine learning model that extracts a first feature contained in the data to be analyzed and a second feature contained in the hint data, based on the selected data to be analyzed and the selected hint data, A procedure for calculating a first index to evaluate the aforementioned first feature, The procedure for calculating a second index to evaluate the aforementioned second feature, A computer-readable recording medium containing a program for executing a procedure for updating the weights of the machine learning model based on the first and second indicators.

Images