Data evaluation method, data evaluation device, and program

Abstract

To provide a technique for appropriately evaluating the uniqueness of data.SOLUTION: A data evaluation method includes: a generation step of generating evaluation embedded data in which evaluation target data is projected to embedded space by inputting the evaluation target data to a learning model on which machine learning is performed so as to project input data to the embedded space; a calculation step of calculating, using the evaluation embedded data and reference embedded data in which a plurality of pieces of reference data serving as references of the evaluation target data are projected to the embedded space, density of the evaluation target data in the embedded space to which each piece of reference data is projected; and an evaluation step of evaluating uniqueness of the evaluation target data on the basis of the density.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a data evaluation method, a data evaluation device, and a program. [Background technology]

[0002] Patent Document 1 discloses a technique for reducing the likelihood that images taken under similar conditions will be selected when selecting some images from a plurality of images that make up a video. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-161105 Summary of the Invention [Problem to be solved by the invention]

[0004] There is an increasing need to control the diversity of data contained in datasets such as image datasets. For example, the technology of Patent Document 1 merely reduces the likelihood that images captured under similar conditions will be selected, but does not necessarily result in a high diversity of selected images. In order to further increase the diversity of selected images, it is necessary to extract highly unique images with few similar images. Conversely, in order to reduce the diversity of selected images, it is necessary to extract images with low uniqueness. Therefore, a technology that can appropriately evaluate the uniqueness of data is desired. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a data evaluation method, comprising: a generation step of generating evaluation embedded data in which evaluation target data is projected into an embedding space by inputting the evaluation target data into a learning model that has been machine-learned to project input data into the embedding space, a calculation step of calculating, using the evaluation embedded data and reference embedded data in which a plurality of reference data serving as a reference for the evaluation target data are projected into the embedding space, a density of the evaluation target data in the embedding space into which each of the reference data is projected, and an evaluation step of evaluating the uniqueness of the evaluation target data based on the density.

[0006] According to a second aspect of the present disclosure, there is provided a data evaluation device. The data evaluation device includes a memory unit that stores a learning model that has been machine-learned to project features of input data into an embedding space, reference embedded data in which multiple reference data serving as a reference for data to be evaluated are projected into the embedding space by the learning model, and a processor. The processor executes the following processes: acquiring the data to be evaluated; inputting the data to be evaluated into the learning model to generate evaluation embedded data in which the data to be evaluated is projected into the embedding space; calculating, using the evaluation embedded data and the reference embedded data, the density of the data to be evaluated in the embedding space into which each reference data is projected; and evaluating the uniqueness of the data to be evaluated based on the density.

[0007] According to a third aspect of the present disclosure, there is provided a program that causes a computer to perform the following functions: inputting evaluation target data into a learning model that has been machine-learned to project features of the input data into an embedding space, thereby generating evaluation embedded data in which the evaluation target data is projected into the embedding space; using the evaluation embedded data and reference embedded data in which a plurality of reference data serving as a reference for the evaluation target data are projected into the embedding space, calculating a density of the evaluation target data in the embedding space into which each of the reference data is projected; and evaluating the uniqueness of the evaluation target data based on the density. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a data processing system. [Figure 2] FIG. 1 is a conceptual diagram illustrating a learning method for a learning model. [Figure 3] 10 is a flowchart of a pre-processing. [Figure 4] 10 is a flowchart of an evaluation process. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a probability density function. [Figure 6] FIG. 10 is an explanatory diagram showing an example of an evaluation result of evaluation target data. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. First embodiment: 1 is a block diagram showing a schematic configuration of a data processing system 50 according to the first embodiment. The data processing system 50 is used to evaluate the uniqueness of various types of data such as image data, spectral data, and character string data.

[0010] The uniqueness of data represents the degree to which the number of other data similar to the data in the data group to which the data belongs is small. The greater the number of other data similar to the data in the data group to which the data belongs, above a predetermined level, the higher the uniqueness of the data. Generally, whether or not two pieces of data are similar to each other can be evaluated based on the distance between the pieces of data or the similarity between the pieces of data. In this case, for example, Euclidean distance or Manhattan distance can be used as the distance between the pieces of data. Furthermore, for example, cosine similarity or similarity based on cross entropy can be used as the similarity. Furthermore, the uniqueness of each piece of data correlates with the diversity of each piece of data. Specifically, the greater the amount of unique data contained in a data group, the greater the diversity of each piece of data contained in the data group. Hereinafter, a data group with a high level of diversity of each piece of data or a data set with a high level of diversity of each piece of data will also be simply referred to as a "highly diverse data group" or a "highly diverse data set."

[0011] The data processing system 50 includes a data evaluation device 100. In this embodiment, the data evaluation device 100 is configured by a computer including one or more processors 101, a storage unit 102 including ROM and RAM, an input / output interface 103, and an internal bus 104. The processor 101, the storage unit 102, and the input / output interface 103 are connected via the internal bus 104 to enable bidirectional communication. The input / output interface 103 is connected to an output device 105 and an input device 106. In this embodiment, the storage unit 102 stores a program 155, a learning model 160, and reference embedded data 165. The processor 101 executes the program 155 stored in the storage unit 102 to realize various functions, including those of an acquisition unit 110, a generation unit 115, a calculation unit 120, an evaluation unit 125, and a learning execution unit 140.

[0012] The output device 105 outputs the evaluation results obtained by the data evaluation device 100. In this embodiment, the output device 105 is configured as a display device that outputs the evaluation results as visual information. The display device is configured, for example, by a liquid crystal panel or an organic EL panel. The output device 105 as a display device may be configured, for example, as a touch panel that can accept touch operations from the user Ur. In this case, the output device 105 may also function as the input device 106, for example. In other embodiments, the output device 105 may be configured, for example, as a speaker that outputs the evaluation results as audio information, or as a device that outputs the evaluation results by transmitting them to an external computer or recording medium.

[0013] The learning model 160 is trained to project input data into an embedding space. In the embedding space, each piece of data is projected so that the feature vectors of more similar data are closer in distance. The embedding space in this embodiment is a 64-dimensional vector space. The embedding space is also called a feature space. The feature vectors in the embedding space are also called embedding vectors or embedded representations.

[0014] In this embodiment, the learning model 160 has been trained by self-supervised learning (SSL). The learning model 160 is configured as a machine learning model using Resnet. More specifically, the learning model 160 is configured as a Resnet model that is a backbone included in an encoder in a network structure that has been trained using SimSiam. Details of the learning model 160 will be described later.

[0015] The acquiring unit 110 acquires a dataset to be evaluated. The dataset to be evaluated is a dataset including data to be evaluated. The data to be evaluated is data to be evaluated for uniqueness by the data evaluation device 100. In this embodiment, the dataset to be evaluated is made up of a plurality of data to be evaluated.

[0016] In this embodiment, the evaluation target dataset is included in an input dataset DS input by a user Ur via an input device 106. The input device 106 is configured by, for example, a mouse and a keyboard. Note that the input dataset DS in this embodiment includes, in addition to the evaluation target dataset, a training dataset and a reference dataset, which will be described later.

[0017] The generating unit 115 generates evaluation embedding data in which the evaluation target data is projected into the embedding space. Specifically, the generating unit 115 generates the evaluation embedding data by inputting the evaluation target data into the trained learning model 160. The evaluation embedding data may be generated for each evaluation target data. The generating unit 115 stores the generated evaluation embedding data in the storage unit 102.

[0018] Furthermore, the generator 115 in this embodiment generates reference embedded data 165 in which multiple reference data included in the reference dataset are projected into the embedding space. The reference data is data that serves as a reference for the evaluation target data. Specifically, the reference data is data that serves as a reference for calculating the density of the evaluation target data, which will be described later. It is preferable that at least a portion of the reference dataset corresponds to the evaluation target data. "The data correspond to each other" includes, for example, each piece of data being identical, each piece of data being the same class, each piece of data being generated using the same generative model, or one piece of data being training data for machine learning and the other piece of data being test data. Specifically, for example, with regard to the data class, if the evaluation target data is image data classified as "airplane," it is preferable that the reference dataset includes image data similarly classified as "airplane."

[0019] The calculation unit 120 calculates the density of the evaluation target data using the reference embedded data 165 and the evaluation embedded data. The "density of the evaluation target data" here refers to the density of the reference embedded data 165 at the coordinates where the evaluation embedded data is located in the embedding space into which each reference data is projected.

[0020] The calculation unit 120 in this embodiment calculates the density of the evaluation target data using kernel density estimation (KDE). Specifically, the calculation unit 120 first calculates a probability density function representing the probability distribution of each reference data in the embedded space using KDE, based on the embedded space into which each reference data is projected. The calculated probability density function represents the distribution density of data points in the embedded space. The calculation unit 120 then calculates the density of each evaluation target data based on the calculated probability density function and the vector coordinates of each evaluation data projected into the embedded space. Note that in other embodiments, the embedded space into which the probability density function is calculated may include not only each projected reference data, but also, for example, one or more projected evaluation target data.

[0021] The evaluation unit 125 evaluates the uniqueness of the evaluation target data based on the density of the evaluation target data calculated by the calculation unit 120. For example, when evaluation target data with a relatively high uniqueness is projected into the embedding space, the projected evaluation target data is located farther from each reference data in the embedding space. As a result, the density of the evaluation target data in the embedding space becomes relatively low. Conversely, when evaluation target data with a relatively high uniqueness is projected into the embedding space, the projected evaluation target data is located closer to each reference data in the embedding space. As a result, the density of the evaluation target data in the embedding space becomes relatively high. As such, the density and uniqueness of the evaluation target data are suitably correlated, so the evaluation unit 125 can appropriately evaluate the uniqueness of the evaluation target data based on the density of the evaluation target data.

[0022] In this embodiment, the evaluation unit 125 evaluates the uniqueness of the evaluation target data based on the result of normalizing the density of the evaluation target data. In this disclosure, "normalization" includes so-called min-max normalization and "standardization." Standardization is also called Z-score normalization. In this embodiment, the evaluation unit 125 evaluates the uniqueness of the evaluation target data based on the result of normalizing the density of the evaluation target data by Z-score normalization.

[0023] More specifically, the evaluation unit 125 first normalizes each density in the population by Z-score normalization, using the evaluation target data and each reference data as a population, so that the average density in the population is 0 and the variance is 1. Then, if the density of the normalized evaluation target data is equal to or greater than a predetermined first threshold, the evaluation unit 125 determines that the uniqueness of the evaluation target data is low. Furthermore, if the density of the normalized evaluation target data is equal to or less than a second threshold that is less than the first threshold, the evaluation unit 125 determines that the uniqueness of the evaluation target data is high. Furthermore, if the density of the normalized evaluation target data is greater than the second threshold and less than the first threshold, the evaluation unit 125 determines that the uniqueness of the evaluation target data is at an average level. Note that in other embodiments, the evaluation unit 125 does not have to evaluate the uniqueness of the evaluation target data using three levels as described above, but may instead use two levels, four levels, or more levels.

[0024] The learning execution unit 140 executes machine learning of the learning model 160. In this embodiment, the learning execution unit 140 uses at least a portion of a reference dataset as a training dataset to train the learning model 160. The training dataset is a dataset for training the learning model 160. The training dataset in this embodiment includes the entire reference dataset. That is, in this embodiment, the learning execution unit 140 has already trained using the entire reference dataset as a training dataset. In addition, in this embodiment, it can also be said that the portion of the training dataset and the entire reference dataset are the same data.

[0025] FIG. 2 is a conceptual diagram illustrating a learning method of the learning model 160 in this embodiment. FIG. 2 shows a model structure SM for realizing learning by SimSiam. The model structure SM includes an encoder En and a predictor Pd. The encoder En is configured by a neural network. The encoder En includes a backbone Bb and a projector Pj as a head. The projector Pj is configured by a fully connected layer including an output layer among the layers of the neural network that configures the encoder En. The predictor Pd is a neural network configured by two fully connected layers.

[0026] When input data Dt, such as image data, is input to a model structure SM, first and second data augmentations DA, which are different data augmentations, are performed on the input data Dt. The first data Dt1, which is the input data Dt subjected to the first data augmentation, and the second data Dt2, which is the input data Dt subjected to the second data augmentation, are each input to an encoder En. When the input data Dt is image data, the data augmentation DA may be, for example, vertical flipping, horizontal flipping, rotation, enlargement, reduction, cropping, translation, color jitter, brightness jitter, contrast jitter, saturation jitter, hue jitter, grayscaling, Gaussian blur, partial masking, etc. "Different data augmentations" means that the types or numbers of data augmentations DA to be performed are different. The types and numbers of data augmentations to be performed as the first and second data augmentations are each determined probabilistically.

[0027] The encoder En outputs a feature vector z1 of the input first data Dt1 and a feature vector z2 of the input second data Dt2.

[0028] The predictor Pd converts and outputs the input feature vector so that the feature vector based on the first data Dt1 matches the feature vector based on the second data Dt2. The feature vector z1 and feature vector z2 output from the encoder En are each input to the predictor Pd alternatively. FIG. 2 shows a situation in which the feature vector z1 is input to the predictor Pd. When the feature vector z1 is input to the predictor Pd, the predictor Pd outputs a feature vector p1 based on the feature vector z1. On the other hand, when the feature vector z2 is input to the predictor Pd, the predictor Pd outputs a feature vector p2 based on the feature vector z2. Note that the feature vector p2 is not shown in FIG. 2.

[0029] In the similarity evaluation SA, the similarity between a feature vector based on the first data Dt1 and a feature vector based on the second data Dt2 is calculated. Specifically, in the similarity evaluation SA, the similarity D(p1, z2) between feature vector p1 and feature vector z2 and the similarity D(p2, z1) between feature vector z1 and feature vector p2 are calculated. The similarity D(p1, z2) is expressed by the following formula (1). Note that the similarity D(p2, z1) is expressed by a formula obtained by replacing p1 with p2 and z2 with z1 in formula (1).

[0030]

number

[0031] In the model structure SM, learning is performed so as to minimize the loss function L. The loss function L is expressed by the following equation (2).

[0032]

number

[0033] In learning in the model structure SM, the parameters of the encoder En and the predictor Pd are updated using gradient descent and backpropagation so that the loss function L is minimized. However, in learning in the model structure SM, a gradient stopping operation GS is executed. That is, the above formula (2) can be expressed by the following formula (3). In formula (3), stopgrad represents gradient stopping.

[0034]

number

[0035] That is, in updating the parameters of the encoder En, the feature vector z1 in the similarity D(p2, z1) and z2 in the similarity D(p1, z2) are treated as constants. Note that Fig. 2 shows how the gradient stopping operation GS is performed for the second data Dt2. Specifically, in Fig. 2, the encoder En receives the gradient Gd for the first data Dt1, but does not receive the gradient Gd for the second data Dt2.

[0036] Generally, collapse can be a problem in SSL. Collapse means that the learning model is trained so that a fixed feature vector is output from the encoder regardless of the data input to the encoder. In SimSiam, it is known that collapse is suppressed by arranging a predictor Pd corresponding to one of the first data Dt1 and the second data Dt2 and executing a gradient stopping operation GS corresponding to the other data. Therefore, by using SimSiam to train the learning model 160, the learning model 160 can be trained effectively without using negative samples or a momentum encoder.

[0037] As the learning model 160, a part or all of the encoder En can be used. In this embodiment, the backbone Bb of the encoder En is used as the learning model 160. That is, when data is input to the learning model 160, a feature vector based on the data is output by the learning model 160. The feature vector thus output corresponds to embedded data obtained by projecting the input data into the embedding space.

[0038] The type of data augmentation DA performed during training of the learning model 160 affects the factors that contribute to the uniqueness of the evaluation target data. For example, when grayscaling is used as the data augmentation DA, differences in hue contribute less to the uniqueness of the evaluation target data than when grayscaling is not used. Similarly, when left-right flipping is used as the data augmentation DA, differences in left-right orientation between image data contribute less to the uniqueness of the evaluation target data. Therefore, for example, by setting the type of data augmentation DA according to the desired factors, the uniqueness of the evaluation target data can be more effectively evaluated.

[0039] Fig. 3 is a flowchart of the pre-processing in this embodiment. For example, every time an input data set DS is input to the data evaluation device 100 via the input device 106, the processor 101 starts the pre-processing shown in Fig. 3. The data evaluation method in this embodiment is realized by the pre-processing in Fig. 3 and the evaluation processing described below.

[0040] In steps S105 and S110, the processor 101 executes a learning process for training the learning model 160 through machine learning. First, in step S105, the acquisition unit 110 acquires a learning dataset from the input dataset DS. In step S110, the learning execution unit 140 executes machine learning of the learning model 160 using the learning dataset acquired in step S105.

[0041] In step S115, the acquisition unit 110 acquires a reference dataset. As described above, in this embodiment, the reference dataset is included in the training dataset. In step S120, the generation unit 115 generates reference embedded data 165 using the training model 160 trained in step S110 and the reference dataset acquired in step S115. The generation unit 115 stores the reference embedded data 165 thus generated in the storage unit 102. The process of generating the reference embedded data 165, as in step S120, is also referred to as a pre-generation process. The pre-generation process is executed prior to the generation process, which will be described later.

[0042] 4 is a flowchart of the evaluation process in this embodiment. The evaluation process is started by the processor 101, for example, every time the pre-processing in FIG. 3 is completed. Note that in another embodiment, the processor 101 may start the evaluation process, for example, after the pre-processing is completed, in response to a predetermined input by the user Ur via the input device 106 as a trigger.

[0043] In step S205, the acquisition unit 110 acquires an evaluation target dataset from the input dataset DS.

[0044] In step S210, the generation unit 115 inputs each evaluation target data included in the evaluation target dataset acquired in step S205 into the learning model 160. As a result, evaluation embedding data is generated. Specifically, in step S210 in this embodiment, evaluation embedding data is generated by projecting each evaluation target data into the embedding space. The generation unit 115 stores the generated evaluation embedding data in the storage unit 102. The process of generating evaluation embedding data, such as step S210, is also referred to as a generation process.

[0045] In step S215, the calculation unit 120 calculates the density of the evaluation target data in the embedding space using the evaluation embedding data generated in step S210 and the reference embedding data 165 generated in step S120 of Fig. 3. In step S215, the density is calculated for each evaluation target data. The process of calculating the density of the evaluation target data, such as in step S215, is also referred to as a calculation process.

[0046] In step S220, the evaluation unit 125 evaluates the uniqueness of the evaluation target data based on the density of the evaluation target data calculated in step S215. In step S220, the evaluation unit 125 evaluates the uniqueness of the evaluation target data based on the result of normalizing the density of the evaluation target data, as described above. The process of evaluating the uniqueness of the evaluation target data, such as step S220, is also referred to as the evaluation process.

[0047] In step S225, the evaluation unit 125 uses the output device 105 to output the evaluation result of step S220.

[0048] Fig. 5 is an explanatory diagram showing an example of a probability density function calculated by the calculation unit 120. Fig. 6 is an explanatory diagram showing an example of the evaluation results of each evaluation target data.

[0049] The probability density function in FIG. 5 and the evaluation results in FIG. 6 were obtained using the learning model 160, the reference embedded data 165, and the evaluation target dataset prepared under the following conditions. A dataset consisting of multiple image data was used as the learning dataset for training the learning model 160. Each image data included in the learning dataset was classified into one of multiple predetermined classes. The multiple classes include an "airplane" class. The reference embedded data 165 was obtained by inputting each image data of the airplane class included in the learning dataset as reference data into the trained learning model 160. A dataset including multiple airplane class image data as evaluation target data was used as the evaluation target dataset.

[0050] In Fig. 5, the probability density function, which is actually calculated as a function representing the density of data points in a 64-dimensional embedding space, is schematically represented in a two-dimensional XY space having mutually orthogonal X and Y coordinates. In Fig. 5, areas of the XY space with higher density are indicated by darker hatching.

[0051] FIG. 6 shows a data group Gr1 including only evaluation target data determined to have low uniqueness in the evaluation process, and a data group Gr2 including only evaluation target data determined to have high uniqueness. FIG. 6 shows 10 image data as examples of evaluation target data included in each data group. In FIG. 6, the portion of the pixels included in each image data corresponding to an airplane is hatched. All of the image data included in data group Gr1 was image data depicting the entire body of an airplane flying in the air near the vertical center of each image. On the other hand, the image data included in data group Gr2 varied compared to data group Gr1 in terms of, for example, the position in which the airplane was captured, the size of the airplane, and whether the airplane was in the air or on the ground. In other words, the uniqueness and diversity of each data in data group Gr2 was higher than that of data group Gr1.

[0052] According to the data evaluation method of the present embodiment described above, the density of the evaluation target data is calculated using evaluation embedding data in which the evaluation target data is projected into the embedding space and reference embedding data 165 in which each reference data is projected into the embedding space, and the uniqueness of the evaluation target data is evaluated based on the calculated density of the evaluation target data. Since the density of data in the embedding space correlates favorably with the uniqueness of the data, this allows the uniqueness of the evaluation target data to be appropriately evaluated. Furthermore, in this embodiment, the density of each evaluation target data is calculated individually as a density based on each reference data in the reference embedding space, so the uniqueness of each evaluation target data can be evaluated individually.

[0053] Furthermore, in this embodiment, the learning model 160 has been trained by self-supervised learning. By training the learning model 160 by self-supervised learning in this manner, the learning model 160 can be appropriately trained so as to more accurately evaluate uniqueness without using labeled training data. In this embodiment, the learning model 160 thus appropriately trained can be used to evaluate uniqueness, allowing for more appropriate evaluation of the uniqueness of the data to be evaluated. Furthermore, the effort required to prepare a training dataset for training the learning model 160 can be reduced. Specifically, in this embodiment, for example, compared to when the learning model 160 is trained by supervised learning, there is no need to prepare a training dataset including labeled data, and therefore the effort required to prepare the training dataset can be reduced.

[0054] Furthermore, in this embodiment, the learning model 160 has been trained using at least a portion of the reference data set. In this way, when the reference data is input to the learning model 160, the learning model 160 can more appropriately output a feature vector of the reference data, compared to, for example, when no reference data set is used for training the learning model 160. As a result, the learning model 160 can more appropriately project the reference data into the embedding space and generate more appropriate reference embedded data 165. Therefore, the uniqueness of the data to be evaluated can be more appropriately evaluated.

[0055] In this embodiment, the learning model 160 has been trained using the entire reference data set. Therefore, the learning model 160 can more appropriately project the reference data into the embedding space, and generate more appropriate reference embedded data 165. Therefore, the uniqueness of the evaluation target data can be more appropriately evaluated.

[0056] Furthermore, this embodiment includes a pre-generation step of generating reference embedded data 165 prior to the generation step. Therefore, the reference embedded data 165 is generated in the pre-generation step, and the generated reference embedded data 165 can be used to evaluate the uniqueness of the evaluation target data.

[0057] Furthermore, in this embodiment, in the evaluation step, the uniqueness of the evaluation target data is evaluated based on the result of normalizing the density of the evaluation target data, which makes it possible to more easily evaluate the uniqueness of the evaluation target data.

[0058] B. Other Embodiments: (B-1) In the above embodiment, the learning model 160 has been trained using SimSiam. In contrast, the learning model 160 may be trained using various methods other than SimSiam, as long as it has been trained to project input data into an embedding space. For example, the learning model 160 may be trained using SimCLR, BYOL, or SwAV. Furthermore, the learning model 160 may be trained using various learning methods, such as metric learning, other than SSL. In this case, the learning model 160 may be trained using, for example, unsupervised learning, supervised learning, semi-supervised learning, or reinforcement learning. Specifically, the learning model 160 may be a model trained by supervised learning or the like to perform class classification of input data.

[0059] (B-2) In the above embodiment, the training dataset includes the entire reference dataset. In contrast, the training dataset may include, for example, only a portion of the reference dataset, or may not include the reference dataset at all. The training dataset may be identical to the reference dataset. All or part of the training dataset may be identical to all or part of the reference dataset. All or part of the training dataset may be identical to all or part of the dataset to be evaluated. All or part of the reference dataset may be identical to all or part of the dataset to be evaluated.

[0060] (B-3) In the above embodiment, the data evaluation method includes a pre-generation step, but it does not necessarily have to include a pre-generation step. In this case, for example, the reference embedded data 165 may be prepared in advance by a computer external to the data evaluation device 100. In this case, in step S215 of FIG. 4, the reference embedded data 165 prepared by the external computer may be used to calculate the density of the evaluation target data.

[0061] (B-4) In the above embodiment, the learning process is performed in the data evaluation device 100. However, the learning process does not have to be performed in the data evaluation device 100. In this case, machine learning of the learning model 160 may be performed, for example, in a computer external to the data evaluation device 100.

[0062] (B-5) In the above embodiment, the uniqueness of the evaluation target data is evaluated based on the result of normalizing the density of the evaluation target data in the evaluation step. In contrast, the uniqueness of the evaluation target data may be evaluated without normalizing the density of the evaluation target data.

[0063] (B-6) In the above embodiment, the evaluation unit 125 may evaluate the diversity of the entire evaluation target dataset. For example, the evaluation unit 125 may evaluate the diversity of the evaluation target dataset based on the arithmetic mean value of the densities of multiple evaluation target data included in the evaluation target dataset. In this case, the evaluation unit 125 may evaluate the diversity as low if the arithmetic mean value is equal to or greater than a threshold, and may evaluate the diversity as high if the arithmetic mean value is less than the threshold. The evaluation unit 125 may also evaluate the diversity of the evaluation target dataset based on the uniqueness of multiple evaluation target data included in the evaluation target dataset. In this case, the evaluation unit 125 may evaluate the diversity as high if the number or proportion of evaluation target data determined to be highly unique is equal to or greater than a threshold, and may evaluate the diversity as low if the number or proportion is less than the threshold. The evaluation result of the diversity of the evaluation target dataset may be output using, for example, the output device 105.

[0064] (B-7) In the above embodiment, KDE is used in the calculation step. However, instead of KDE, other non-parametric density estimation methods such as histogram density estimation (HDE) or a parametric density estimation method may be used in the calculation step.

[0065] C. Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following forms. The technical features in the above embodiments corresponding to the technical features in each form described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0066] (1) According to a first aspect of the present disclosure, there is provided a data evaluation method, comprising: a generation step of generating evaluation embedded data in which evaluation target data is projected into an embedding space by inputting the evaluation target data into a learning model that has been machine-learned to project input data into the embedding space; a calculation step of calculating, using the evaluation embedded data and reference embedded data in which a plurality of reference data serving as a reference for the evaluation target data are projected into the embedding space, a density of the evaluation target data in the embedding space into which each of the reference data is projected; and an evaluation step of evaluating the uniqueness of the evaluation target data based on the density. According to this aspect, the uniqueness of the evaluation target data can be appropriately evaluated.

[0067] (2) In the above embodiment, the learning model may have been trained by self-supervised learning. According to this embodiment, the uniqueness of the evaluation target data can be evaluated using a learning model that has been appropriately trained so as to more accurately evaluate uniqueness.

[0068] (3) In the above embodiment, the learning model may have been trained using at least some of the plurality of reference data. According to this embodiment, the learning model can generate more appropriate reference embedding data. Therefore, the uniqueness of the evaluation target data can be more appropriately evaluated.

[0069] (4) In the above embodiment, the learning model may have been trained using all of the plurality of reference data. According to this embodiment, the learning model can generate more appropriate reference embedding data. Therefore, the uniqueness of the evaluation target data can be more appropriately evaluated.

[0070] (5) In the above embodiment, a pre-generation step of generating the reference embedded data by inputting each of the reference data into the learning model prior to the generation step may be provided. According to this embodiment, the reference embedded data is generated in the pre-generation step, and the generated reference embedded data can be used to evaluate the uniqueness of the evaluation target data.

[0071] (6) In the above aspect, the evaluation step may evaluate the uniqueness based on a result of normalizing the density. According to this aspect, the uniqueness of the evaluation target data can be evaluated more easily.

[0072] (7) In the above aspect, in the evaluation step, the uniqueness of the evaluation target data may be evaluated as low if the normalized density is equal to or greater than a predetermined threshold. According to this aspect, it is possible to effectively evaluate that the uniqueness of the evaluation target data is low.

[0073] (8) According to a second aspect of the present disclosure, there is provided a data evaluation device. The data evaluation device includes: a learning model trained through machine learning to project features of input data into an embedding space; reference embedded data obtained by projecting a plurality of reference data serving as a reference for data to be evaluated into the embedding space by the learning model; and a processor. The processor executes the following steps: acquiring the data to be evaluated; inputting the data to be evaluated into the learning model to generate evaluation embedded data obtained by projecting the data to be evaluated into the embedding space; calculating, using the evaluation embedded data and the reference embedded data, the density of the data to be evaluated in the embedding space into which each of the reference data is projected; and evaluating the uniqueness of the data to be evaluated based on the density.

[0074] (9) According to a third aspect of the present disclosure, there is provided a program that causes a computer to perform the following functions: inputting evaluation target data into a learning model that has been machine-learned to project features of the input data into an embedding space, thereby generating evaluation embedded data in which the evaluation target data is projected into the embedding space; using the evaluation embedded data and reference embedded data in which multiple reference data serving as standards for the evaluation target data are projected into the embedding space, calculating a density of the evaluation target data in the embedding space into which each of the reference data is projected; and evaluating the uniqueness of the evaluation target data based on the density.

[0075] In addition to the above-described embodiments, the present disclosure can be realized in the form of, for example, a data processing system, a non-transitory recording medium on which a program is recorded, a program product, etc. Note that the program product may be provided as a recording medium on which a program is recorded, or may be provided as a program product that can be distributed via a network, for example. [Explanation of symbols]

[0076] 50...data processing system, 100...data evaluation device, 101...processor, 102...memory unit, 103...input / output interface, 104...internal bus, 105...output device, 106...input device, 110...acquisition unit, 115...generation unit, 120...calculation unit, 125...evaluation unit, 140...learning execution unit, 155...program, 160...learning model, 165...reference embedded data

Claims

1. a generation step of inputting evaluation target data into a learning model that has been machine-learned to project input data into an embedding space, thereby generating evaluation embedding data in which the evaluation target data is projected into the embedding space; a calculation step of calculating a density of the evaluation target data in the embedding space into which each of the reference data is projected, using the evaluation embedding data and reference embedding data in which a plurality of reference data serving as a reference for the evaluation target data is projected into the embedding space; an evaluation step of evaluating the uniqueness of the evaluation target data based on the density.

2. 2. The data evaluation method according to claim 1, A data evaluation method, wherein the learning model has been trained by self-supervised learning.

3. 2. The data evaluation method according to claim 1, A data evaluation method, wherein the learning model has been trained using at least a portion of the plurality of reference data.

4. 2. The data evaluation method according to claim 1, A data evaluation method, wherein the learning model has been trained using all of the plurality of reference data.

5. 2. The data evaluation method according to claim 1, A data evaluation method comprising a pre-generation step of generating the reference embedded data by inputting each of the reference data into the learning model prior to the generation step.

6. 6. A data evaluation method according to claim 1, comprising: A data evaluation method, wherein in the evaluation step, the uniqueness is evaluated based on a result of normalizing the density.

7. 7. The data evaluation method according to claim 6, A data evaluation method, wherein in the evaluation step, if the normalized density is equal to or greater than a predetermined threshold, the uniqueness of the evaluation target data is evaluated as low.

8. a memory unit that stores a learning model that has been machine-learned to project the features of input data into an embedding space, and reference embedding data that is obtained by projecting a plurality of reference data that serve as a reference for the evaluation target data into the embedding space by the learning model; a processor; The processor: A process of acquiring the evaluation target data; A process of generating evaluation embedding data in which the evaluation target data is projected into the embedding space by inputting the evaluation target data into the learning model; a process of calculating a density of the evaluation target data in the embedding space into which each of the reference data is projected, using the evaluation embedding data and the reference embedding data; and evaluating the uniqueness of the evaluation target data based on the density. Data evaluation device.

9. A function of generating evaluation embedding data in which the evaluation target data is projected into the embedding space by inputting the evaluation target data into a learning model that has been machine-learned to project the features of the input data into the embedding space; a function of calculating the density of the evaluation target data in the embedding space into which each of the reference data is projected, using the evaluation embedding data and reference embedding data in which a plurality of reference data serving as a reference for the evaluation target data is projected into the embedding space; and a program that causes a computer to realize a function of evaluating the uniqueness of the evaluation target data based on the density.

Images