Missing value estimation method, machine learning method, and missing value estimation device

Abstract

To provide a technique allowing for more simply estimating missing values in data.SOLUTION: A missing value estimation method includes the steps of: obtaining input data in which part of a plurality of category feature quantities for expressing features of an object is missing; obtaining, from a storage unit that stores the plurality of feature quantity vectors corresponding to the plurality of category feature quantities and having equal dimensions, a plurality feature quantity vectors of a plurality of category feature quantities in which there is no missing included in the input data; and estimating a missing value in the input data from the plurality of obtained feature quantity vectors using a learning model machine-learned based on learning data including masked category feature quantities.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to techniques for estimating missing values ​​in data. [Background technology]

[0002] Patent Document 1 discloses a missing value prediction device that takes as input a taxonomy, which is a class hierarchical structure that serves as background knowledge for each dimension, adjusts the granularity of the classes for each dimension based on the taxonomy for each dimension, and estimates missing values ​​from the results of the granularity adjustment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2014-096118 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology of Patent Document 1 requires that the taxonomy be designed and input manually, which requires a great deal of effort and time.

[0005] An object of the present invention is to provide a technique for more simply estimating missing values ​​in data. [Means for solving the problem]

[0006] In order to solve the above problem, a missing value estimation method according to an embodiment of the present invention includes: 1. A computer-implemented method for missing value estimation, comprising:The method includes a first acquisition step of acquiring input data missing some of a plurality of categorical features for expressing features of a target; a second acquisition step of acquiring a plurality of feature vectors of a plurality of non-missing categorical features included in the input data from a storage unit that stores a plurality of feature vectors corresponding to the plurality of categorical features and having equal dimensions; and an estimation step of estimating missing values ​​of the input data from the acquired plurality of feature vectors using a learning model machine-learned based on learning data including the masked categorical features.

[0007] Another aspect of the present invention is a machine learning method, the method comprising: 1. A computer implemented machine learning method, comprising: The method includes a first acquisition step of acquiring input data including a plurality of category features for expressing features of a target; a masking step of masking some of the plurality of category features included in the input data; a second acquisition step of acquiring a plurality of feature vectors of unmasked category features included in the masked input data from a storage unit that stores a plurality of feature vectors corresponding to the plurality of category features and having equal dimensions; an estimation step of estimating a feature vector of a masked category feature from the plurality of feature vectors of the unmasked category features using a learning model; and an update step of updating the feature vectors of the plurality of category features included in the input data stored in the storage unit and parameters of the learning model based on the feature vector estimated in the estimation step.

[0008] Yet another aspect of the present invention is a missing value estimation device, which includes a first acquisition unit that acquires input data missing some of a plurality of categorical features for expressing a feature of a target, a second acquisition unit that acquires a plurality of feature vectors of a plurality of categorical features that are not missing and included in the input data from a storage unit that stores a plurality of feature vectors corresponding to the plurality of categorical features and having the same dimension, and an estimation unit that estimates missing values ​​of the input data from the plurality of acquired feature vectors by using a learning model machine-learned based on learning data including masked categorical features. Effect of the Invention

[0009] According to the present invention, a technique is provided that enables missing values ​​of data to be estimated more simply. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates a configuration of a missing value estimation device according to an embodiment. [Diagram 2] FIG. 10 is a diagram for explaining a vector representation of a category feature according to an embodiment. [Diagram 3] FIG. 2 is a diagram for explaining a learning process in the missing value estimation device of FIG. [Figure 4] 4 is a diagram illustrating an example of a relationship between a feature vector estimated in the learning process of FIG. 3 and some of the feature vectors stored in a feature vector storage unit. FIG. [Diagram 5] 2 is a flowchart showing a machine learning process of the missing value estimation device in FIG. 1 . [Figure 6] 2 is a flowchart showing missing value estimation and imputation processing of the missing value estimation device of FIG. 1. [Figure 7] FIG. 2 is a diagram for explaining vector representation of continuous values ​​in the missing value estimation device of FIG. 1. [Figure 8] 1 is a diagram illustrating a configuration of a product recommendation system according to an embodiment. [Figure 9] 9 is a flowchart showing the processing of the product recommendation system of FIG. 8. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Before describing the embodiment in detail, the knowledge on which the present disclosure is based will be described. It is desirable to use a machine learning model to analyze customer data (hereinafter also referred to as user data) input by a user who is a customer to a company's website, and to estimate the user's favorite products based on the data. The customer data includes multiple category features such as gender and hobbies. However, the customer data often includes missing values ​​because the user has not input some category features or because of a communication error. Since the machine learning model cannot handle missing values, it is necessary to complement the missing values ​​with appropriate values ​​as a preprocessing step. If the complemented value is not valid, it will have a negative effect on the machine learning, so it is desirable to complement the missing values ​​more accurately.

[0012] In addition, a technique for complementing missing values ​​of categorical features using a deep learning model is known. When categorical features are input into a deep learning model, the categorical features are expressed as one-hot vectors, but one-hot vectors cannot express the semantic information of categorical features. For example, categorical features related to hobbies include "reading," "soccer," and "golf." Although "soccer" and "tennis" are considered to be similar, the distances between the one-hot vectors representing "reading," "soccer," and "golf" are equal, so the similarity cannot be expressed. Therefore, when one-hot vectors are used, it is difficult to complement missing values ​​by considering the similarity of categorical features.

[0013] Therefore, in the embodiment, the categorical features are expressed as vectors so as to express the semantic information of the categorical features, and the optimal vector expression is automatically acquired by machine learning.

[0014] 1 shows a configuration of a missing value estimation device 1 according to an embodiment. The missing value estimation device 1 uses, for example, a known BERT (Bidirectional Encoder Representations from Transformers) to machine-learn vector representations of categorical features of user data and parameters of a missing value estimation model, and estimates and complements missing values ​​of the user data using the learned vector representations and the missing value estimation model.

[0015] The missing value estimation device 1 includes a processing unit 10, a missing user data storage unit 30, a feature vector storage unit 32, and a completed user data storage unit 34. The processing unit 10 includes a first acquisition unit 12, a mask unit 14, a second acquisition unit 16, an estimation unit 18, an update unit 22, and a completion unit 24. The estimation unit 18 includes a missing value estimation model 20.

[0016] The configuration of the processing unit 10 can be realized in hardware by the CPU, memory, and other LSI of any computer, and in software by a program loaded into memory, but here, functional blocks realized by the cooperation of these are depicted. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by only hardware, only software, or a combination of both.

[0017] The missing user data storage unit 30 stores missing user data of a plurality of users. The missing user data includes missing records of each of the plurality of users. The missing records are, for example, customer data input by a user, and are missing some of a plurality of categorical features for expressing the characteristics of the user. The missing categorical features are called missing values. The categorical features include, for example, features related to categories such as gender, age group, marital status, and hobbies, and are also called categorical features or categorical variables.

[0018] The missing user data is used as a learning dataset for machine learning in the processing unit 10. When the machine learning is completed, the processing unit 10 estimates and complements missing values ​​of each missing record of the missing user data.

[0019] The missing user data storage unit 30 may store non-missing user data of multiple users. The non-missing user data can also be used for machine learning to increase the amount of learning data.

[0020] The feature vector storage unit 32 stores a plurality of feature vectors corresponding to a plurality of category features. There is a one-to-one correspondence between the category features and the feature vectors. The feature vector is a vector representation of the corresponding category feature. The plurality of feature vectors are optimized by machine learning. The feature vector storage unit 32 also stores parameters of the missing value estimation model 20. The parameters are also optimized by machine learning.

[0021] 2 is a diagram for explaining vector representation of category features according to an embodiment. For example, a case will be described where a user record is [male, 30s, single, ...]. "Male" is a category feature of the category "gender", "30s" is a category feature of the category "age group", and "single" is a category feature of the category "marital status". A user record can also include multiple category features other than these.

[0022] e(male), e(30s), e(single), etc. are each a feature vector and a d-dimensional row vector. For example, e(male) represents the feature vector of the category feature "male". Unlike a one-hot vector, a feature vector may have multiple components with values ​​other than 0, and each component may have a value other than 0 and 1. The dimensions of each of the multiple feature vectors are equal. Therefore, any two feature vectors can represent the similarity between two corresponding category features. d may be, for example, 100 or more, and can be appropriately determined by simulation. d is set in advance by a user of the missing value estimation device 1 before machine learning.

[0023] Let the sequence of vectors xi corresponding to this user's record be [e(male) T ;e(30s) T ;e(single) T The vector sequence xi is represented as a matrix with d rows and E columns, i.e., [x,y,y,z ...

[0024] 3 in addition to Fig. 1, a learning process in the missing value estimation device 1 will be described. The learning process is executed by masking some of the multiple category features of the learning data, and can be called masked learning.

[0025] FIG. 3 is a diagram for explaining the learning process in the missing value estimation device 1 of FIG. 1. The first acquisition unit 12 acquires a missing record of a certain user from the missing user data storage unit 30 as input data, and supplies the acquired missing record to the mask unit 14 and the update unit 22. An example of the missing record is [NA, 50s, single, reading, ...]. "NA" represents a missing value, and in this case, the categorical feature related to gender is missing. As described above, when the missing user data storage unit 30 also includes user data that is not missing, the first acquisition unit 12 may acquire a record that does not include a missing value as input data.

[0026] The mask unit 14 masks some of the non-missing category features included in the missing record acquired by the first acquisition unit 12, and supplies the masked record to the second acquisition unit 16. In the example of Fig. 3, the mask unit 14 masks the hobby-related category feature "reading" in the missing record, and the masked feature in the masked record is represented as "MASK". The mask unit 14 may mask the non-missing category features.

[0027] The second acquiring unit 16 acquires a plurality of feature vectors corresponding to a plurality of unmasked and non-missing category features included in the masked records from the feature vector storage unit 32. These feature vectors have been updated in the previous learning.

[0028] The second acquisition unit 16 generates a sequence of vectors corresponding to the masked records based on the acquired feature vectors. At this time, the second acquisition unit 16 performs the operations shown in FIG. 3 as “e(mask)” for each of the masked categorical features and the missing values. T " to generate a sequence of vectors. That is, in the sequence of vectors, the column vectors corresponding to the masked category features and the column vectors corresponding to the missing values ​​are the same. The sequence of vectors does not include the feature vector of the masked category feature "reading". The second acquisition unit 16 supplies the generated sequence of vectors to the estimation unit 18. Note that since the order of the multiple category features of the missing record is meaningless, it is not necessary to supply the estimation unit 18 with a vector for identifying the order of the multiple category features.

[0029] The second acquisition unit 16 also acquires parameters for the missing value estimation model 20 from the feature vector storage unit 32, and supplies them to the estimation unit 18. These parameters were updated in the previous learning.

[0030] The missing value estimation model 20 is a deep learning model. An example of a deep learning model is the Transformer.

[0031] The estimation unit 18 estimates a feature vector h of a masked category feature from the sequence of vectors supplied by using the supplied parameters and the missing value estimation model 20. This process corresponds to the estimation unit 18 estimating a feature vector h of a masked category feature from a plurality of feature vectors of a plurality of unmasked category features by using the supplied parameters and the missing value estimation model 20. It can also be said that the estimation unit 18 estimates a feature vector h of a masked category feature from a plurality of feature vectors of a plurality of category features other than the masked category feature. When a plurality of category features of a missing record are masked in the mask unit 14, the estimation unit 18 estimates a feature vector h of a plurality of masked category features.

[0032] In the example of FIG. 3, the estimation unit 18 estimates a masked feature vector h of “reading” from the feature vector of “NA”, the feature vector of “50s”, the feature vector of “single”, etc., included in the vector sequence.

[0033] Fig. 4 shows an example of a relationship between the feature vector h estimated in the learning process of Fig. 3 and some of the multiple feature vectors stored in the feature vector storage unit 32. As described above, the dimension of the feature vector is, for example, 100 or more dimensions, but in Fig. 4, it is simplified to two dimensions to make it easier to understand the similarity of the multiple feature vectors. Fig. 4 shows a situation after multiple learning rounds, and the multiple feature vectors other than the estimated feature vector h have been updated in the previous learning rounds.

[0034] The estimated feature vector h is closer to the current feature vector of the hobby “reading” compared to the current feature vectors of the hobbies “soccer” and “golf”.

[0035] The update unit 22 acquires a plurality of current feature vectors of the plurality of non-missing category features of the missing record and one or more current feature vectors of one or more category features of the same category as the masked category feature from the feature vector storage unit 32. The update unit 22 updates the plurality of current feature vectors stored in the feature vector storage unit 32 and parameters of the missing value estimation model 20 based on the estimated feature vector h and the plurality of current feature vectors of the plurality of non-missing category features of the missing record and one or more current feature vectors of the one or more category features of the same category as the masked category feature.

[0036] Specifically, the update unit 22 compares the estimated feature vector h with each of the multiple current feature vectors of the multiple non-missing category features of the missing record and the one or more current feature vectors of the one or more category features of the same category as the masked category feature, and updates the multiple current feature vectors based on the comparison result. In the example of FIG. 4, the update unit 22 updates the feature vector of the masked category feature "reading" so as to move it closer to the feature vector h, and to move each feature vector of "soccer" and "golf" of the same category "hobby" as the masked category feature "reading" away from the feature vector h. For example, the update unit 22 matches the feature vector of the masked category feature "reading" with the estimated feature vector h. The update unit 22 also updates the feature vectors of "50s" and "single" included in the missing record by moving them based on the estimated feature vector h. A known BERT process can be used to update the feature vectors and the parameters of the missing value estimation model 20.

[0037] When each feature vector and parameter are updated, the mask unit 14 changes the category feature to be masked in the acquired missing record, and masks another part of the non-missing category feature. For example, the mask unit 14 unmasks "reading" and masks "single". Then, the estimation unit 18 estimates the feature vector h of the masked category feature, and the update unit 22 updates each feature vector and parameter based on the feature vector h. The processing unit 10 repeats these processes until the end condition of the learning by the current missing record is satisfied. The end condition of the learning by the current missing record may be, for example, that each of the non-missing category features of the current missing record has been masked at least once. When the end condition of the learning by the current missing record is satisfied, the first acquisition unit 12 acquires another missing record from the missing user data storage unit 30, and repeats the above-mentioned process using the acquired missing record as learning data.

[0038] By multiple learning rounds, the vector representation of multiple categorical features can be optimized according to the contents of the multiple missing records used in learning, and the optimized vector representation can be automatically acquired. If learning proceeds properly, for example, when "reading" is masked, the difference between the estimated feature vector h and the current feature vector of "reading" becomes small. Since the dimensions of multiple feature vectors are the same, the similarity of categorical features can be expressed.

[0039] When the machine learning is completed, the missing value estimation device 1 executes a missing value estimation process. The first acquisition unit 12 acquires missing records of a user to be subjected to missing value estimation from the missing user data storage unit 30 as input data, and supplies the acquired missing records to the second acquisition unit 16 and the complementation unit 24. The missing records may be records used in learning or may not be records used in learning.

[0040] The second acquisition unit 16 acquires from the feature vector storage unit 32 a plurality of feature vectors of a plurality of non-missing category features included in the missing record supplied from the first acquisition unit 12, and generates a sequence of vectors corresponding to the missing record based on the acquired plurality of feature vectors. At this time, the second acquisition unit 16 generates a sequence of vectors corresponding to the missing record based on the feature vectors acquired. T The second acquisition unit 16 embeds a predetermined vector representing "a vector sequence" into the second acquisition unit 16 to generate a sequence of vectors. The second acquisition unit 16 supplies the generated sequence of vectors to the estimation unit 18.

[0041] The estimation unit 18 estimates missing values ​​from the supplied vector sequence using a machine-learned missing value estimation model 20. This process corresponds to the estimation unit 18 estimating missing values ​​of the input data from the acquired multiple feature vectors using the missing value estimation model 20 that has been machine-learned based on learning data including masked categorical features. When the missing record includes multiple missing values, the estimation unit 18 estimates the multiple missing values. The estimation unit 18 supplies the categorical features that are the estimated missing values ​​to the completion unit 24.

[0042] The complementing unit 24 complements the missing values ​​of the missing records with the category features supplied from the estimation unit 18, and generates a complemented record. The complementing unit 24 stores the complemented record in the complemented user data storage unit 34. The processing unit 10 repeats these processes to complement the missing values ​​of each of the multiple missing records in the missing user data storage unit 30, and stores the multiple complemented records in the complemented user data storage unit 34.

[0043] The similarity of categorical features can be expressed by multiple feature vectors with the same dimensions optimized by machine learning, and missing values ​​are estimated from multiple feature vectors using a machine learning learning model, which reduces manual work and enables more accurate automatic estimation of missing values. This makes it easier to estimate missing values ​​in data.

[0044] Fig. 5 is a flowchart showing the machine learning process of the missing value estimation device 1 of Fig. 1. An example is shown in which a missing record is used as learning data. The first acquisition unit 12 acquires one missing record from the missing user data storage unit 30 (S10), and the mask unit 14 masks a part of the multiple non-missing category features of the missing record (S12). At this time, if there is an existing mask, the mask unit 14 changes the category feature to be masked.

[0045] The second acquisition unit 16 acquires a plurality of feature vectors of a plurality of unmasked category features of the missing record from the feature vector storage unit 32 (S14), and the estimation unit 18 estimates a feature vector h of the masked category feature from the acquired plurality of feature vectors using the missing value estimation model 20 (S16).

[0046] The update unit 22 compares the estimated feature vector h with each of a plurality of feature vectors determined according to the missing record and the masked category features (S18), and updates the plurality of feature vectors and the parameters of the missing value estimation model 20 in the feature vector storage unit 32 based on the comparison result (S20). If learning with the current missing record is not to be ended (N in S22), the process returns to S12. If learning with the current missing record is to be ended (Y in S22) or learning is not to be ended (N in S24), the process returns to S10. If learning is to be ended (Y in S24), the process ends.

[0047] Fig. 6 is a flowchart showing the missing value estimation and completion process of the missing value estimation device 1 of Fig. 1. The process of Fig. 6 may be executed repeatedly. The first acquisition unit 12 acquires a missing record of a target user (S30), and the second acquisition unit 16 acquires a plurality of feature vectors of a plurality of non-missing category features of the missing record from the feature vector storage unit 32 (S32).

[0048] The estimation unit 18 uses the machine-learned missing value estimation model 20 to estimate the missing values ​​from the multiple acquired feature vectors (S34), and the completion unit 24 completes the missing values ​​of the missing records and stores the completed records in the completed user data storage unit 34 (S36), thereby completing the processing.

[0049] Next, a case where the record further includes continuous values ​​for expressing the user's characteristics will be described, focusing on the differences from the above-mentioned processing.

[0050] FIG. 7 is a diagram for explaining vector representation of continuous values ​​in the missing value estimation device 1 of FIG. 1. For example, a case will be described where a user record is [male, 30s, single, 68 kg, ...]. "68 kg" represents weight and is a continuous value rather than a categorical feature. In other words, it is assumed that the user can input any numeric value for weight. Other continuous values ​​include, for example, age and height.

[0051] The feature vector corresponding to the weight is a d-dimensional row vector, and its components include cos(π0.68), cos(2π0.68), cos(3π0.68), etc. In other words, the multiple components of the feature vector of continuous values ​​are multiple values ​​obtained by converting the continuous values ​​based on multiple periodic functions with different periods. The periodic functions and periods are determined in advance. Each period of the periodic functions can be appropriately determined by simulation.

[0052] The sequence of vectors xi corresponding to this user's record is also represented as a matrix with d rows and E columns, and includes a column vector obtained by transposing the row vector corresponding to the weight.

[0053] First, learning will be described. When the input data, which is a missing record or a record not including a missing value, includes a categorical feature and a continuous value, the second acquisition unit 16 converts the continuous value by the above-mentioned multiple periodic functions to derive multiple components, thereby acquiring the above-mentioned feature vector in which the continuous value is expressed as a vector, and generating a vector sequence. The feature vector of the continuous value does not need to be stored in the feature vector storage unit 32.

[0054] The masking unit 14 may mask the continuous values. If the continuous values ​​are masked, the estimating unit 18 estimates a masked continuous feature vector h from the sequence of vectors provided using the provided parameters and the missing value estimation model 20.

[0055] The update unit 22 updates the multiple current feature vectors of the multiple categorical features stored in the feature vector storage unit 32 and the parameters of the missing value estimation model 20 based on the estimated feature vector h, the multiple non-missing categorical features of the missing record, the multiple current feature vectors of the non-missing continuous values, and the one or more current feature vectors of the one or more categorical features of the same category as the masked categorical feature. On the other hand, the update unit 22 does not update the feature vector expressing a continuous value, regardless of whether the target of the mask is a continuous value or a categorical feature. In other words, the feature vector corresponding to the continuous value is a fixed vector specific to the continuous value defined by the above-mentioned multiple periodic functions, and is not updated by learning. The feature vector of the categorical feature associated with the continuous value is updated to express the association with the feature vector of the continuous value.

[0056] After the learning is completed, when estimating missing values, if the input data, that is, the missing records, includes categorical features and continuous values, the second acquisition unit 16 also acquires the above-mentioned feature vector in which the continuous values ​​are expressed as vectors, and generates a sequence of vectors.

[0057] The estimation unit 18 estimates missing values ​​from the supplied vector sequence using a machine-learned missing value estimation model 20. This allows the estimation of missing values ​​that are continuous values ​​when continuous values ​​are missing. Also, when categorical features are missing, the estimation of missing values ​​can be made while also reflecting the degree of association between the missing values ​​and continuous values.

[0058] Next, an application example of the missing value estimation device 1 will be described. 8 shows a configuration of a product recommendation system 100 according to an embodiment. The product recommendation system 100 uses a machine-learned product recommendation model to estimate products that a user is likely to be interested in based on user data that the user inputs to a website, and presents the estimated recommended products to the user.

[0059] 1 and a product recommendation device 40. It is assumed that the above-described machine learning has been completed in the missing value estimation device 1. The product recommendation device 40 includes a receiving unit 42, a determining unit 44, and a presenting unit 48.

[0060] The configuration of the product recommendation device 40 can be realized in hardware by the CPU, memory, and other LSI of any computer, and in software by programs loaded into memory, but here we depict functional blocks realized by the cooperation of these. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by hardware alone, software alone, or a combination of both.

[0061] The receiving unit 42 receives user data input by a user to a website via a browser. The user data includes, for example, a plurality of category features of the records in FIG. 2. When at least one category feature of the user data is missing, the receiving unit 42 supplies the user data, which is a missing record, to the missing value estimation device 1. Based on the supplied user data, the missing value estimation device 1 estimates and complements the missing values ​​as described above, and returns the complemented user data to the product recommendation device 40.

[0062] On the other hand, if the category feature of the received user data is not missing, the receiving unit 42 supplies the user data to the determining unit 44 .

[0063] The determination unit 44 includes a machine-learned product recommendation model 46. The determination unit 44 uses the trained product recommendation model 46 to determine products to be recommended to the user from the complemented user data supplied from the missing value estimation device 1 or the user data supplied from the reception unit 42, and supplies information on the determined recommended products to the presentation unit 48.

[0064] The product recommendation model 46 is machine-trained in advance using data for each of a plurality of users, which is a combination of user data of the user and products purchased or interested in the user, as learning data. The product recommendation model 46 is configured to estimate products that are likely to interest the user when the user data of the user is given. The product recommendation model 46 can be configured using known technology.

[0065] The product recommendation model 46, which is a machine learning model, cannot handle user data with missing categorical features. However, when the user data entered by the user includes missing values, the missing values ​​of the user data are complemented by the missing value estimation device 1, and the user information is represented more accurately, so that recommended products can be determined more accurately.

[0066] The presentation unit 48 presents the supplied information on the recommended product to the user, for example, via a browser, using at least one of images, videos, characters, and audio.

[0067] Fig. 9 is a flowchart showing the processing of the product recommendation system 100 of Fig. 8. The reception unit 42 receives user data from a user (S50), and if the user data is free of missing data (N in S52), the process proceeds to S56. If the user data is free of missing data (Y in S52), the missing value estimation device 1 estimates and complements the missing data (S54). The determination unit 44 uses the product recommendation model 46 to determine a product to be recommended to the user from the user data free of missing data (S56). The presentation unit 48 presents information on the determined recommended product to the user (S58).

[0068] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each processing step, and that such modifications are also within the scope of the present invention.

[0069] For example, the missing value estimation device 1 may be configured as a machine learning device that performs only the above-mentioned machine learning and does not perform missing value estimation and completion. In this case, the missing value estimation device 1 may not include the completion unit 24 and the completed user data storage unit 34. Furthermore, the missing value estimation device 1 may be configured to perform only missing value estimation and completion using the trained feature vector and the missing value estimation model 20 and not perform machine learning. In this case, the missing value estimation device 1 may not include the mask unit 14 and the update unit 22.

[0070] In addition, the input data may be any data including a plurality of category features for expressing the characteristics of a target, and is not limited to user data. For example, the input data may be data including a plurality of category features for expressing the characteristics of an object such as a vehicle or a product. [Explanation of symbols]

[0071] 1...missing value estimation device, 10...processing unit, 12...first acquisition unit, 14...masking unit, 16...second acquisition unit, 18...estimation unit, 20...missing value estimation model, 22...updating unit, 24...completion unit, 30...missing user data storage unit, 32...feature vector storage unit, 34...complemented user data storage unit.

Claims

1. A computer-implemented method for missing value estimation, comprising: A first acquisition step of acquiring input data in which some of a plurality of categorical features for expressing features of a target are missing; a second acquisition step of acquiring, from a storage unit storing a plurality of feature vectors corresponding to a plurality of category features and having the same dimension, a plurality of feature vectors of a plurality of category features that are not missing and are included in the input data; an estimation step of estimating missing values ​​of the input data from the acquired plurality of feature vectors using a learning model machine-learned based on learning data including masked categorical features; A missing value estimation method comprising:

2. When the input data further includes continuous values ​​for expressing features of the target, in the second acquisition step, a feature vector is acquired which expresses the continuous values ​​as a vector, and a plurality of components of the feature vector of the continuous values ​​are a plurality of values ​​obtained by transforming the continuous values ​​based on a plurality of periodic functions each having a different period. The missing value estimation method according to claim 1 .

3. A computer-implemented machine learning method, comprising: A first acquisition step of acquiring input data including a plurality of categorical features for expressing features of a target; a masking step of masking a part of a plurality of categorical features included in the input data; a second acquisition step of acquiring, from a storage unit storing a plurality of feature vectors corresponding to a plurality of category features and having the same dimension, a plurality of feature vectors of unmasked category features included in the masked input data; an estimation step of estimating a feature vector of a masked categorical feature from a plurality of feature vectors of a plurality of unmasked categorical features using a learning model; an updating step of updating a plurality of feature vectors of a plurality of category features included in the input data stored in the storage unit and parameters of the learning model based on the feature vector estimated in the estimating step; A machine learning method comprising:

4. When the input data further includes continuous values ​​for expressing features of the target, in the second acquisition step, a feature vector is acquired which expresses the continuous values ​​as a vector, and a plurality of components of the feature vector of the continuous values ​​are a plurality of values ​​obtained by transforming the continuous values ​​based on a plurality of periodic functions each having a different period. The machine learning method according to claim 3 .

5. If the input data includes a continuous value, the updating step does not update the feature vector of the continuous value. The machine learning method according to claim 4 .

6. a first acquisition unit that acquires input data in which some of a plurality of category features for expressing a feature of a target are missing; a second acquisition unit that acquires a plurality of feature vectors of a plurality of non-missing category features included in the input data from a storage unit that stores a plurality of feature vectors corresponding to a plurality of category features and having the same dimension; an estimation unit that estimates missing values ​​of the input data from a plurality of acquired feature vectors using a learning model that is machine-learned based on learning data including masked categorical features; A missing value estimation device comprising:

Images