Information processing device, control method, and program

Abstract

This information processing device comprises: a similarity acquisition unit that acquires a similarity between an explanatory sentence explaining a feature quantity in machine learning training data and a definition sentence indicating a definition of correct answer data in the training data; and a determination unit that determines, on the basis of the acquired similarity, whether or not to delete the feature quantity from the training data.

Description

Information Processing Apparatus, Control Method, and Program

[0001] The present disclosure relates to an information processing apparatus, a control method, and a program.

[0002] When training a machine learning model using a large number of feature quantities or performing inference using a machine learning model, there may be a leak in which a feature quantity having a direct relationship with an objective variable (correct data) is set as training data.

[0003] Japanese Patent Application Laid-Open No. 2021-536050

[0004] When the data to be treated as training data is complex, it is difficult to distinguish whether the feature quantity included in the training data is a feature quantity that causes a leak or an effective feature quantity that happens to have a correlation.

[0005] One aspect of the present disclosure provides an information processing apparatus, a control method, and a program capable of determining a feature quantity that causes a leak.

[0006] An information processing apparatus according to one aspect of the present disclosure includes a similarity acquisition unit that acquires a similarity between an explanatory text that explains a feature quantity in training data of machine learning and a definition text that indicates a definition of correct data in the training data, and a determination unit that determines whether to delete the feature quantity from the training data based on the acquired similarity.

[0007] A diagram showing a specific example of a leak. A diagram showing an example of an information processing apparatus. A diagram showing an example of an explanatory text or a definition text. A flowchart showing a processing flow of an information processing apparatus. A diagram showing a modified example of an information processing apparatus. A diagram showing an example of a hardware configuration of an information processing apparatus.

[0008] Hereinafter, embodiments according to one aspect of the present disclosure will be described with reference to the drawings. Note that the embodiments described below are examples, and the embodiments to which the present disclosure is applied are not limited to the following embodiments.

[0009] Figure 1 shows a specific example of a leak. Figure 1 shows an example task in which a machine learning model is trained using store payment history and location information obtained from mobile devices and base stations, such as the number of facility visits and application usage counts, as well as the ground truth data described later, as training data, and infers from service usage logs. This task is a binary classification task to infer whether a user is likely to buy product Y at store A or not. In this case, the ground truth data is "users who bought product Y at store A, which is the target of the inference," as shown in the positive example. Incidentally, the negative example is users who did not buy product Y at store A.

[0010] In the example in Figure 1, the feature data used includes "User ID," "Number of Payments at Store A," "Number of Visits to B," and "Number of Uses of C." "User ID" is an ID used to uniquely identify the user. "Number of Payments at Store A" indicates the number of times the user made a payment at Store A. "Number of Visits to B" indicates the number of times the user visited Facility B. "Number of Uses of C" indicates the number of times the user used application C on their mobile device.

[0011] There is a very strong correlation between the "number of transactions at Store A" feature and the ground truth condition, "users who bought product Y at Store A." Since the "number of transactions at Store A" for "users who bought product Y at Store A" is always 1 or greater, this correlation is strong in the inference. It can be said that a user who made a payment at Store A is more likely to be a user who bought product Y than a user with a "number of transactions at Store A" of 0.

[0012] In this case, the machine learning model only needs to determine that users with one or more "number of transactions at store A" are "users likely to buy product Y at store A," thus achieving high accuracy. Because the machine learning model can determine the correct answer using only one feature, it can be determined that a leak (in this case, the correct answer information is included in the feature) has occurred. Since a leak has occurred, there is a risk that proper machine learning is not being performed.

[0013] To perform proper machine learning, it is necessary to remove such features (e.g., "Number of transactions at Store A" in the example in Figure 1) beforehand. However, when there are a large number of features, it is difficult to determine which features are causing leaks.

[0014] Therefore, the information processing device according to this embodiment determines whether or not to delete features from the training data based on the similarity between the descriptive text that explains the features in the machine learning training data and the definition text that shows the definition of the correct answer data in the training data.

[0015] Figure 2 shows an example of an information processing device 100 according to an embodiment of the present disclosure. The information processing device 100 receives the definition statement and explanatory statement described above as input. The information processing device 100 outputs feature quantities from which feature quantities that have been determined to cause leaks have been removed. The explanatory statement and definition statement input to the information processing device 100 are sentences created in natural language in advance by the user.

[0016] In Figure 2, the information processing device 100 includes a vector acquisition unit 101, a similarity acquisition unit 102, a determination unit 103, and a removal unit 104.

[0017] The vector acquisition unit 101 acquires the input definition and description by converting them into semantic vectors and outputs them to the similarity acquisition unit 102. The similarity acquisition unit 102 calculates the cosine similarity between the input semantic vectors, acquires it as similarity, and outputs the acquired similarity to the determination unit 103. This similarity is acquired for each description, i.e., for each type of feature in the training data (in the example in Figure 1, this would be the number of transactions at store A, the number of visits to store B, etc.). Also, since the similarity is cosine similarity, a higher similarity indicates greater similarity.

[0018] When the similarity score is input from the similarity acquisition unit 102, the determination unit 103 determines whether or not to delete features from the training data based on the similarity score, and outputs the determination result indicating whether or not to delete to the removal unit 104. The removal unit 104 outputs the removed features, which are the features that have been determined to be deleted.

[0019] Figure 3 shows an example of explanatory and defining statements input to the information processing device 100. Figure 3 shows "Number of transactions at store A," "Number of visits to B," and "Number of uses of C" as features. The explanatory statements shown in Figure 3 are pre-written in natural language by the user, corresponding to the features. The explanatory statement corresponding to "Number of transactions at store A" is "Number of times a transaction was made at store A using payment method a." The explanatory statement corresponding to "Number of visits to B" is "Number of visits to facility B." The explanatory statement corresponding to "Number of uses of C" is "Number of times application C was used."

[0020] The vector acquisition unit 101 converts each of these descriptive sentences and definition sentences into semantic vectors and outputs them to the similarity acquisition unit 102. The similarity acquisition unit 102 obtains the similarity between the semantic vectors converted from the descriptive sentences and the semantic vectors converted from the definition sentences. In the case of Figure 3, the similarity acquisition unit 102 obtains the similarity between "the number of times payment was made using payment method a at store A" and "the number of users who bought product Y at store A, which is the target of inference." The similarity acquisition unit 102 obtains the similarity between "the number of times application C was used" and "the number of users who bought product Y at store A, which is the target of inference."

[0021] In this way, the similarity acquisition unit 102 acquires similarity scores corresponding to "number of transactions at store A," "number of visits to store B," and "number of uses of store C," and outputs these similarity scores to the determination unit 103. The determination unit 103 then determines whether or not to remove features from the training data based on the similarity scores.

[0022] Figure 4 is a flowchart showing the processing flow of the information processing device 100. In Figure 4, it is assumed that there are N types of feature quantities, each assigned a number from 1 to N.

[0023] The similarity acquisition unit 102 acquires similarity scores corresponding to N features (step S101) and outputs the N similarity scores to the determination unit 103. The determination unit 103 initializes the loop counter k to 1 (step S102). The determination unit 103 determines whether the similarity score corresponding to the k-th feature is greater than or equal to a threshold (step S103). If the cosine similarity described above is used as this threshold, for example it could be 0.8, but it may be determined appropriately depending on the operation and experimental results.

[0024] If the similarity corresponding to the k-th feature is less than the threshold (step S103: NO), the determination unit 103 proceeds to step S105. If the similarity corresponding to the k-th feature is greater than or equal to the threshold (step S103: YES), the determination unit 103 determines that the k-th feature is to be deleted (step S104).

[0025] The determination unit 103 increments k by 1 (step S105) and determines whether k is greater than N (step S106). If k is less than or equal to N (step S106: NO), the determination unit 103 returns to step S103. If k is greater than N (step S106: YES), the determination has been completed for all N features, so the deletion unit 107 outputs the N features with the features that the determination unit 103 has deleted removed (step S107), and the process ends.

[0026] Figure 5 shows a modified example of the information processing device 100. In the information processing device 100 shown in Figure 5, feature quantities and ground truth data are input to the determination unit 103. In this configuration, the determination unit 103 determines the leak target based on two factors: the semantic similarity between the definition statement and the explanation statement, and the result of pre-training the model and calculating the feature inference contribution.

[0027] <Effects> According to the embodiment described above, it is possible to determine the features that cause leakage. In the flowchart described above, a list of features determined to be removed may be output without removing the features. In this embodiment, similarity was calculated using semantic vectors and cosine similarity, but any method capable of calculating the similarity between a definition and an explanation may be used, for example, a large-scale language model may be used to calculate the similarity.

[0028] <Example Hardware Configuration> The information processing device 100 in one embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure. Figure 6 is a diagram showing an example of the hardware configuration of the information processing device 100 according to one embodiment of the present disclosure. The above-described information processing device 100 may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0029] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the information processing device 100 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0030] Each function in the information processing device 100 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.

[0031] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, and so on.

[0032] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each part of the information processing device 100 may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0033] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.

[0034] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., Compact Disc, Digital Multipurpose Disc, Blu-ray® Disc), a smart card, flash memory (e.g., a card, stick, key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0035] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0036] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0037] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0038] Furthermore, the information processing device 100 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0039] Furthermore, each functional unit of the information processing device 100 may be implemented in a distributed manner on the cloud. Also, each functional unit may be implemented in multiple information processing devices. Furthermore, the same functional unit may be realized by multiple information processing devices.

[0040] One aspect of this disclosure is useful for machine learning.

[0041] 100 Information processing device 101 Vector acquisition unit 102 Similarity acquisition unit 103 Determination unit 104 Removal unit 1001 Processor 1002 Memory 1003 Storage 1004 Communication device 1005 Input device 1006 Output device 1007 Bus

Claims

1. An information processing device comprising: a similarity acquisition unit that acquires the similarity between an explanatory text describing a feature in machine learning training data and a definition text that shows the definition of the correct answer data in the training data; and a determination unit that determines whether or not to delete the feature from the training data based on the acquired similarity.

2. The information processing apparatus according to claim 1, further comprising a removal unit that outputs removed features obtained by removing the features determined by the determination unit to be removed from the training data.

3. A control method for an information processing device, comprising: obtaining the similarity between an explanatory statement describing a feature in machine learning training data and a definition statement indicating the definition of the correct answer data in the training data; and determining whether or not to delete the feature from the training data based on the obtained similarity.

4. A program that causes a computer to obtain the similarity between a descriptive statement explaining a feature in machine learning training data and a definition statement indicating the definition of the correct answer data in the training data, and to determine whether or not to remove the feature from the training data based on the obtained similarity.

Images