Analysis device, analysis method, and analysis program
By dividing tables with duplicate keys into groups and performing internal joins in parallel, the analysis device reduces memory usage and processing time during secure table joins, addressing the memory inefficiencies of conventional methods.
Patent Information
- Application Number
- PCT/JP2024/008158
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-01
- Filing Date
- 2024-03-04
- Publication Date
- 2024-11-07
AI Technical Summary
Conventional secure table join methods using secure calculation require excessive memory usage due to the expansion of the right table by duplicating records, which increases memory consumption during the inner join process.
The analysis device divides a table with duplicate key values into multiple record groups, performs an internal join using secure calculation for each group, and then conducts a row join on the resulting tables, reducing memory usage by avoiding the expansion of the right table.
This approach significantly reduces memory usage during secure table joins by performing internal joins in parallel, thereby shortening processing time and minimizing memory requirements.
Smart Images

Figure JP2024008158_07112024_PF_FP_ABST
Abstract
Description
Analytical device, analytical method, and analytical program
[0001] The present invention relates to an analysis device, an analysis method, and an analysis program.
[0002] A secure computation system is known that performs statistical computations on data while keeping the data secret and provides the resulting statistics to users. For example, the secure computation system is sometimes used to analyze data in fields such as the medical field, which handles important personal information.
[0003] Also, a method of joining tables using secure computation is known (see, for example, Patent Document 3).
[0004] International Publication No. 2019 / 124260 JP 2020-042128 A JP 2014-139640 A
[0005] Nippon Telegraph and Telephone Corporation, Secure Computation System and Its Principles, [online], [Retrieved November 24, 2022], Internet <URL: https: / / www.rd.ntt / sil / project / sc / secure_computation.html>
[0006] However, conventional table joining using secure computation has the problem of using a large amount of memory.
[0007] For example, in a conventional inner join of tables using secure computation, a procedure may be performed to expand the right table by duplicating records in the right table according to the number of specific records included in the left table. In this case, memory usage increases to store the expanded right table.
[0008] In order to solve the above-mentioned problems and achieve the object, the analysis device of the present invention is characterized by having a splitting unit that splits a first table including records with duplicate key values into multiple record groups in which the key values do not overlap, an inner join unit that performs an inner join between each of the multiple record groups and a second table having the key using secure computation, and a row join unit that performs a row join on the multiple tables obtained by the inner join.
[0009] According to the present invention, it is possible to reduce the amount of memory used in joining tables using secure computation.
[0010] FIG. 1 is a diagram showing an example of the configuration of an analysis system according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of an analysis device according to an embodiment. FIG. 3 is a diagram explaining a procedure for table joining according to an embodiment. FIG. 4 is a flowchart showing a processing flow of the analysis device according to an embodiment. FIG. 5 is a diagram showing an example of a computer that executes an analysis program. FIG. 6 is a diagram explaining a conventional procedure for table joining.
[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings, in which: FIG. 1 is a block diagram showing an analysis apparatus, an analysis method, and an analysis program according to an embodiment of the present invention;
[0012] First, the configuration of the analysis system will be described with reference to Fig. 1. The analysis system is a system for analyzing data using secure computation.
[0013] 1 , the analysis system 1 includes a secure computation system 10. The secure computation system 10 is connected to a providing device 20 and a providing device 30 via a network N. For example, the network N is the Internet. The secure computation system 10 is also connected to a terminal device 40.
[0014] The providing device 20 and the providing device 30 are devices on the data provider side. The providing device 20 and the providing device 30 provide (register) data to the secure computing system 10.
[0015] The data provided by the providing device 20 and the providing device 30 includes information that is desirably kept confidential (e.g., personal information such as an individual's name and address). For example, the providing device 20 and the providing device 30 provide data related to medical receipts and DPCs (Diagnosis Procedure Combinations) used in medical institutions.
[0016] The secure computation system 10 has a data storage unit 11 and a data processing unit 12. The data storage unit 11 includes a plurality of storage devices (storage device 111, storage device 112, storage device 113) that store data by secret sharing. The data processing unit 12 includes a plurality of computation devices (computation device 121, computation device 122, computation device 123) that process data by secure computation. Note that the number of storage devices and the number of computation devices are not limited to the example shown in FIG. 1 .
[0017] The secure computation system 10 can perform secret sharing and secure computation according to the method described in Non-Patent Document 1 (URL: https: / / www.rd.ntt / sil / project / sc / secure_computation.html).
[0018] First, data provided to the secure computing system 10 is divided into multiple shares (fragmentation). Then, each of the multiple shares is distributed and stored in multiple storage devices included in the data storage unit 11. In the example of Fig. 1, the provided data is divided into three shares. Then, storage device 111, storage device 112, and storage device 113 each store one share.
[0019] The data processing unit 12 performs secure computation on the shares stored in the data storage unit 11. The data processing unit 12 performs secure computation through multi-party computation using multiple computing devices. In the example of FIG. 1 , the data processing unit 12 performs secure computation using computing device 121, computing device 122, and computing device 123.
[0020] The data processing unit 12 can perform various statistical calculations without restoring the shares. For example, the data processing unit 12 can perform table operations such as sorting and joining, tally the number of records, calculate statistical quantities such as sums, averages, maximum values, minimum values, and sample variances, and perform statistical tests such as t-tests. Furthermore, the data processing unit 12 can perform statistical analyses such as regression analysis and principal component analysis.
[0021] The analysis device 13 analyzes the data using the data processing unit 12. The analysis device 13 provides the analysis result to the terminal device 40 on the data user side based on the result of the secure computation executed by the data processing unit 12. The user can obtain the data analysis result via the terminal device 40.
[0022] For example, the secure computing system 10 may be provided with data on the attributes and physical condition of each individual. The data on the attributes and physical condition is personal information that is preferably kept confidential. The data on the attributes and physical condition includes, for example, age, sex, height, weight, etc. The data storage unit 11 stores fragmented shares of the provided data in each storage device.
[0023] Note that each individual share is meaningless data on its own. Therefore, it is not possible to restore the original data from a single share. However, by collecting multiple shares, it is possible to restore the original data.
[0024] Although users of the data cannot view the registered data itself, they can view the analysis results of the data via the analysis device 13 and the terminal device 40. For example, if the data includes the gender and weight of individuals, users cannot view the gender and weight of each individual, but can view the "average weight of men," which is the analysis result of the data.
[0025] As an example, the data storage unit 11 can perform secret sharing using a technique known as Shamir's threshold secret sharing scheme. In this case, the data storage unit 11 stores, as shares, three coordinates passing through a polynomial whose intercept is the original data in each server. Furthermore, since the slope of the polynomial is determined randomly, the shares are not necessarily the same each time even if the original data is the same. Note that the original data may be a numerical value or data that has already been converted into a numerical value.
[0026] The secure computing system 10 can restore the original data from multiple shares. If the polynomial is a linear expression, the secure computing system 10 can find the intercept (corresponding to the original data) from the intersection of the axis and a line connecting two coordinates (corresponding to shares). On the other hand, since a line cannot be determined from a single coordinate, the original data cannot be restored.
[0027] As described above, the data processing unit 12 can perform secure computation on the original data without restoring the shares. For example, the result of adding shares represented by coordinates corresponds to the share obtained by adding the original data of each share.
[0028] In response to a request from the terminal device 40, the analysis device 13 causes the data processing unit 12 to perform processing using secure computation. Note that the data processing unit 12 or the terminal device 40 may realize functions equivalent to those of the analysis device 13. For example, the analysis system 1 may be configured without the analysis device 13. In that case, the terminal device 40 is connected to the data processing unit 12 and performs processing equivalent to that of the analysis device 13. Furthermore, statistical computation based on shares may be performed by the terminal device 40 rather than the data processing unit 12.
[0029] In the first embodiment, an example will be described in which the analysis device 13 joins tables using secure computation. The tables that the analysis device 13 joins are, for example, tables included in a relational database (RDB) in which multiple tables are associated with each other.
[0030] Here, table joins in conventional secure computation will be described with reference to Fig. 6. Fig. 6 is a diagram illustrating the conventional procedure for table joins. Note that an inner join is an operation that extracts records from two tables that have matching values in one or more specific columns (hereinafter referred to as join keys) and joins the extracted records. For example, an inner join corresponds to an INNER JOIN in SQL.
[0031] In the example of Fig. 6, a conventional analysis device (hereinafter referred to as analysis device 13a) joins table 51a and table 52a, and in the example of Fig. 6, the "ID" column is the join key.
[0032] First, the analysis device 13a expands the tables 51a and 52a as follows (step S1a).
[0033] The analysis device 13a generates a table 61a by adding a "Seqno" column to the table 51a, which is the left table. The analysis device 13a assigns different numbers to the "Seqno" column for records with duplicate values in the "ID" column.
[0034] For example, since there are two records in table 51a whose "ID" column has the value "A0001", the analysis device 13a assigns "0" to the "Seqno" column of the first of the two records in table 61a, and assigns "1" to the "Seqno" column of the second record.
[0035] For example, since there is only one record in table 51a whose "ID" column has a value of "A0002," the analysis device 13a assigns "0" to the "Seqno" column of that one record in table 61a. For example, since there is only one record in table 51a whose "ID" column has a value of "A0003," the analysis device 13a assigns "0" to the "Seqno" column of that one record in table 61a.
[0036] The analysis device 13a generates a table 62a by adding a "Seqno" column to the right table, table 52a. The analysis device 13a then duplicates each record in table 62a according to the maximum number of duplicates of the records in table 51a. The analysis device 13a performs duplication so that the number of duplicates in table 62a for each original record is equal to the maximum number of duplicates in table 51a.
[0037] Because the maximum number of duplicates for table 51a, which is the left table, is 2, the analysis device 13a duplicates each record of table 52a twice. This is equivalent to the analysis device 13a adding one record identical to each record (maximum number of duplicates - 1) to table 62a, which is table 52a in a state where only the "Seqno" column has been added.
[0038] As with the case of table 61a, the analysis device 13a assigns different numbers to the "Seqno" column for records in the duplicated table 62a that have duplicate values in the "ID" column. For example, of two records in table 62a where the "ID" column is "A0001" and the "Drug Name" column is "Capecitabine," the analysis device 13a assigns "0" to the "Seqno" column for the first record and "1" to the "Seqno" column for the second record.
[0039] The analysis device 13a performs an inner join between the tables 61a and 62a using the "ID" and "Seqno" columns as join keys (step S2a). Note that while the join keys in the inner join between the tables 61a and 62a are the "ID" and "Seqno" columns, the join key in the overall procedure for the inner join between the tables 51a and 52a is the "ID" column.
[0040] In this way, in conventional techniques, large tables such as table 62a are generated, resulting in increased memory usage. In contrast, the analysis device 13 of the first embodiment can reduce the amount of memory usage when joining tables using secure computation compared to conventional techniques.
[0041] The configuration of the analysis device 13 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the analysis device according to the embodiment.
[0042] The following describes each unit of the analysis device 13. As shown in FIG. 2, the analysis device 13 includes a communication unit 131, an input unit 132, an output unit 133, a storage unit 134, and a control unit 135.
[0043] The communication unit 131 communicates data with other devices. For example, the communication unit 131 is a network interface card (NIC). The communication unit 131 can send and receive data to and from other devices.
[0044] The input unit 132 is an interface for receiving data input, and is connected to input devices such as a mouse and a keyboard.
[0045] The output unit 133 is an interface for outputting data and is connected to input devices such as a display and a speaker.
[0046] The storage unit 134 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. Note that the storage unit 134 may also be a data-rewritable semiconductor memory such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 134 stores an operating system (OS) and various programs executed by the analysis device 13.
[0047] The control unit 135 controls the entire analyzer 13. The control unit 135 is, for example, an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The control unit 135 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory.
[0048] The control unit 135 functions as various processing units by running various programs. For example, the control unit 135 has a dividing unit 1351, an inner combining unit 1352, a row combining unit 1353, and an output control unit 1354.
[0049] The procedure for table merging will be described with reference to Fig. 3, along with the functions of each processing unit of the control unit 135. Fig. 3 is a diagram illustrating the procedure for table merging according to an embodiment. For the sake of explanation, Fig. 3 shows the contents of each table in a readable natural language format, but in reality, the process shown in Fig. 3 is performed by secure computation on tables stored in an unreadable shared format (for example, a list of meaningless numerical values).
[0050] In the example of Fig. 3, tables 51 and 52 are joined. Also, in the example of Fig. 3, the "ID" column is the join key. Tables 51 and 52 are examples of a first table and a second table, respectively. In both tables 51 and 52, there is a duplicate record in which the value of the "ID" column, which is the join key, is "A0001."
[0051] First, the dividing unit 1351 keeps one record and deletes the other records for each record group with overlapping join keys included in table 51 (step S1). For example, the dividing unit 1351 generates record group 61 by deleting from table 51 a record whose "ID" column value is "A0001" and whose "Disease Name" column value is "Diabetes".
[0052] Furthermore, the dividing unit 1351 generates a record group 62 that includes the deleted record, i.e., a record whose "ID" column value is "A0001" and whose "Disease" column value is "Diabetes." If the record group 62 includes a record with a duplicate join key, the dividing unit 1351 generates further record groups by recursively performing the process that was performed on the table 51.
[0053] In other words, in step S1, the dividing unit 1351 divides the table 51, which includes records with duplicate key values, into a plurality of record groups with no duplicate key values.
[0054] Next, the inner join unit 1352 performs an inner join between each of the multiple tables and the table 52 having the key by secure computation (step S2). At this time, the inner join unit 1352 can execute multiple inner join processes in parallel. For example, the inner join between the record group 61 and the table 52 and the inner join between the record group 61 and the table 52 may be executed in parallel.
[0055] The processor of each computing device in the data processing unit 12 has multiple cores. In this case, the internal join unit 1352 can assign multiple internal join processes to each of the multiple cores and execute them in parallel.
[0056] The row join unit 1353 performs a row join on the multiple tables obtained by the inner join (tables 71 and 72 in FIG. 3) (step S3). As a result, table 81 is obtained as a result of inner joining table 51 and table 52. A row join is an operation that integrates one table in the row direction with another table. A row join corresponds to, for example, a UNION in SQL.
[0057] The output control unit 1354 outputs the table 81 to the terminal device 40. The output control unit 1354 may also use the table 81 to perform further statistical analysis and output the results.
[0058] 4 is a flowchart showing the processing flow of the analysis device according to the embodiment. As shown in FIG. 4, the analysis device 13 acquires the left table and the right table to be joined (step S101). The processing of step S101 may be to cause the data processing unit 12 to acquire the tables stored in the data storage unit 11 as shares.
[0059] Here, the left table contains records with duplicate join key values. The join key may be one or more columns set as the primary key, or one or more columns specified by the user, etc.
[0060] Next, the analysis device 13 divides the left table into a plurality of record groups whose join keys do not overlap (step S102).
[0061] Next, the analysis device 13 inner joins each of the divided record groups with the right table (step S103). The analysis device 13 may execute multiple inner join processes included in step S103 in parallel.
[0062] The analysis device 13 then performs row joins on the multiple tables obtained by the inner join (step S104). The analysis device 13 may execute the row join processes in parallel.
[0063] The analysis device 13 then outputs the table obtained by the row join (step S105). The analysis device 13 may also use the table obtained by the row join to further calculate statistics and output the results as analysis results.
[0064] [Effects of the embodiment] As described above, the analysis device 13 has a splitting unit 1351, an inner join unit 1352, and a row join unit 1353. The splitting unit 1351 splits a first table including records with duplicate join key values into multiple record groups with no duplicate join key values. The inner join unit 1352 performs inner joins between each of the multiple tables and a second table having a join key using secure computation. The row join unit 1353 performs row joins on the multiple tables obtained by the inner join.
[0065] As explained using Fig. 6, in conventional table joins using secure computation, the tables to be joined are expanded, resulting in increased memory usage. On the other hand, in the first embodiment, the join is performed using secure computation without expanding the tables to be joined, thereby reducing memory usage.
[0066] The inner join unit 1352 performs inner joins in parallel between each of the multiple tables and a second table having a join key. By performing multiple inner joins in parallel in this way, the time required for processing is shortened.
[0067] [System Configuration, etc.] The components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic. The program may be executed not only by the CPU but also by other processors such as a GPU.
[0068] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.
[0069] [Program] In one embodiment, the analysis device 13 can be implemented by installing an analysis program that executes the above-described analysis process as package software or online software on a desired computer. For example, by having an information processing device execute the above-described analysis program, the information processing device can function as the analysis device 13. The information processing device referred to here includes desktop and notebook personal computers. Other information processing devices also include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).
[0070] The analysis device 13 can also be implemented as an analysis server device that provides services related to the above-mentioned analysis processing to a client terminal device used by a user. For example, the analysis server device is implemented as a server device that provides an analysis service that takes two tables to be joined as input and outputs the joined table.
[0071] 5 is a diagram showing an example of a computer that executes an analysis program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.
[0072] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.
[0073] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the analysis device 13 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing processes similar to those of the functional configuration of the analysis device 13 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).
[0074] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.
[0075] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.
[0076] REFERENCE SIGNS LIST 1 Analysis system 10 Secure computation system 11 Data storage unit 12 Data processing unit 13 Analysis device 131 Communication unit 132 Input unit 133 Output unit 134 Storage unit 135 Control unit 1351 Split unit 1352 Internal join unit 1353 Row join unit 1354 Output control unit
Claims
1. An analysis device comprising: a splitting unit that splits a first table including records with duplicate key values into multiple record groups in which the key values do not overlap; an inner join unit that uses secure computation to perform an inner join between each of the multiple record groups and a second table having the key; and a row join unit that performs a row join on the multiple tables obtained by the inner join.
2. The analysis device according to claim 1, wherein the inner join section performs inner joins in parallel between each of the plurality of record groups and a second table having the key.
3. An analysis method executed by an analysis device, comprising: a division step of dividing a first table including records with duplicate key values into multiple record groups in which the key values do not overlap; an inner join step of performing an inner join between each of the multiple record groups and a second table having the key using secret calculation; and a row join step of performing a row join on the multiple tables obtained by the inner join.
4. An analysis program that causes a computer to execute the following steps: a division step of dividing a first table including records with duplicate key values into multiple record groups in which the key values do not overlap; an inner join step of performing an inner join between each of the multiple record groups and a second table having the key using secret computation; and a row join step of performing a row join on the multiple tables obtained by the inner join.
Citation Information
Patent Citations
Equal coupling system and method of table by secret calculation
JP2014139640A
Secret statistic processing system, method, statistical processing apparatus, and program
JP2020042128A
Secure computation system and method
WO2019124260A1
Coupling method of sum set integration processing and equivalent linking processing and database system and program
JP2011242851A
Computer system and data management method
WO2012131927A1
