Analytical device, analytical method, and analytical program

By dividing tables with duplicate keys into groups and performing parallel secure joins, the device reduces memory usage and processing time for secure table joins, addressing the memory inefficiencies of conventional methods.

JP2024160651APending Publication Date: 2024-11-14NTT COMM CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023075878
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Conventional table join using secure calculation consumes a large amount of memory due to the expansion of tables, particularly in inner joins where the right table is duplicated based on the number of specific records in the left table.

Method used

The analysis device divides a first table with duplicate key values into multiple record groups with non-duplicate key values, performs an internal join of each group with a second table using secure calculation, and then combines the results in a row join, reducing memory usage and processing time.

Benefits of technology

This approach significantly reduces memory consumption and processing time for table joins by avoiding the expansion of the right table, while maintaining data security through secure calculation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160651000001_ABST
    Figure 2024160651000001_ABST
Patent Text Reader

Abstract

To provide an analytical device capable of reducing the amount of memory used in join of tables by a secure computation.SOLUTION: An analytical device includes a division section, an internal join section, and a line join section. The division section divides a first table including records in which values of a join key are duplicated into a plurality of record groups in which the values of the join key are not duplicated. The internal join section performs internal join of each of the plural record groups and a second table having the join key by a secure computation. The line join section performs line join on a plurality of tables obtained by the internal join.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an analysis device, an analysis method, and an analysis program. [Background technology]

[0002] Conventionally, there is known a secure computation system that performs statistical computations while keeping data secret and provides the statistical values ​​obtained as a result of the computations to a user. For example, the secure computation system may be used for analyzing data in the medical field, etc., which handles important personal information.

[0003] Also, a method of joining tables using secure computation is known (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 124260 [Patent Document 2] JP 2020-042128 A [Patent Document 3] JP 2014-139640 A [Non-patent literature]

[0005] [Non-Patent Document 1] Nippon Telegraph and Telephone Corporation, Secure Computation System and Its Principles, [online], [Retrieved November 24, 2022], Internet<URL:https: / / www.rd.ntt / sil / project / sc / secure_computation.html> Summary of the Invention [Problem to be solved by the invention]

[0006] However, the conventional table joining using secure computation has a problem of large memory usage.

[0007] For example, in a conventional inner join of tables using secure computation, a procedure may be performed to extend the right table by duplicating records in the right table according to the number of specific records included in the left table. In this case, memory usage increases in order to hold the extended right table. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems and achieve the object, the analysis device of the present invention is characterized by having a splitting unit that splits a first table including records with duplicate key values ​​into multiple record groups in which the key values ​​do not overlap, an inner join unit that performs an inner join between each of the multiple record groups and a second table having the key using secret computation, and a row join unit that performs a row join on the multiple tables obtained by the inner join. Effect of the Invention

[0009] According to the present invention, it is possible to reduce the amount of memory used in joining tables using secure computation. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an analysis system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the analysis device according to the embodiment. [Diagram 3] FIG. 3 is a diagram for explaining a procedure for joining tables according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing a process flow of the analysis device according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a computer that executes an analysis program. [Figure 6] FIG. 6 is a diagram for explaining a conventional procedure for joining tables. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of an analysis device, an analysis method, and an analysis program according to the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment described below.

[0012] First, the configuration of the analysis system will be described with reference to Figure 1. The analysis system is a system for analyzing data using secure computation.

[0013] 1, the analysis system 1 includes a secure computation system 10. The secure computation system 10 is connected to a providing device 20 and a providing device 30 via a network N. For example, the network N is the Internet. The secure computation system 10 is also connected to a terminal device 40.

[0014] The providing device 20 and the providing device 30 are devices on the data provider side. The providing device 20 and the providing device 30 provide (register) data to the secure computing system 10.

[0015] The data provided by the providing device 20 and the providing device 30 includes information that is desirably kept secret (for example, personal information such as an individual's name and address). For example, the providing device 20 and the providing device 30 provide data related to receipts and DPC (Diagnosis Procedure Combination) used in medical institutions.

[0016] The secure computation system 10 includes a data storage unit 11 and a data processing unit 12. The data storage unit 11 includes a plurality of storage devices (storage device 111, storage device 112, storage device 113) that store data by secret sharing. The data processing unit 12 includes a plurality of computation devices (computation device 121, computation device 122, computation device 123) that process data by secure computation. Note that the number of storage devices and the number of computation devices are not limited to the example shown in FIG. 1.

[0017] The secure computation system 10 can perform secret sharing and secure computation according to the method described in Non-Patent Document 1 (URL: https: / / www.rd.ntt / sil / project / sc / secure_computation.html).

[0018] First, data provided to the secure computing system 10 is divided (fragmented) into a plurality of shares. Then, each of the plurality of shares is distributed and stored in a plurality of storage devices included in the data storage unit 11. In the example of Fig. 1, the provided data is divided into three shares. Then, storage device 111, storage device 112, and storage device 113 each store one share.

[0019] Data processing unit 12 performs secure computation on the shares accumulated in data accumulation unit 11. Data processing unit 12 performs secure computation by multi-party computation using multiple computing devices. In the example of FIG. 1, data processing unit 12 performs secure computation by computing device 121, computing device 122, and computing device 123.

[0020] The data processing unit 12 can perform various statistical calculations without restoring the shares. For example, the data processing unit 12 can perform table operations such as sorting and joining, tallying the number of records, calculating statistics such as sums, averages, maximum values, minimum values, and sample variances, and performing statistical tests such as t-tests. Furthermore, the data processing unit 12 can perform statistical analyses such as regression analysis and principal component analysis.

[0021] The analysis device 13 analyzes the data by using the data processing unit 12. The analysis device 13 provides the analysis result to the terminal device 40 on the data user side based on the result of the secure computation executed by the data processing unit 12. The user can obtain the data analysis result via the terminal device 40.

[0022] For example, the secure computing system 10 may be provided with data on the attributes and physical state of each individual. The data on the attributes and physical state is personal information that is preferably kept confidential. The data on the attributes and physical state includes, for example, age, sex, height, weight, etc. The data storage unit 11 stores the provided data into fragmented shares in each storage device.

[0023] Each share is meaningless on its own. Therefore, it is not possible to restore the original data from a single share. However, by collecting multiple shares, it is possible to restore the original data.

[0024] Although users of the data cannot view the registered data itself, they can view the results of data analysis via the analysis device 13 and the terminal device 40. For example, if the data includes the gender and weight of individuals, the users cannot view the gender and weight of each individual, but can view the "average weight of men," which is the result of the data analysis.

[0025] As an example, the data storage unit 11 can perform secret sharing using a technique known as Shamir's threshold secret sharing scheme. In this case, the data storage unit 11 stores, as shares, three coordinates passing through a polynomial with the original data as an intercept in each server. In addition, since the slope of the polynomial is determined randomly, the shares are not necessarily the same each time even if the original data is the same. The original data may be a numerical value, or may be data that has already been converted into a numerical value.

[0026] The secure computation system 10 can restore the original data from multiple shares. If the polynomial is a linear expression, the secure computation system 10 can find an intercept (corresponding to the original data) from the intersection of a line connecting two coordinates (corresponding to shares) and the axis. On the other hand, since a line cannot be determined from a single coordinate, the original data cannot be restored.

[0027] As described above, the data processing unit 12 can perform secure computation on the original data without recovering the shares. For example, the result of adding the shares represented by coordinates corresponds to the share resulting from adding the original data of each share.

[0028] The analysis device 13 causes the data processing unit 12 to execute processing using secure computation in response to a request from the terminal device 40. Note that the data processing unit 12 or the terminal device 40 may realize functions equivalent to those of the analysis device 13. For example, the analysis system 1 may be configured not to include the analysis device 13. In that case, the terminal device 40 is connected to the data processing unit 12 and executes processing equivalent to that of the analysis device 13. Furthermore, statistical computation based on shares may be executed by the terminal device 40 instead of the data processing unit 12.

[0029] In the first embodiment, an example will be described in which the analysis device 13 performs table joining using secure computation. Note that the tables to be joined by the analysis device 13 are, for example, tables included in a relational database (RDB) in which multiple tables are associated with each other.

[0030] Here, table join in conventional secure computation will be explained with reference to Fig. 6. Fig. 6 is a diagram explaining the conventional procedure for table join. Note that inner join is an operation for extracting records with matching values ​​in one or more specific columns (hereinafter, join keys) from two tables, and joining the extracted records. For example, inner join corresponds to INNER JOIN in SQL.

[0031] In the example of Fig. 6, a conventional analysis device (hereinafter, analysis device 13a) joins table 51a and table 52a. Also, in the example of Fig. 6, the "ID" column is the join key.

[0032] First, the analysis device 13a expands the tables 51a and 52a as follows (step S1a).

[0033] The analysis device 13a generates a table 61a by adding a "Seqno" column to the left table 51a. The analysis device 13a assigns different numbers to the "Seqno" column for records that have duplicate values ​​in the "ID" column.

[0034] For example, since there are two records in table 51a whose "ID" column has the value "A0001", the analysis device 13a assigns a "0" to the "Seqno" column of the first of the two records in table 61a, and assigns a "1" to the "Seqno" column of the second record.

[0035] For example, since there is only one record whose "ID" column value in table 51a is "A0002", analysis device 13a assigns "0" to the "Seqno" column of that one record in table 61a. For example, since there is only one record whose "ID" column value in table 51a is "A0003", analysis device 13a assigns "0" to the "Seqno" column of that one record in table 61a.

[0036] The analysis device 13a generates table 62a by adding a "Seqno" column to table 52a, which is the right table. Then, the analysis device 13a duplicates each record of table 62a according to the maximum number of duplicates of the records in table 51a. The analysis device 13a performs duplication so that the number of duplicates in table 62a of each record of the duplication source becomes equal to the maximum number of duplicates in table 51a.

[0037] Because the maximum number of duplicates for table 51a, which is the left table, is two, the analysis device 13a duplicates each record of table 52a into two. This is equivalent to the analysis device 13a adding one record identical to each record (the maximum number of duplicates minus 1) to table 62a, which is in a state where only the "Seqno" column has been added to table 52a.

[0038] As in the case of table 61a, the analytical device 13a assigns different numbers to the "Seqno" column for records in which the "ID" column of the duplicated table 62a has duplicate values. For example, of two records in which the "ID" column of table 62a is "A0001" and the "Drug Name" column is "Capecitabine," the analytical device 13a assigns "0" to the "Seqno" column of the first record and assigns "1" to the "Seqno" column of the second record.

[0039] The analysis device 13a performs an inner join between the tables 61a and 62a using the "ID" and "Seqno" columns as join keys (step S2a). Note that while the join keys in the inner join between the tables 61a and 62a are the "ID" and "Seqno" columns, the join key in the entire procedure of the inner join between the tables 51a and 52a is the "ID" column.

[0040] In this way, in the conventional technology, a large-sized table such as table 62a is generated, and memory usage increases. In contrast, the analysis device 13 of the first embodiment can reduce memory usage in combining tables using secure computation compared to the conventional technology.

[0041] The configuration of the analysis device 13 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the analysis device according to the embodiment.

[0042] The following describes each part of the analysis device 13. As shown in FIG.

[0043] The communication unit 131 communicates data with other devices. For example, the communication unit 131 is a network interface card (NIC). The communication unit 131 can transmit and receive data with other devices.

[0044] The input unit 132 is an interface for receiving input of data, and is connected to input devices such as a mouse and a keyboard.

[0045] The output unit 133 is an interface for outputting data, and is connected to input devices such as a display and a speaker.

[0046] The storage unit 134 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. The storage unit 134 may be a semiconductor memory in which data can be rewritten, such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 134 stores an operating system (OS) and various programs executed by the analysis device 13.

[0047] The control unit 135 controls the entire analysis device 13. The control unit 135 is, for example, an electronic circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 135 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory.

[0048] The control unit 135 functions as various processing units by the operation of various programs. For example, the control unit 135 has a division unit 1351, an internal combination unit 1352, a row combination unit 1353, and an output control unit 1354.

[0049] The procedure for table merging will be described with reference to Fig. 3, along with the function of each processing unit of the control unit 135. Fig. 3 is a diagram for explaining the procedure for table merging according to the embodiment. For the sake of explanation, Fig. 3 shows the contents of each table in a state in which they can be read as natural language, but in reality, the process shown in Fig. 3 is performed by secure computation on tables accumulated in an unreadable share state (for example, a list of numerical values ​​that appear meaningless).

[0050] In the example of Fig. 3, table 51 and table 52 are joined. Also, in the example of Fig. 3, the "ID" column is the join key. Tables 51 and 52 are examples of a first table and a second table, respectively. In both tables 51 and 52, there is a duplicate record in which the value of the "ID" column, which is the join key, is "A0001."

[0051] First, the splitting unit 1351 leaves one record and deletes the other records for each record group with overlapping join keys included in table 51 (step S1). For example, the splitting unit 1351 generates record group 61 by deleting a record whose "ID" column value is "A0001" and whose "Disease name" column value is "Diabetes" from table 51.

[0052] Furthermore, the splitting unit 1351 generates a record set 62 having the deleted records, i.e., records having an "ID" column value of "A0001" and a "Disease" column value of "Diabetes". If the record set 62 includes records with duplicate join keys, the splitting unit 1351 generates further record sets by recursively performing the process performed on the table 51.

[0053] In other words, in step S1, the dividing unit 1351 divides the table 51 including records with duplicate key values ​​into a plurality of record groups with no duplicate key values.

[0054] Next, the inner join unit 1352 performs inner join between each of the multiple tables and the table 52 having the key by secure computation (step S2). At this time, the inner join unit 1352 can execute multiple inner join processes in parallel. For example, the inner join between the record set 61 and the table 52 and the inner join between the record set 61 and the table 52 may be executed in parallel.

[0055] The processor of each computing device in the data processing unit 12 has multiple cores. In this case, the internal join unit 1352 can assign multiple internal join processes to the multiple cores and execute them in parallel.

[0056] The row join unit 1353 performs row join on the multiple tables (tables 71 and 72 in FIG. 3) obtained by the inner join (step S3). As a result, table 81 is obtained as a result of inner joining table 51 and table 52. Row join is an operation for integrating one table in the row direction with another table. Row join corresponds to UNION in SQL, for example.

[0057] The output control unit 1354 outputs the table 81 to the terminal device 40. Furthermore, the output control unit 1354 may use the table 81 to perform further statistical analysis and output the result.

[0058] Fig. 4 is a flowchart showing a process flow of the analysis device according to the embodiment. As shown in Fig. 4, the analysis device 13 acquires a left table and a right table to be joined (step S101). The process of step S101 may be to cause the data processing unit 12 to acquire the tables accumulated in the data accumulation unit 11 as shares.

[0059] Here, the left table contains records with duplicate values ​​for the join key. The join key may be one or more columns set as a primary key, or one or more columns specified by a user or other user.

[0060] Next, the analysis device 13 divides the left table into a plurality of record groups whose join keys do not overlap (step S102).

[0061] Next, the analysis device 13 performs inner join on each of the divided multiple record groups with the right table (step S103). The analysis device 13 may execute multiple inner join processes included in step S103 in parallel.

[0062] Here, the analysis device 13 performs row joining on the multiple tables obtained by the inner join (step S104). The analysis device 13 may execute the row joining process in parallel.

[0063] Then, analysis device 13 outputs the table obtained by row joining (step S105). Analysis device 13 may further perform calculation of statistics or the like using the table obtained by row joining, and output the result as an analysis result.

[0064] [Effects of the embodiment] As described above, the analysis device 13 has a splitting unit 1351, an inner join unit 1352, and a row join unit 1353. The splitting unit 1351 splits a first table including records with duplicated join key values ​​into multiple record groups with no duplicated join key values. The inner join unit 1352 performs inner joins between each of the multiple tables and a second table having a join key by secure computation. The row join unit 1353 performs row joins on the multiple tables obtained by the inner join.

[0065] As described with reference to Fig. 6, in conventional table joins using secure computation, the tables to be joined are expanded, resulting in increased memory usage. On the other hand, in the first embodiment, the join is performed using secure computation without expanding the tables to be joined, resulting in reduced memory usage.

[0066] The inner join unit 1352 performs inner joins in parallel between each of the multiple tables and a second table having a join key. By performing multiple inner joins in parallel in this way, the time required for processing is shortened.

[0067] [System configuration, etc.] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. Note that the program may be executed not only by the CPU but also by other processors such as a GPU.

[0068] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

[0069] [program] In one embodiment, the analysis device 13 can be implemented by installing an analysis program that executes the above-mentioned analysis processing as package software or online software on a desired computer. For example, the above-mentioned analysis program can be executed by an information processing device, thereby making the information processing device function as the analysis device 13. The information processing device here includes desktop and notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System), and further slate terminals such as PDA (Personal Digital Assistant), etc.

[0070] Furthermore, the analysis device 13 can be implemented as an analysis server device that provides services related to the above-mentioned analysis processing to a client, the client being a terminal device used by a user. For example, the analysis server device is implemented as a server device that provides an analysis service that uses two tables to be joined as input and outputs the joined table.

[0071] 5 is a diagram showing an example of a computer that executes an analysis program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0072] The memory 1010 includes a read only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0073] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the analysis device 13 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same process as the functional configuration of the analysis device 13 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0074] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0075] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070. [Explanation of symbols]

[0076] 1. Analysis system 10. Secure Computing System 11 Data Storage Section 12 Data Processing Section 13 Analyzer 131 Communications Department 132 Input section 133 Output section 134 Storage section 135 Control Unit 1351 Split section 1352 Internal joint 1353 line joiner 1354 Output control section

Claims

1. a division unit that divides a first table including records having duplicate key values ​​into a plurality of record groups each having a unique key value; an inner join unit that performs an inner join between each of the plurality of record groups and a second table having the key by secure computation; a row join unit that performs row joins on the multiple tables obtained by the inner join; An analytical device comprising:

2. 2. The analysis apparatus according to claim 1, wherein the inner join section performs inner joins in parallel between each of the plurality of record groups and a second table having the key.

3. An analytical method performed by an analytical device, comprising: a division step of dividing a first table including records having duplicate key values ​​into a plurality of record groups each having no duplicate key values; an inner join step of performing an inner join between each of the plurality of record groups and a second table having the key by secure computation; a row join process for joining rows of the multiple tables obtained by the inner join; An analytical method comprising the steps of:

4. a division step of dividing a first table including records having duplicate key values ​​into a plurality of record groups each having no duplicate key values; an inner join step of performing an inner join between each of the plurality of record groups and a second table having the key by secure computation; a row join step of performing row join on the multiple tables obtained by the inner join; An analysis program characterized by causing a computer to execute the above.

Citation Information

Patent Citations

  • Equal coupling system and method of table by secret calculation

    JP2014139640A

  • Secret statistic processing system, method, statistical processing apparatus, and program

    JP2020042128A

  • Secure computation system and method

    WO2019124260A1