Processing system, processing method, and processing program

By compressing files before storage in secret sharing systems, the system addresses inefficiencies in disk usage by reducing surplus sectors, enhancing storage efficiency and security.

JP2024160633APending Publication Date: 2024-11-14NTT COMM CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023075851
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Secret sharing storage systems face inefficiencies due to surplus sectors when storing small files, as each fragment file size is smaller than the sector size, leading to increased disk usage.

Method used

A processing system that compresses multiple files into one file before uploading to secret sharing storage, reducing surplus area by encrypting and distributing the compressed file across multiple servers.

Benefits of technology

Reduces surplus disk space usage by 1.1 times, improving efficiency by 1.8 times compared to non-compressed storage, while maintaining data security through secret sharing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160633000001_ABST
    Figure 2024160633000001_ABST
Patent Text Reader

Abstract

To reduce a surplus area when storing each fragment file on a disk in a secret distributed storage.SOLUTION: A processing system 1 comprises: a client 10 that uploads a file to be stored; and a secret distributed storage 30 for distributing and storing fragment files obtained by dividing the uploaded file in a plurality of storage servers 30-1 to 30-3. The client 10 includes a backup section 11 for compressing a plurality of files to be stored into one file at a predetermined compression rate and then uploading the compressed file. The secret distributed storage 30 includes: the plurality of storage servers 30-1 to 30-3; and a distribution server 20 for encrypting the compressed file, then dividing the encrypted file into fragment files, and distributing and storing each fragment file in the plurality of storage servers 30-1 to 30-3.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a processing system, a processing method, and a processing program. [Background technology]

[0002] When handling personal or important information, it is necessary to store the data securely. Encrypting data using symmetric key cryptography or public key cryptography is one method of storing data securely, but if the ciphertext is stolen from the server, there is a risk that the data may be restored in the future. Therefore, a technology called secret sharing has been proposed, which makes it impossible to restore the data from the stolen ciphertext alone, even if one server is attacked. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-140310 A [Non-patent literature]

[0004] [Non-Patent Document 1] Dai Igarashi, Kota Tsuyuzaki, Yuto Kawahara, “SHSS: An ultra-fast secret sharing library for object storage”, Research Report on Security Psychology and Trust (SPT), 2015-SPT-14, vol 26, pp.1-8, [online], [Retrieved February 22, 2023], Internet <URL:https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=pages_view_main&active_action=repository_view_main_item_detail&item_id=142625&item_no=1&page_id=13&block_id=8> [Non-Patent Document 1] Adi Shamir, How to share a secret, Communications of the ACM, vol. 22, issue 11, pp.612--613, Nov., 1979, [online], [Retrieved February 22, 2023], Internet<URL:https: / / dl.acm.org / doi / abs / 10.1145 / 359168.359176> Summary of the Invention [Problem to be solved by the invention]

[0005] Secret sharing is a technique that encrypts input data by splitting it into fragment files (for example, called shares) and distributing and storing each fragment file on different servers. Each fragment file cannot be restored by itself.

[0006] However, when storing a large number of small files of around a few KB in size, secret sharing storage poses the problem that the size of each fragmented file is smaller than the sector size, resulting in excess sectors for each file and an increase in disk usage.

[0007] The present invention has been made in consideration of the above, and aims to provide a processing system, processing method, and processing program that can reduce excess area when storing each fragment file on disk in secret sharing storage. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems and achieve the objective, the processing system of the present invention is a processing system having a client that uploads a file to be stored, and a secret sharing storage that divides the uploaded file into fragment files and stores them in a distributed manner across a plurality of storage servers, wherein the client has a backup unit that compresses the multiple files to be stored into one file at a predetermined compression rate and then uploads the compressed file, and the secret sharing storage has the multiple storage servers and a distribution server that encrypts the compressed file, divides it into fragment files, and stores each fragment file in a distributed manner across the multiple storage servers. Effect of the Invention

[0009] According to the present invention, it is possible to reduce the surplus area when storing each fragment file on a disk in a secret sharing storage. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an overview of distributed storage in a typical secret sharing storage. [Diagram 3] FIG. 3 is a diagram for explaining an outline of the process according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining an outline of conventional processing. [Diagram 5] FIG. 5 is a sequence diagram showing a processing procedure of the processing method according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating a computer that executes a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of a processing system, a processing method, and a processing program according to the present application will be described in detail with reference to the drawings. Note that the processing system, the processing method, and the processing program according to the present application are not limited to these embodiments.

[0012] In the following embodiments, the processing flow of a processing system, a processing method, and a processing program according to the embodiments will be described in order, and finally, the effects of the embodiments will be described.

[0013] [Embodiment Mode] First, an embodiment will be described. The embodiment will be described taking as an example a case where file data (files) to be stored are stored in a plurality of servers.

[0014] In the embodiment, the facility compresses multiple uploaded files into one file and then uploads the file to the secret sharing storage, thereby efficiently reducing the surplus area of ​​the file. This allows efficient generation of fragment files.

[0015] [Processing system configuration] The configuration of a processing system according to an embodiment will be described below. Fig. 1 is a block diagram showing an example of the configuration of a processing system according to an embodiment.

[0016] In the following, an example will be described in which the processing system 1 is configured with a client 10 that uploads files and a secret sharing storage 30, as shown in Fig. 1. Note that the configuration shown in Fig. 1 is merely an example, and the specific configuration and the number of each device are not particularly limited.

[0017] The client 10 acquires a file to be stored, and uploads the acquired file to be stored to the secret sharing storage 30. For example, an operator of the client 10 selects a file to be uploaded via a WebUI screen for the processing system 1 displayed on a Web browser, and uploads the file to the secret sharing storage 30.

[0018] The client 10 has a backup unit 11 that performs upload processing on files to be stored. The backup unit 11 has a compression unit 12 that compresses a plurality of files into one file at a predetermined compression rate.

[0019] The backup unit 11 acquires files to be stored. In the backup unit 11, the compression unit 12 compresses the multiple files to be stored into one file at a predetermined compression rate.

[0020] The compression unit 12 compresses multiple files into one file at a compression rate of 100% or less. For example, the compression unit 12 compresses multiple files into one file at a compression rate of 100%. That is, the compression unit 12 combines multiple files into one file. Also, the compression unit 12 compresses multiple files into one file at a compression rate of 75%.

[0021] The backup unit 11 compresses a plurality of files to be stored into one file at a predetermined compression rate, and then uploads the compressed file to the secret sharing storage 30.

[0022] Secret sharing storage 30 stores the uploaded file (one file after compression) in fragmented fragment files distributed across multiple servers. Each fragment file is meaningless data, and the original file cannot be restored with a single fragment file, so information is not leaked. However, the original file can be restored when a certain number of fragment files are collected.

[0023] The secret sharing storage 30 is a processing system that includes the secret sharing storage 30. In the secret sharing storage 30, one compressed file uploaded from the client 10 is distributed and stored in the form of fragmented files.

[0024] After encrypting the file data, the secret sharing storage 30 distributes and stores (secretly distributes) the fragmented files in a plurality of storage servers 30-1 to 30-3. The fragmented files are distributed and stored in a plurality of storage servers 30-1 to 30-3 in a state in which it is possible to identify which facility the data was uploaded from.

[0025] The secret sharing storage 30 includes a distribution server 20 in front of a plurality of storage servers 30-1 to 30-3. The distribution server 20 includes a secret sharing engine 21 that controls the secret sharing.

[0026] The secret sharing engine 21 encrypts one compressed file uploaded by the client 10, divides it into fragment files, and distributes and stores each fragment file among a plurality of storage servers 30-1 to 30-3.

[0027] [Explanation of distributed storage overview] Next, an overview of distributed storage in a general secret sharing storage will be described with reference to Fig. 2. Fig. 2 is a diagram showing an overview of distributed storage in a general secret sharing storage.

[0028] First, when a client uploads a file to the secret sharing storage, the distribution server encrypts the uploaded file using secret sharing and divides it into n fragment files (Figure 2 (1)).

[0029] Then, the distributed server distributes the fragment files F1, F2, . . . , F n are the disks D1, , and D k+1 , ...

[0030] [Overview of the processing in the first embodiment] Next, an outline of the process in the embodiment 1 will be described. Fig. 3 is a diagram for explaining the outline of the process in the embodiment.

[0031] In Fig. 3, an example is explained in which there are 10 4KB files as uploaded files to be stored. An example is explained in which the most space-efficient Computational method is adopted, with the minimum configuration (k, n) = (2, 3) (k: number of fragments required for restoration, n: total number of fragment files). An example is explained in which the distributed server 20 divides one file into two, and generates two fragment files and one parity file of the same size as the fragment files (corresponding to m files).

[0032] The client 10 acquires the uploaded file group Fg, which includes 10 files with a data size of 4 KB each, as storage targets. Next, the client 10 compresses the uploaded file group Fg into one file at a compression rate of 75% ((1) in FIG. 3). That is, the client 10 compresses the uploaded file group Fg into one uploaded file Fa of 30 KB. The client 10 then uploads the compressed uploaded file Fa to the secret sharing storage.

[0033] Next, the distribution server 20 encrypts the upload file Fa, and then splits it into two fragment files F1 and F2 to generate one parity file (fragment file) P1. The size of the fragment files F1 and F2 is 15 KB (box W1 in FIG. 3) because 30 KB (upload file size) is split into two. The size of the parity file P1 is also 15 KB.

[0034] The distribution server 20 distributes and stores the fragment files F1, F2 and the parity file P1 on the disks D1 to D3 of the different storage servers 30-1 to 30-3.

[0035] Here, we will explain the usage of each of the disks D1 to D3 when storing each of the fragment files F1, F2 and the parity file P. If the size of the storage area (sector) of the disk D1 is 4 KB, the size of the fragment file F1 is 15 KB, so that 16 KB of storage area for 4 sectors is used to store the fragment file F1.

[0036] The surplus disk usage is 1 KB (box W2 in Figure 3), which is 16 KB (sector size) minus 15 KB (size of fragment file F1), and the disk usage expansion rate is kept at 1.1 times.

[0037] For comparison, a case where 10 4 KB files are uploaded will be described. FIG 4 is a diagram for explaining an outline of conventional processing.

[0038] As shown in FIG. 4, 10 files, each of which is a 4 KB upload file, are uploaded to the secret sharing storage 30 from the backup unit 11P of the client 10P.

[0039] The distribution server 20 encrypts the uploaded file for each file, and divides the encrypted file into two fragment files F 1p ,F 2p and one parity file (fragment file) P 1p Generate the fragment file F 1p ,F 2p The size of the parity file P is 2KB because 4KB (upload file size) is divided into two (frame W11 in Figure 4). 1p The size is also 2KB.

[0040] Next, the distribution server 20 reads the fragment file F 1p ,F 2p and parity file P 1p are distributed and stored in the disks D1 to D3 of the different storage servers 30-1 to 30-3.

[0041] Distribution server 20 executes the above process for the 10 uploaded files. As a result, 10 2 KB fragment files are stored on each of disks D1 to D3 of storage servers 30-1 to 30-3.

[0042] For example, disk D1 has 4KB sectors and 2KB fragment file F. 1p Therefore, the disk surplus per file is reduced from 4KB (sector size) to 2KB (fragment file size). 1p The result is 2KB, which is the size of the

[0043] For this reason, in the example in Figure 4, 2 KB of storage space per file becomes surplus, and disk usage doubles. In the example in Figure 4, since there are 10 uploaded files, 40 KB of storage space equivalent to 10 sectors is used, and the surplus space swells to 20 KB. As a result of the size of each fragment file being smaller than the sector size, surplus sectors are created for each file, and disk usage swells.

[0044] In contrast, in this embodiment, the facility compresses multiple uploaded files into one file before uploading it to the secret sharing storage, so that the surplus area of ​​the file is efficiently reduced. As a result, in this embodiment, efficient fragment files can be generated. In the example of FIG. 3, the surplus area when storing data on disks D1 to D3 of storage servers 30-1 to 30-3 can be reduced to 1 KB. Therefore, in this embodiment, the disk usage expansion rate can be kept to 1.1 times, and the efficiency can be improved by 1.8 times compared to the case where data is stored without compression.

[0045] Also, in FIG. 3, an example is shown in which the compression rate is 75%, but even when the compression rate is 100%, the size of the fragment files can be increased and stored on disks D1 to D3 of storage servers 30-1 to 30-3. Therefore, it is clear that the surplus space required for storage on disks D1 to D3 of storage servers 30-1 to 30-3 can be reduced compared to when the files are stored without compression.

[0046] [Processing Procedure] Fig. 5 is a sequence diagram showing the processing procedure of the processing method according to the embodiment. As shown in Fig. 5, the client 10 acquires the upload files to be stored (step S1). Then, the client 10 compresses the multiple upload files to be stored into one file at a predetermined compression rate (step S2), and uploads the compressed file to the secret sharing storage 30 (step S3).

[0047] In the secret sharing storage 30, the distribution server 20 encrypts the compressed file and divides it into fragment files (step S4), and distributes and stores each fragment file among a plurality of storage servers 30-1 to 30-3 (steps S5 and S6).

[0048] [Effects of the embodiment] In this manner, in the embodiment, the facility compresses multiple uploaded files into one file and then uploads the files to the secret sharing storage, thereby efficiently reducing the surplus area of ​​the files. As a result, in the embodiment, efficient fragment files can be generated.

[0049] [System configuration, etc.] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be realized in whole or in part by a CPU or GPU and a program analyzed and executed by the CPU or GPU, or can be realized as hardware using wired logic.

[0050] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

[0051] [program] Also, a program can be created in which the processes executed by the client 10, the storage servers 30-1 to 30-3, and the servers 50-1 to 50-3 described in the above embodiment are written in a language executable by a computer. For example, a program can be created in which the processes executed by the client 10, the storage servers 30-1 to 30-3, and the servers 50-1 to 50-3 in the above embodiment are written in a language executable by a computer. In this case, the computer executes the program, thereby obtaining the same effect as in the above embodiment. Furthermore, such a program can be recorded in a computer-readable recording medium, and the program recorded in the recording medium can be read and executed by a computer to realize the same process as in the above embodiment.

[0052] Fig. 6 is a diagram showing a computer that executes a program. As shown in Fig. 6, a computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070, and these components are connected by a bus 1080.

[0053] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012, as exemplified in Fig. 6. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090, as exemplified in Fig. 6. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0054] 6, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the above programs are stored in, for example, the hard disk drive 1090 as program modules in which instructions to be executed by the computer 1000 are written.

[0055] Moreover, the various data described in the above embodiment are stored as program data, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes various processing procedures.

[0056] Note that the program module 1093 and program data 1094 relating to the program are not limited to being stored in the hard disk drive 1090, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via a disk drive or the like. Alternatively, the program module 1093 and program data 1094 relating to the program may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)) and read by the CPU 1020 via the network interface 1070.

[0057] The above-described embodiments and their modifications are included in the technology disclosed in this application, as well as in the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0058] 1 Processing System 10 Clients 11 Backup Department 12 Compression section 30-1~30-3 Storage Server

Claims

1. A processing system having a client that uploads a file to be stored, and a secret sharing storage that distributes and stores fragment files obtained by dividing the uploaded file in a plurality of storage servers, The client: A backup section that compresses multiple files to be stored into one file at a specified compression rate and then uploads the compressed file having The secret sharing storage includes: the plurality of storage servers; a distribution server that encrypts the compressed file, divides the file into fragment files, and distributes and stores each fragment file in the plurality of storage servers; A processing system comprising:

2. 2. The processing system according to claim 1, wherein the backup unit compresses the plurality of files to be stored into one file at a compression rate of 100% or less.

3. The processing system described in claim 1, characterized in that the distributed server encrypts the compressed file, divides it into k fragment files, generates m parity files of the same size as the fragment files, and stores the fragment files and the parity files in a distributed manner on disks of different storage servers.

4. A processing method executed by a processing system having a client that uploads a file to be stored, a distribution server that fragments the uploaded file, and a secret sharing storage having a plurality of storage servers that distribute and store the fragmented files, comprising: a step of the client compressing the plurality of files to be stored into one file at a predetermined compression rate and then uploading the compressed file; a step of the distribution server encrypting the compressed file, dividing the file into fragment files, and distributively storing each fragment file in the plurality of storage servers; A processing method comprising the steps of:

5. A processing program for causing a computer to execute the method, On the client computer, compressing the plurality of files to be stored into one file at a predetermined compression rate and then uploading the compressed file; Run the command, Computers as distributed servers a step of encrypting the compressed file, dividing the file into fragment files, and distributing and storing each fragment file in a plurality of storage servers; A processing program characterized by executing the above.

Citation Information

Patent Citations

  • Data distribution device, distributed data conversion device, and data restoration device

    JP2013140310A