Method and system for storing genomic data in a file structure containing protective metadata

Abstract

1. A method (100) comprising: receiving (120) a genomic dataset including genomic data of one or more of a plurality of fields or attributes of different data types; generating (130) a protected metadata structure for the genomic dataset including one or more of: (i) specifications for selective encryption of one or more data components and regions of the genomic data in an annotation table; (ii) specifications for selective signing of one or more data components and regions of the genomic data in the annotation table; (iii) user key information; and (iv) an access control policy; compressing (140) the genomic data and the protected metadata structure using one or more compression algorithms to generate a compressed genomic dataset and a compressed protected metadata structure; and storing (150) the compressed genomic dataset and the compressed protected metadata structure in a container data structure in memory.

Description

Technical Field

[0001]

[0001] This disclosure generally relates to methods and systems for storing large amounts of data together with associated metadata, and more particularly to the compression and storage of genomic data.

Background Art

[0002]

[0002] High-throughput genomic sequencing (HTS) is an important tool in genomics research and has numerous applications for discovery, diagnosis, and other methodologies. In many cases, the results of HTS are further processed to obtain a higher level of information. Generally, as a secondary analysis, a process of aggregating information estimated from single reads and their alignment to the genome into more complex results is known. In most HTS-based biological research, the output of the secondary analysis is usually represented as various types of annotations related to one or more genomic intervals of a reference sequence.

[0003]

[0003] In fact, in biological research, typically, genomic annotation data such as mapping statistics, quantitative browser tracks, variants, genomic function annotations, gene expression data, and Hi-C contact matrices are created. These diverse downstream genomic data are currently represented in various formats such as VCF, BED, WIG, and many more. These formats typically include a roughly defined semantics, which causes, among other numerous problems, problems such as interoperability, frequent need for conversion between formats, difficulty in visualizing multimodal data, and complex information exchange.

[0004]

[0004] Furthermore, the lack of a single format to accommodate diverse types of genome annotation data has hindered work on compression algorithms, leading to the widespread use of general compression algorithms that do not perform optimally. These algorithms do not take advantage of the fact that annotation data typically contains multiple fields (attributes) with different statistical properties, but instead compress them together. Moreover, these prior art storage mechanisms lack functional metadata to support advanced features such as selective encryption of sensitive information and digital signatures of said information. [Overview of the project] [Problems that the invention aims to solve]

[0005]

[0005] A unified data format for efficient representation and compression of diverse genome annotation data for file storage and data transfer remains necessary. Furthermore, metadata needs to be associated with and stored in order to enable selective encryption of sensitive information and digital signing of information. [Means for solving the problem]

[0006]

[0006] This disclosure relates to the inventive method and system for storing genomic data, along with functional metadata integrated into the file structure, within a data structure including a file structure. Various embodiments and implementations herein relate to a system or method for receiving genomic data and storing the genomic data within a data structure including a file structure. Genomic data may be any of a wide variety of genomic data types, including, but not limited to, genomic variants (VCFs), gene expressions, genomic functional annotations (e.g., BED, GTF, GFF, GFF3, GenBank, etc.), quantitative browser tracks (e.g., Wig, BigWig, BedGraph, etc.), and / or chromosome conformation captures (e.g., HiC files, etc.). A protective metadata structure is generated for the genomic dataset. The protective metadata structure includes one or more of the following: (i) specifications for the selective encryption of one or more data components and regions of the genomic data within the annotation table data; (ii) specifications for the selective signing of one or more data components and regions of the genomic data within the annotation table data; (iii) user key information; and (iv) access control policies. The genomic data and protective metadata structure are compressed using a compression algorithm, and the compressed data is then stored in a container data structure in memory.

[0007]

[0007] Generally, a method is provided for storing genome data in a data structure including a file structure. The method includes the steps of: receiving a genome dataset containing one or more genome data from a plurality of fields or attributes of different data types; generating a protective metadata structure for the genome dataset, which includes one or more of the following: (i) specifications for selective encryption of one or more data components and regions of the genome data in an annotation table; (ii) specifications for selective signing of one or more data components and regions of the genome data in an annotation table; (iii) user key information; and (iv) access control policies; compressing the genome data and protective metadata structure using one or more compression algorithms to generate a compressed genome dataset and a compressed protective metadata structure; and storing the compressed genome dataset and the compressed protective metadata structure in a container data structure in memory.

[0008]

[0008] According to one embodiment, the method further comprises the steps of encrypting or decrypting individual data components and payload blocks of genome data, and optionally compressing or decompressing them to facilitate random access.

[0009]

[0009] According to one embodiment, the method further comprises the step of selecting one or more data components or payload blocks of a particular region of genomic data in an annotation table, which include one or more identifying information from a data component ID, a matrix index range, a genomic coordinate range, and a sample ID, for the application of encryption and / or digital signature.

[0010]

[0010] According to one embodiment, the method further comprises the steps of detecting duplicates between selected data components or regions in an annotation table, notifying the user of the detected duplicates and / or automatically removing the detected duplicates from the selected data components or regions to ensure that each data component or payload block is not encrypted more than once.

[0011]

[0011] According to one embodiment, the method further comprises the steps of ordering, concatenating, and serializing selected data components and payload blocks in an annotation table for generating / verifying a digital signature.

[0012]

[0012] According to one embodiment, the method further comprises the steps of extracting all digital signatures generated for selected data components and / or areas in an annotation table, obtaining a verification key and verifying each of the extracted digital signatures, and presenting the signature information and optionally providing the scope, signer ID, and signing date and time along with the signature information.

[0013]

[0013] According to one embodiment, the method further comprises the steps of: identifying any selected data components and / or areas in an annotation table to which encryption has been applied; authenticating a user requesting data retrieval and verifying whether the user has sufficient access rights if any part of the selected data components and / or areas is encrypted; obtaining a decryption key and decrypting each of the encrypted data components and / or areas; optionally performing data integrity verification; and presenting the retrieved data and any associated signatures and / or verification results.

[0014]

[0014] According to one embodiment, the method further comprises the steps of: identifying any previously encrypted and / or signed updated data components and / or regions; reapplying encryption to the previously encrypted updated data; generating a new digital signature for the updated data to replace the old digital signature; compressing the updated data components and / or payload blocks as necessary; and storing the updated data and / or digital signature in an annotation table.

[0015]

[0015] According to one embodiment, the method further includes the step of locking selected data components and payload blocks protected by a digital signature, allowing only authenticated users who have sufficient access rights to update the protected data.

[0016]

[0016] According to a second aspect, a system is provided for storing genome data in a data structure including a file structure. The system includes a genome dataset containing one or more genome data from a plurality of fields or attributes of different data types, a data structure configured to store genome data, a data compression algorithm, and a processor configured to generate a protective metadata structure for the genome dataset, which includes one or more of the following: (i) (1) specifications for selective encryption of one or more data components and regions of the genome data in an annotation table, (2) specifications for selective signing of one or more data components and regions of the genome data in an annotation table, (3) user key information, and (4) access control policies; (ii) compress the genome data and protective metadata structure using the data compression algorithm to generate a compressed genome dataset and a compressed protective metadata structure; and (iii) store the compressed genome dataset and the compressed protective metadata structure in the data structure.

[0017]

[0017] In various implementations, the processor or controller is associated with one or more storage media (collectively referred to herein as “memory,” such as volatile and non-volatile computer memories including RAM, PROM, EPROM, and EEPROM, floppy disks, compact disks, optical disks, magnetic tapes, etc.). In some implementations, the storage media are encoded by one or more programs that, when executed on one or more processors and / or controllers, perform at least some of the functions described herein. Various storage media are either fixed within the processor or controller, or portable so that one or more programs stored on the storage media can be loaded into the processor or controller to perform the various embodiments described herein. The terms “program” or “computer program” are used herein in a general sense to refer to any type of computer code (e.g., software or microcode) that can be used to program one or more processors or controllers.

[0018]

[0018] It should be understood that all combinations of the concepts described above and the further concepts detailed below (provided that such concepts are not mutually contradictory) are intended to be part of the subject matter of the invention disclosed herein. In particular, all combinations of claimed subject matter listed at the end of this disclosure are intended to be part of the subject matter of the invention disclosed herein. It should also be understood that terms expressly adopted herein, which are also found in any disclosure incorporated by reference, should be given meanings that are most consistent with the specific concepts disclosed herein.

[0019]

[0019] These and other aspects of various embodiments will become clear and clarified by referring to the embodiments described below.

[0020]

[0020] In the drawings, like reference numerals generally refer to the same parts throughout different figures. Also, the drawings are not necessarily to scale, and emphasis is placed on illustrating the principles of the various embodiments.

Brief Description of the Drawings

[0021] [Figure 1]

[0021] FIG. is a flowchart of a method for packaging genomic data according to one embodiment. [Figure 2]

[0022] FIG. is a schematic diagram of a genomic data storage system according to one embodiment. [Figure 3]

[0023] FIG. is a schematic diagram of a data file structure according to one embodiment. [Figure 4]

[0024] FIG. is a flowchart of a method for data encryption / decryption according to one embodiment. [Figure 5]

[0025] FIG. is a flowchart of a method for data integrity verification according to one embodiment. [Figure 6]

[0026] FIG. is a flowchart of a method for data acquisition according to one embodiment. [Figure 7]

[0027] FIG. is a flowchart of a method for data update according to one embodiment.

Modes for Carrying Out the Invention

[0022]

[0028] This disclosure describes various embodiments of systems and methods for storing genomic data and protective metadata within a data structure. The applicant recognizes and understands that it is beneficial to provide methods and systems that include a unified data format for efficiently representing and compressing diverse genomic annotation data. The genomic data storage system receives a genomic dataset containing one or more fields or attributes of different data types. The system generates a protective metadata structure for the genomic dataset that includes one or more of the following: (i) specifications for selective encryption of one or more data components and regions of the genomic data in the annotation table; (ii) specifications for selective signing of one or more data components and regions of the genomic data in the annotation table; (iii) user key information; and (iv) access control policies. The genomic data and protective metadata are compressed using a compression algorithm, and the compressed data is then stored in a container data structure memory.

[0023]

[0029] By extending metadata and security frameworks using stored genomic data, advanced functionality is provided to enhance data management and analysis, which is particularly important for large-scale collaborative genomic studies. For example, the methods and systems described or assumed herein allow selective encryption and digital signatures to be applied only to sensitive information determined by the user, thereby reducing the computational load and processing overhead required to enhance data security and privacy. Another important advantage of integrating functional metadata into the overall file format is that such essential metadata is organized and readily available as part of the data file, and is not easily lost or misplaced during data transfer and migration. Furthermore, stronger data protection is achieved because data security and privacy are designed within the file format rather than being provided through the storage platform or file management software. In addition, because the syntax and processing mechanisms of information and protective metadata are clearly defined in the standard, users can expect consistent or similar functionality and performance from any compliant software.

[0024]

[0030] Referring to Figure 1, this is a flowchart of a method 100 for storing genomic data and associated protective metadata in a data structure including a file structure using a genomic data storage system in one embodiment. It should be understood that the method described in relation to the figure is provided merely as an example and is not intended to limit the scope of this disclosure. The genomic data storage system may be any of the systems described or assumed herein. The genomic data storage system may be a single system or several different systems.

[0025]

[0031] In step 110 of the method, a genome data storage system is provided. Referring to one embodiment of the genome data storage system 200 illustrated in Figure 2, for example, the system comprises one or more of the following, interconnected via one or more system buses 212: a processor 220, memory 230, user interface 240, communication interface 250, and storage device 260. It is understood that Figure 2 constitutes an abstraction in some respects, and that the actual configuration of the components of system 200 may be more complex and differ from that illustrated. Furthermore, the genome data storage system 200 may be any of the systems described or assumed herein. Other elements and components of the genome data storage system 200 are disclosed and / or assumed elsewhere in this specification.

[0026]

[0032] In step 120 of the method, the genome data storage system receives a genome dataset containing genome data. The genome data may be any of a wide variety of genome data types, including, but not limited to, genome variants (VCFs), gene expression, genome function annotations (e.g., BED, GTF, GFF, GFF3, GenBank, etc.), quantitative browser tracks (e.g., Wig, BigWig, BedGraph, etc.), and / or chromosome conformation captures (e.g., HiC files, etc.). The received genome dataset contains one type of genome data or multiple fields or attributes of different data types. The received genome dataset is immediately available for subsequent steps of the method described or assumed herein, or is stored for future use by this method and other methods. Accordingly, the system has or communicates with a local or remote data storage device configured to store the genome dataset.

[0027]

[0033] In step 130 of the method, the genome data storage system generates a protective metadata structure for the genome dataset. The protective metadata structure is configured to enable a wide range of functionalities, including, among many other functionalities, one or more of the following: support for selective encryption and digital signing. Selective encryption (and therefore decryption) can be performed separately on subsets of the gene data, thus improving the speed of random access. Selective signing may include digital signatures and edit locks on data components and / or selective portions of the genome data.

[0028]

[0034] According to one embodiment, the protective metadata structure for a genome dataset includes one or more of the following: (i) specifications for selective encryption of one or more data components and regions of the genome data in the annotation table; (ii) specifications for selective signing of one or more data components and regions of the genome data in the annotation table; (iii) user key information; and (iv) access control policies.

[0029]

[0035] The generated protected metadata structure is immediately available for subsequent steps of the methods described or assumed herein, or is stored for future use by this method and other methods. Therefore, the system includes, or communicates with, a local or remote data storage device configured to store genome datasets and annotation tables. In particular, some or all of the protected metadata structure is encrypted as described or assumed herein.

[0030]

[0036] In step 122 of the optional method, the system receives user input through the user interface of the genome data storage system, for example. The input may be one or more user preferences, such as encryption selection and / or digital signature. For example, the user may specify genome data and / or annotation table data for encryption. The user may also provide digital signature information for some or all of the genome data and / or annotation table data.

[0031]

[0037] In step 140 of the method, the genome data storage system compresses the genome data and protective metadata structure using a compression algorithm to generate a compressed genome dataset. The compression algorithm may be any algorithm, method, or process for data transformation and compression, including but not limited to the compression algorithms and methods described or assumed herein. The compression algorithm may be a single compression algorithm or multiple compression algorithms.

[0032]

[0038] In step 150 of the method, the compressed genome dataset is stored in a container data structure, such as an annotation table in memory, along with a protective metadata structure. The memory is any memory capable of receiving and storing the compressed data. The memory is associated with a genome data storage system or communicates directly or indirectly with a genome data storage system via wired and / or wireless means. The memory is local memory or remote memory. The memory is cloud-based memory. Many other storage mechanisms and devices are possible.

[0033]

[0039] Accordingly, according to one embodiment, the system includes protective metadata extended to support selective encryption and signing of annotation table data. Thus, the system includes a URI structure for referencing specific data fields and block / chunk payloads for data protection. The system may also include a centralized storage device for encryption and signing parameters and data to improve the efficiency of data security enforcement.

[0034]

[0040] According to one embodiment of a genome data storage system, the system processes an incoming genome dataset, extracts multiple attributes from the genome dataset, and then divides each attribute into multiple chunks of a predetermined size. The chunks are indexed with a master index of a data structure, each having lookup data for each of the multiple chunks. Each chunk is compressed individually using a compression algorithm and stored in an allocated location in the data structure's chunk structure data. Thus, the data structure is configured such that each of the multiple chunks can be decompressed individually. Furthermore, the data structure is configured such that the genome data type, attributes, chunk size, and compression algorithm can be changed without altering the file structure of the data structure.

[0035]

[0041] According to one embodiment, the system enables selective encryption and decryption. Selective encryption / decryption can be performed separately for each block / chunk payload, thereby improving the speed of random access. The system can use an encryption flag in the block header to indicate whether or not a block payload is encrypted. The system can ensure that each block payload is not encrypted more than once by verifying the URI in the EncryptionParameters element. To access the data in the annotation table where encryption parameters are defined, it is necessary to identify all encrypted areas by resolving the URI in the EncryptionParameters element. Any encrypted data accessed should be decrypted for presentation.

[0036]

[0042] According to one embodiment, the system enables selective digital signing. Selective digital signing may include rules for concatenating specific data fields and block / chunk payloads to generate a digital signature. A region may be protected by multiple digital signatures by different users. A boolean editLock attribute can be added to SignatureParameters to indicate whether the signed data should be locked to prevent editing. If editLock is on, editing of the signed data is permitted only to authorized users. After changes are made, the old signature should be discarded or regenerated. To update the data in the annotation table where the signature parameters are defined, it is necessary to identify all signed regions protected by the edit lock by resolving the URI of the EncryptionParameters element.

[0037]

[0043] According to one embodiment, in order to enhance the uniqueness of the signature value and prevent its reuse, each metadata component of the entire XML document may be encrypted and signed, including the table ID, table name, table version, last modified user ID, and last modified time.

[0038]

[0044] Selective encryption / decryption

[0045] Referring to Figure 4, one embodiment of the method 400 for selective encryption and / or decryption of genome data or other data within a genome dataset, and / or digital signing. According to one embodiment, the annotation table is constructed to enable selective encryption and / or decryption of gene data, components, or annotation access unit payloads within the annotation table.

[0039]

[0046] In step 410 of the method, the genome data storage system receives identification information for data to be encrypted or decrypted and / or digitally signed. The data identified for encryption, decryption and / or digital signing may be a protected metadata structure or any data within the genome dataset. The identified data may include individual data components and / or payload blocks of the genome data, which can greatly facilitate access to that data. The identification information may be received from a user of the genome data storage system and may be received through the system's user interface. Thus, the system facilitates data selection for protecting data security and / or privacy through the user interface.

[0040]

[0047] According to one embodiment, one or more data components or payload blocks of specific regions of genomic data within an annotation table, which are identified for encryption, decryption, and / or digital signing, are identified by specifying one or more combinations of one or more of the following: data component ID, matrix index range, genomic coordinate range, and sample ID. Many other methods are possible for identifying data.

[0041]

[0048] In step 420 of the method, the genome data storage system analyzes the data identification information to determine whether there are duplicates between selected data components or regions in the annotation table. If duplicates are found, in step 430 the system notifies the user of the found duplicates and / or automatically removes the found duplicates from the selected data components or regions to ensure that each data component or payload block is not encrypted more than once. If no duplicates are found, the method proceeds to the next step. According to one embodiment, the user is notified via the system's user interface.

[0042]

[0049] In step 440 of the method, the selected data components and payload blocks in the annotation table are ordered, concatenated, and / or serialized for the generation / verification of a digital signature. The selected data may be ordered, concatenated, and / or serialized using any method suitable for preparing the data for the digital signature.

[0043]

[0050] In step 450 of the method, the genome data storage system encrypts, decrypts, or digitally signs data within a protected metadata structure or genome dataset identified for encryption, decryption, and / or digital signing. The identified data includes individual data components and / or payload blocks of the genome data. According to one embodiment, the system optionally compresses or decompresses the data while encrypting, decrypting, and / or digitally signing it. The encrypted, decrypted, and / or digitally signed data may then be stored in memory in step 460 of the method.

[0044]

[0051] Data Integrity Verification

[0052] Referring to Figure 5, a method 500 for data integrity verification in one embodiment is shown. According to one embodiment, the annotation table is constructed to allow selective integrity verification of gene data, components, or annotation access unit payloads within the annotation table.

[0045]

[0053] In step 510 of the method, the genome data storage system receives identification information for data integrity verification. The data identified for integrity verification may be a protected metadata structure or any data within the genome dataset. The identified data may include individual data components and / or payload blocks of the genome data. The identification information can be received from a user of the genome data storage system and can be received through the system's user interface. Thus, the system facilitates the selection of data for integrity verification through the user interface.

[0046]

[0054] In step 520 of the method, the genome data storage system identifies and extracts all digital signatures generated for selected data components and / or regions in the annotation table.

[0047]

[0055] In step 530 of the method, the system obtains a verification key from one of several sources and verifies each of the identified and extracted digital signatures. Verification using the verification key can be performed using one of a wide variety of methods.

[0048]

[0056] In step 540 of the method, the system provides the user with signature information through the user interface of the genome data storage system, etc. The signature information includes, but is not limited to, the scope, signer ID and signing date and time, and other information, as well as any information associated with the digital signature and data.

[0049]

[0057] Data acquisition

[0058] Referring to Figure 6, there is a method 600 for data acquisition in one embodiment. According to one embodiment, the annotation table is constructed to allow selective acquisition of data which may be gene data, components, or annotation access unit payloads within the annotation table.

[0050]

[0059] In step 610 of the method, the genome data storage system receives identification information for data to be retrieved. The data to be retrieved may be a protected metadata structure or any data within the genome dataset. The data to be retrieved may include individual data components and / or payload blocks of the genome data. The identification information may be received from a user of the genome data storage system and may be received through the system's user interface. Thus, the system facilitates the selection of data to be retrieved through the user interface.

[0051]

[0060] In step 620 of the method, the genome data storage system examines selected data components and / or regions in the annotation table to identify such data that is encrypted.

[0052]

[0061] In step 630 of the method, if any of the selected data is encrypted, the genome data storage system authenticates the user requesting the data to determine whether the user has sufficient access privileges to access the encrypted data.

[0053]

[0062] In step 640 of the method, the genome data storage system obtains a decryption key and decrypts each of the encrypted data components and / or regions. Optionally, the system performs data integrity verification during or after encryption as described or assumed herein.

[0054]

[0063] In step 650 of the method, the genome data storage system provides the acquired data to the user through the genome data storage system's user interface, etc. The acquired data is accompanied by any relevant signatures and / or verification results, among other possible data or information.

[0055]

[0064] Data update

[0065] Referring to Figure 7, there is a method 700 for updating data in a stored genome data file in one embodiment. According to one embodiment, the annotation table is constructed to allow selective updating of data which may be gene data, components, or annotation access unit payloads within the annotation table.

[0056]

[0066] In step 710 of the method, the genome data storage system receives identification information for data to be updated. The data to be updated may be either a protected metadata structure or data within a genome dataset. The data to be updated may include individual data components and / or payload blocks of the genome data. The identification information can be received from a user of the genome data storage system and can be received through the system's user interface. Thus, the system facilitates the selection of data to be updated via the user interface.

[0057]

[0067] In step 720 of the method, the genome data storage system examines selected data components and / or regions in the annotation table to identify such data that is encrypted or digitally signed.

[0058]

[0068] In step 722 of the optional method, for any data identified as locked to prevent editing, the genome data storage system authenticates the user and determines whether the user has sufficient access privileges to that data.

[0059]

[0069] In step 730 of the method, the genome data storage system reapplies encryption to the previously encrypted updated data and / or generates a new digital signature for the updated data to replace the old digital signature. Thus, optionally, the user or system may choose to lock selected data components and payload blocks protected by the digital signature, allowing only authenticated users with sufficient access privileges to update the protected data.

[0060]

[0070] In step 740 of the method, the genome data storage system compresses the updated data components and / or payload blocks. In step 750 of the method, the system stores the updated data and / or digital signatures in an annotation table.

[0061]

[0071] Genome data storage structure and data format

[0072] The genome data storage structure in which the received genome data and associated annotation tables are packaged employs one of a wide variety of formats. While a specific format is described below with reference to one embodiment, it should be understood that this format is merely an example of a data structure used by the genome data storage systems described or assumed herein. Similarly, the data format within the genome data storage structure employs one of a wide variety of formats. While a specific format is described below with reference to one embodiment, it should be understood that this format is merely an example of a data format used by the genome data storage systems described or assumed herein.

[0062]

[0073] Referring to Figure 3, this is one embodiment of the top-level container hierarchy for genome datasets and associated annotation tables. In this format, top-level container boxes called files, dataset groups, and datasets are used. A dataset contains annotation tables (atcn) that hold data. In Figure 3, all container boxes, including dataset groups (dgcn), datasets (dtcn), annotation tables (atcn), attribute groups (agcn), and annotation access units (aauc), can exist in multiple instances. For example, the "..." symbol after a box indicates that there may be multiple instances of that particular box structure.

[0063]

[0074] According to one embodiment, information and protection metadata can be stored in annotation table metadata and annotation table protection data structures, respectively. These are contained within the gen_info box in KLV(Key, Length, Value) format using the following syntax, although other syntaxes are also possible.

number

[0064]

[0075] In one embodiment, the key field specifies the type of data structure using a four-character code, which is "atmd" for annotation table metadata and "atpr" for annotation table protection. The length field specifies the number of bytes that make up the entire gen_info structure, including all three fields: key, length, and value. The syntax for the value fields of annotation table metadata and annotation table protection is defined in Table 1 and Table 2, respectively.

[0065]

[0076] Table 1 - Syntax for Annotation Table Metadata

[0066] [Table 1]

[0067]

[0077] Table 2 - Syntax for Annotation Table Protection

[0068] [Table 2]

[0069]

[0078] Annotation Table Protection Metadata

[0079] According to one embodiment, an annotation table protection gen_info box with the key "atpr" holds parameters for data protection, including encryption and digital signatures, and rules for access control applied to the information metadata and block payload within the annotation table. This is in the form of an XML document with a root element "AnnotationTableProtection". The document is compressed by the LZMA algorithm, and the compressed bytes are stored in the AT_protection_value() element of the gen_info box. The output of the decoding process is an XML document with a root node AnnotationTableProtection consisting of the following four main components:

[0070]

[0080] (1) Any number of "KeyTransportAES" elements. Each defines a key identified by the keyName element and its key transfer parameters. Details regarding key transfer parameters and mechanisms can be found in section 7.2.4 of ISO / LEC23092-3.

[0071]

[0081] (2) Any number of "EncryptionParameters" elements, each having a required encryptedLocations attribute that specifies a URI referring to a data target and the associated encryption algorithm and key. In particular, the following rules apply: (i) IV elements shall be present, (ii) TAG elements shall not be present, and (iii) configurationID attributes shall be present. If an access unit belongs to a set resolved by a URI, the AccessUnitEncryptionParameters element of the access unit protection shall include one or more wrappedKey elements, each referring to a configurationID. The keys associated with EncryptionParameters shall be able to unwrap the corresponding wrappedKey.

[0072]

[0082] (3) Any number of "SignatureParameters" elements in SignatureType. Each holds a signature value and its associated parameters, and includes a signing method and one or more reference elements. Each of the one or more reference elements has a URI attribute to specify a URI that refers to the data target. Detach signing, envelope signing, and envelope signing are supported. If decryption is required, signature verification is performed before decryption.

[0073]

[0083] (4) A “privacy_rules” element containing a valid access control policy specified in accordance with the OASIS Extended Access Control Markup Language (XACML) version 3.0 specification. The privacy rules specify who can perform a given action and under what conditions. Further details can be found in section 7.3 of ISO / IEC 23092-3.

[0074]

[0084] Data container protection metadata has a limited scope of application. Generally, its parameters are used to encrypt or sign information metadata at the same level or the container protection metadata at the immediately lower level, and its policy rules are used to control access to any resources within the container. In the case of annotation table protection, the policy rules also manage the protection of block payloads within the enclosed annotation access unit.

[0075]

[0085] According to one embodiment, among the many targets, there are at least three types of data protection targets, including, but not limited to, (1) specific elements in an XML document for metadata and protected gen_info boxes, (2) data fields in metadata and protected gen_info boxes, and (3) block payloads in annotation access units containing data from selected areas of an annotation table.

[0076]

[0086] For the first type of target containing specific XML elements, the syntax and processing rules for data encryption and digital signing recommended by the W3C Working Group apply directly. In the case of encryption, the inclusion of an element of type EncryptedData provides the option to provide a specific plaintext value as encrypted content. In this scenario, the mechanism for transmitting knowledge about the key is established through a separate channel. In the case of digital signing, each data object to be signed may include a signature element defined in the xmldsig schema.

[0077]

[0087] For the second and third types of targets, including data fields in the gen_info box and block payloads in annotation access units, the XML elements KeyTransportAES, EncryptionParameters, and SignatureParameters within the protected metadata are used, along with essentially the same syntax as described in ISO / IEC 23092-3. Details regarding key acquisition and encryption parameters, in particular, can be found in sections 7.2.4 and 7.2.5 of ISO / IEC 23092-3. However, the data encryption and signature framework has several aspects that need to be extended or modified for annotation table data, namely (1) URI structures for identifying specific data fields and block payloads to be protected, (2) encryption and decryption processes for block payloads in annotation access units, and (3) rules for concatenating specific data fields and block payloads to generate digital signatures, which will be discussed in subsequent sections.

[0078]

[0088] According to one embodiment, encryption and signing of annotation table protection metadata can be enabled by specifying the URI and associated parameters of the annotation table protection metadata within the protection metadata of the containing dataset, in order to protect the confidentiality and integrity of annotation table protection metadata containing sensitive security information. With appropriate access control settings, only authenticated and authorized users can read, update, or sign on to the protection metadata. If signing is possible, only the most recent signature is retained. Furthermore, to prevent the protection metadata and its corresponding signature from being replaced by previous older versions, optional string-type LastUpdateUser elements and datetime-type LastUpdateTime elements can be included in the XML document for encryption and signing, along with the corresponding update records, which include the last update user and last update time entered in the secure access history within the annotation table metadata. Similarly, to ensure that the protection metadata can only be used for tables with specific IDs, names, and versions, optional string-type TableID elements, TableName elements, and TableVersion elements can also be included. In this case, the protection metadata must be updated with appropriate encryption and signing whenever the table ID or version changes.

[0079]

[0089] Structure of a URI (Uniform Resource Identifier)

[0090] The URI structure is defined to reference a specific gen_info box component or annotation access unit payload within the annotation table in order to enable selective encryption or signing. The following are some general rules regarding URI syntax: (i) Text within curly braces, including the braces themselves, shall be replaced with an alphanumeric sequence conforming to the description of each entry. (ii) In the semantic table, parameters marked with an asterisk (*) are mandatory, and otherwise optional. (iii) Fields as optional may be left blank if not used in the selection of a target, i.e., the target covers all values ​​of the field. (iv) URIs may be shortened by removing redundant trailing fields and slashes.

[0080]

[0091] The keys and parameters for encrypting and signing the bytes of the element AT_protection_value() within the annotation table protection may be specified in the protection metadata of the higher-level dataset container. Table 3 contains the URI structure for this purpose.

[0081]

[0092] Table 3.ann_table / {ann_table_id} / protection

[0082] [Table 3]

[0083]

[0093] As shown in Table 4, annotation table protection can use the following URI structure to reference a specific data field in the metadata gen_info box within the same annotation table.

[0084]

[0094] Table4.metadata / {md_fields}

[0085] [Table 4]

[0086]

[0095] Note that in the case of metadata field encryption, each field is encrypted separately using its associated encryption parameters, and the original bytes are replaced with ciphertext. In the case of digital signatures, the bytes of the selected fields are concatenated in the same order as defined in the annotation table metadata syntax, and the concatenated bytes are signed. The resulting signature is then stored in the protection box as an XML signature element.

[0087]

[0096] Annotation table protection allows the use of a URI structure within Table 5 to reference specific regions of an annotation table to which data protection is applied. The URI can correspond to any block payload within an annotation access unit that overlaps with the target region, and the block payload can be specified by a combination of genome coordinates, row / column index, sample ID, or attribute values.

[0088]

[0097] Table 5.AT_region / {AG_classes} / {range_type_1}={range_1} / {range_type_2}={range_2} / desc_ids={desc_IDs} / attr_ids={attr_IDs}

[0089] [Table 5] TIFF0007846390000007.tif255168TIFF0007846390000008.tif46170

[0090]

[0098] Assuming a two-dimensional annotation table, such as a mutation detection file, with genome coordinates in the rows and sample IDs in the columns, two examples of URI structures and the targets they represent are shown below.

[0091]

[0099] (1) AT_region / 0 / range_genome=chrl#chr2:1-100000 / range_attr:2:l=“Sample1”-“Sample10” refers to a block payload of all descriptors and attributes that (i) belong to the primary attribute group of class 0, (ii) contain data for the genomic region of chromosome 1 or the first 100,000 nucleotides of chromosome 2, and (iii) correspond to the column between “Sample1” and “Sample10” as defined in the column-related attribute (AG_class=2) of ID1 in the annotation table.

[0092]

[0100] (2) AT_region / all / range_row_idx=10000-20000 / range_col_idx:l-10 / attr_ids=1-5 refers to the block payload of all descriptors and attributes with IDs 1 through 5, including the data within the rectangular region enclosed by rows 10,000 to 20,000 and columns 1 to 10 in the data of the primary attribute group.

[0093]

[0101] According to one embodiment, in the case of encryption of block payloads within an annotation access unit, each block payload referenced by the URI is encrypted separately using the associated encryption parameters, and the original bytes are replaced with ciphertext. In the case of digital signatures, signing is performed on the concatenated bytes of the referenced block payloads, and the resulting signature is then stored in a protective box as an XML signature element.

[0094]

[0102] Selective encryption and decryption

[0103] According to one embodiment, the following is the encryption / decryption process when the value of the encryptedLocations element of the XML EncryptionParameters element matches a URI structure starting with "AT_region" to refer to a target region in the annotation table.

[0095]

[0104] 1. Use the pre-calculated index data of the annotation table index to find tiles that overlap with the target area.

[0096]

[0105] 2. Identify the location of the corresponding block payload within the annotation access unit.

[0097]

[0106] 3. From the EncryptionParameters element, obtain (i) the key and (ii) the configurationID.

[0098]

[0107] 4. From the AccessUnitEncryptionParameters element present in the relevant Annotation Access Unit (AAU) protection box, obtain (i) the cipher (possible values ​​are listed in Table 14 of ISO / IEC 23092-3), (ii) the wrappedKey instance matching the configurationID (obtained in the previous step), (iii) auinIV if the AAU contains an AAU information box, (iv) auinTAG if the AAU contains an AAU information box and the cipher uses GCM mode, (v) aublockIV, and (vi) aublockTAG if the cipher uses GCM mode.

[0099]

[0108] 5. Encrypt / decrypt each block payload identified in step 2 individually using the wrappedKey instance (obtained in the previous step) associated with the attribute / descriptor ID (in tile adjacency mode) or tile index (in attribute adjacency mode) that uniquely identifies the block payload within the AAU.

[0100]

[0109] 6. In both the CTR and GCM encryption modes supported by the framework, the ciphertext and plaintext lengths are the same. Therefore, if encrypted / decrypted data is stored, the encrypted / decrypted data can simply replace the original bytes in the block payload. Accordingly, the encryption flag in the block header should also be updated (0 for plaintext, 1 for ciphertext).

[0101]

[0110] Measures can be taken to ensure that each block payload is not encrypted more than once. When a new set of encryption parameters is set, the URI referencing the target region should be checked against the URI of any existing EncryptionParameters element. If there are overlapping target regions, check whether the same key was used to encrypt each overlapping block payload. If true, the new set of encryption parameters is valid and encryption can be applied to non-overlapping block payloads within the target region. If different keys are used, the URI of the new target region should be modified, for example, by splitting it into multiple URIs, to avoid overlap with existing encrypted regions. This ensures that encrypted regions are always associated with only one encryption key in the protected metadata.

[0102]

[0111] To access the data in the annotation table where encryption parameters are defined, all encrypted regions must be identified by resolving the URIs within the EncryptionParameters element. If the block payload is found to be in one of the encrypted regions, or if its encryption flag in the block header is set to 1, decryption should be applied to the block payload using the key associated with the URI of the encrypted region.

[0103]

[0112] The encryption / decryption process for specific data fields within the gen_info box is similar. The main difference is that a different URI structure, "metadata / {md_fields}", is used to refer to one or more data fields that will be individually encrypted and replaced with the generated ciphertext. As with block payloads, each data field cannot be encrypted more than once. Measures should be taken to ensure that a data field is referenced by a URI of at most one EncryptionParameters element.

[0104]

[0113] Selective digital signatures

[0114] The generation and verification of digital signatures for data in annotation tables involves a set of rules for concatenating bytes of selected data fields or block payloads referenced by a newly introduced URI structure in the annotation tables. A hash and signature algorithm are applied to the concatenated bytes to generate a digital signature, which is stored in the corresponding SignatureParameters element within the protected metadata.

[0105]

[0115] When signing a set of metadata fields selected by a URI in the format "metadata / {md_fields}", the bytes are concatenated in the same order as the fields are defined in the syntax of the annotation table metadata.

[0106]

[0116] When signing a block payload within an annotation access unit selected by a URI of the format "AT_region / {AG_classes} / {range_type_l}={range_l} / {range_type_2}={range_2} / desc_ids={desc_IDs} / attr_ids={attr_IDs}", the following two rules regarding byte concatenation should apply:

[0107]

[0117] (1) In attribute adjacency mode, each annotation access unit contains the payload of all tiles (blocks of rows and columns in the annotation table) associated with the descriptor / attribute. (i) Within the access unit, the block_payload() bytes of the selected tiles are concatenated in ascending order of their tile indices. For 2D data, if the order of the primary tiles in a column is equal to 1, ordering is done first by the column index and then by the row index; otherwise, ordering is done first by the row index and then by the column index. (ii) Next, within the attribute group, the payload bytes of the access units selected in the previous step are concatenated first in ascending order of their descriptor IDs and then in ascending order of their attribute IDs. (iii) Next, the payload bytes of the attribute groups selected in the previous step are concatenated in ascending order of their attribute group classes.

[0108]

[0118] (2) In tile adjacency mode, each annotation access unit contains the payload of all descriptors / attributes associated with the tile in the annotation table. (i) Within the access unit, the block_payload() bytes of the selected descriptors / attributes are concatenated first in ascending order of their descriptor IDs, and then in ascending order of their attribute IDs. (ii) Then, within the attribute group, the payload bytes of the access units selected in the previous step are concatenated in ascending order of their tile indices. With respect to two-dimensional data, if the order of the primary tiles in a column is equal to 1, ordering is done first by the column index, and then by the row index; otherwise, ordering is done first by the row index, and then by the column index. (iii) Then, the payload bytes of the attribute groups selected in the previous step are concatenated in ascending order of their attribute group class.

[0109]

[0119] Unlike encryption, which can only be applied once to a single region, multiple digital signatures by different users can be applied to the same region. Therefore, it is permissible to have multiple SignatureParameters elements that refer to regions that overlap with each other.

[0110]

[0120] To further protect signed data from unauthorized modification, you can add a boolean editLock attribute to SignatureParameters to indicate whether the signed data should be locked to prevent editing. To implement an edit lock, the signature parameters must be securely stored so that only authorized users can modify them.

[0111]

[0121] When editLock is enabled, editing of signed data is permitted only to authenticated users with a higher level of access than all current signers, or to authenticated users with access to all keys of the current signature. After making changes to the signed and locked data and going through the authentication and authorization process, the signature parameters should be updated by discarding any associated SignatureParameters elements or by regenerating the signature if the keys are available. Authorized users can also create new SignatureParameters elements to ensure data integrity within the selected area, regardless of whether editLock is enabled. When editLock is disabled, signed data and its associated signature parameters can be modified by any user.

[0112]

[0122] To modify the data in an annotation table where signature parameters are defined, it is necessary to identify all signed regions protected by edit locks by resolving the URI of the SignatureParameters element. If a data update is required in either a signed region or a locked region, the data update is only authorized after going through the user authentication and authorization process, the old signature is discarded / regenerated, and a new signature is generated upon user request.

[0113]

[0123] The advantages of this encryption and digital signature framework are numerous. First, it allows data to be protected only in selected areas of the annotation table containing sensitive data, thus reducing the overall processing time for data security enforcement. Second, the framework supports high-speed random access to encrypted data. Since encryption is performed separately for each block payload, only selected block payloads need to be decrypted and decompressed, thus improving the response speed of random access. Third, since all encryption and signature parameters and data are centrally stored in the annotation table protection, the framework improves the efficiency of decryption, integrity verification, and data protection for data access and data editing.

[0114]

[0124] Referring to Figure 2, this is a schematic diagram of a system 200 for storing genome data in one embodiment. System 200 is any of the systems described or assumed herein and comprises any of the components described or assumed herein.

[0115]

[0125] According to one embodiment, the system 200 comprises one or more of a processor 220, memory 230, user interface 240, communication interface 250, and storage device 260, interconnected via one or more system buses 212. In some embodiments, the hardware includes a genome data database 270. It should be understood that Figure 2 constitutes an abstraction in some respects, and that the actual configuration of the components of the system 200 may be more complex and differ from that illustrated.

[0116]

[0126] According to one embodiment, the system 200 includes a processor 220 capable of executing instructions stored in a memory 230 or storage device 260 or processing data to perform, for example, one or more steps of a method. The processor 220 is formed from one or more modules. The processor 220 employs any preferred form, including but not limited to a microprocessor, microcontroller, multiple microcontrollers, circuitry, field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), single processor, or multiple processors.

[0117]

[0127] Memory 230 can employ any preferred form, including non-volatile memory and / or RAM. Memory 230 includes various types of memory, such as L1 cache, L2 cache, or L3 cache, or system memory. Thus, memory 230 includes static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read-only memory (ROM), or other similar memory devices. Among many other things, the memory can store the operating system. RAM is used by the processor to temporarily store data. According to one embodiment, the operating system includes code that controls the operation of one or more components of the system 200 when executed by the processor. In embodiments in which the processor performs one or more of the functions described herein in hardware, it is clear that software described as corresponding to such functionality in other embodiments may be omitted.

[0118]

[0128] The user interface 240 includes one or more devices for enabling communication with the user. The user interface can be any device or system that enables the transmission and / or reception of information, and includes a display, mouse, and / or keyboard for receiving user commands. In some embodiments, the user interface 240 includes a command-line interface or a graphical user interface presented to a remote terminal via the communication interface 250. The user interface may be located together with one or more other components of the system, or located separately from the system, and communicate via wired and / or wireless communication networks.

[0119]

[0129] The communication interface 250 includes one or more devices to enable communication with other hardware devices. For example, the communication interface 250 includes a network interface card (NIC) configured to communicate according to the Ethernet protocol. Furthermore, the communication interface 250 implements a TCP / IP stack for communication according to the TCP / IP protocol. Various alternative or additional hardware or configurations for the communication interface 250 are apparent.

[0120]

[0130] The storage device 260 includes one or more machine-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, or similar storage media. In various embodiments, the storage device 260 stores instructions for execution by the processor 220, or data subject to the operation of the processor 220. For example, the storage device 260 stores an operating system 261 for controlling various operations of the system 200.

[0121]

[0131] It is clear that various pieces of information described as being stored in the storage device 260 may also be stored in memory 230, either additionally or as a substitute. In this respect, memory 230 is also considered to constitute a storage device, and storage device 260 is considered to be memory. Various other configurations are also clear. Furthermore, both memory 230 and storage device 260 are considered to be non-transient machine-readable media. The term non-transient as used herein is understood to include all forms of storage devices, including both volatile and non-volatile memory, but excluding transient signals.

[0122]

[0132] Although System 200 is shown as including one of the components described herein, various components may overlap in various embodiments. For example, Processor 220 includes multiple microprocessors configured to perform the methods described herein individually, or to perform steps or subroutines of the methods described herein so that multiple processors cooperate to achieve the functionality described herein. Furthermore, if one or more components of System 200 are implemented in a cloud computing system, various hardware components belong to separate physical systems. For example, Processor 220 includes a first processor in a first server and a second processor in a second server. Many other variations and configurations are possible.

[0123]

[0133] According to one embodiment, the storage device 260 of the system 200 stores one or more algorithms and / or instructions for performing one or more functions or steps of the methods described or assumed herein. For example, the processor 220 includes one or more of the annotation table generation instructions 262, the compression / decompression instructions 263, and / or the storage instructions 264.

[0124]

[0134] According to one embodiment, the annotation table generation command 262 instructs the system to generate or modify an annotation table in the file structure for the genome dataset. The annotation table is configured to enable a wide variety of functionalities, including, among many other functionalities, one or more of the following: support for selective encryption and digital signatures.

[0125]

[0135] According to one embodiment, the compress / decompress command 263 instructs the system to compress the genome data along with the associated annotation table. The compress algorithm can be any algorithm, method, or process for data compression. The compress command also includes a decompression command for decompressing the stored data.

[0126]

[0136] According to one embodiment, the storage command 264 instructs the system to store the compressed genome data and associated annotation tables. The system has or communicates with a local or remote data storage device configured to store the genome dataset and annotation tables.

[0127]

[0137] Processing genome datasets, generating annotation tables, and compressing / decompressing genome data and annotation tables involves millions or billions of calculations, a task that the human brain cannot perform even with a pen and pencil. In fact, genome datasets alone contain millions of pieces of information. For example, next-generation DNA sequencing data contains hundreds of millions or billions of reads.

[0128]

[0138] Furthermore, the method described herein significantly improves the speed and functionality of genome storage systems. For example, by implementing the method described herein, a genome storage system includes an annotation table having protective metadata configured for (i) selective encryption of annotation table data and / or genome data, and (ii) selective signing of annotation table data and / or genome data. Prior art systems are inferior because they cannot provide this functionality. Therefore, the method described herein significantly improves the speed and functionality of genome storage systems.

[0129]

[0139] All definitions defined and used herein should be understood to govern dictionary definitions, definitions incorporated by reference within documents, and / or the ordinary meaning of the defined terms.

[0130]

[0140] As used herein and in the claims, singular elements should be understood to mean “at least one” unless otherwise specified.

[0131]

[0141] As used herein and in the claims, the phrase “and / or” should be understood to mean “either or both” of the elements thus coordinated, that is, elements that exist conjunctively in some cases and disjunctively in others. Multiple elements enumerated using “and / or,” that is, “one or more” of the elements thus coordinated, should be interpreted in the same way. Other elements other than those specifically identified by the phrase “and / or” may exist as they may, whether related to or unrelated to those specifically identified elements.

[0132]

[0142] As used herein and in the claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in an enumeration, “or” or “and / or” should be interpreted as inclusive, meaning that it includes at least one of several or more of the enumerated elements, and optionally, additional unenumerated items. Only terms otherwise specified, such as “one of” or “exactly one of” or, as used in the claims, “consisting of,” refer to including exactly one of several or the enumerated elements. In general, as used herein, the term “or” should be interpreted as indicating an exclusive choice (i.e., “one or the other, and not both”) when preceded by terms of exclusivity such as “either,” “one of,” “one of,” or “exactly one of.”

[0133]

[0143] As used herein and in the claims, the phrase “at least one” in relation to an enumeration of one or more elements means at least one element selected from any one or more of the elements in the enumeration of elements, but it should be understood that it does not necessarily have to include at least one of every element specifically enumerated in the enumeration of elements, nor does it exclude combinations of elements in the enumeration of elements. This definition allows for the presence of elements other than those specifically identified in the enumeration of elements to which the phrase “at least one” refers.

[0134]

[0144] Furthermore, unless otherwise specified, it should be understood that in any method having two or more steps or actions claimed herein, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions are described.

[0135]

[0145] As with the above specification, in the claims, all transitional phrases such as “equipment,” “includes,” “possess,” “have,” “incorporate,” “involve,” “hold,” and “constitute” should be understood as non-restrictive, meaning they include but are not restrictive. Only the transitional phrases “consist of” and “essentially consist of” are closed or semi-closed transitional phrases, respectively.

[0136]

[0146] While several embodiments of the invention have been described and illustrated herein, those skilled in the art will readily conceive of various other means and / or structures for performing the function and / or obtaining one or more of the results and / or advantages described herein, and such variations and / or modifications will each be considered within the scope of the embodiments of the invention described herein. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials and configurations described herein are illustrative, and that actual parameters, dimensions, materials and / or configurations will depend on the specific one or more uses in which the teachings of the invention are used. Those skilled in the art will be able to recognize or elucidate many equivalents to the specific embodiments of the invention described herein using only routine experimentation. Therefore, it should be understood that the embodiments described herein are presented only as examples, and that embodiments of the invention may be carried out in ways other than those specifically described and claimed, within the scope of the appended claims and their equivalents. Embodiments of the invention in this disclosure cover the individual features, systems, articles, materials, kits and / or methods described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the invention of this disclosure, provided that they are not mutually inconsistent.

Claims

1. A method performed by a genome data storage system for storing genome data in a data structure including a file structure, wherein the genome data storage system The steps include receiving a genome dataset containing one or more genome data from multiple fields or attributes of different data types, The steps include generating a protective metadata structure for the genome dataset, which includes a specification for the selective encryption of one or more data components and regions of the genome data within the annotation table, The steps include compressing the genome data and the protected metadata structure using one or more compression algorithms to generate a compressed genome dataset and a compressed protected metadata structure, The steps include storing the compressed genome dataset and the compressed protective metadata structure in a container data structure in memory. How to do it.

2. A method performed by a genome data storage system for storing genome data in a data structure including a file structure, wherein the genome data storage system The steps include receiving a genome dataset containing one or more genome data from multiple fields or attributes of different data types, The steps include generating a protective metadata structure for the genome dataset, which includes a specification for the selective signing of one or more data components and regions of the genome data within the annotation table, The steps include compressing the genome data and the protected metadata structure using one or more compression algorithms to generate a compressed genome dataset and a compressed protected metadata structure, The steps include storing the compressed genome dataset and the compressed protective metadata structure in a container data structure in memory. How to do it.

3. A method performed by a genome data storage system for storing genome data in a data structure including a file structure, wherein the genome data storage system The steps include receiving a genome dataset containing one or more genome data from multiple fields or attributes of different data types, The steps include generating a protective metadata structure for the genome dataset, which includes user key information for verifying whether the user has access rights to one or more data components and regions of the genome data in the annotation table, The steps include compressing the genome data and the protected metadata structure using one or more compression algorithms to generate a compressed genome dataset and a compressed protected metadata structure, The steps include storing the compressed genome dataset and the compressed protective metadata structure in a container data structure in memory. How to do it.

4. A method performed by a genome data storage system for storing genome data in a data structure including a file structure, wherein the genome data storage system The steps include receiving a genome dataset containing one or more genome data from multiple fields or attributes of different data types, The steps include generating a protective metadata structure for the genome dataset, which includes rules for access control applied to one or more data components and regions of the genome data in the annotation table, The steps include compressing the genome data and the protected metadata structure using one or more compression algorithms to generate a compressed genome dataset and a compressed protected metadata structure, The steps include storing the compressed genome dataset and the compressed protective metadata structure in a container data structure in memory. How to do it.

5. The method according to claim 1, wherein the genome data storage system further performs the steps of encrypting or decrypting the individual data components and payload blocks of the genome data, and optionally compressing or decompressing them to facilitate random access.

6. The method according to claim 1, wherein the genome data storage system further performs the step of selecting one or more data components or payload blocks of a specific region of the genome data in an annotation table, which includes one or more identification information such as a data component ID for the application of encryption, a range of matrix indices, a range of genome coordinates, and a sample ID.

7. The method according to claim 2, wherein the genome data storage system further performs the step of selecting one or more data components or payload blocks in a specific region of the genome data in an annotation table, which includes one or more identification information such as a data component ID for applying a digital signature, a range of matrix indices, a range of genome coordinates, and a sample ID.

8. The method according to claim 6, further comprising the steps of: detecting overlaps between selected data components or regions in the annotation table; notifying the user of the detected overlaps and / or automatically removing the detected overlaps from the selected data components or regions to ensure that each data component or payload block is not encrypted more than once.

9. The method according to claim 2, wherein the genome data storage system further performs the step of ordering, concatenating, and serializing the selected data components and payload blocks in the annotation table for the generation and verification of a digital signature.

10. The aforementioned genome data storage system further, for data integrity verification, The steps include extracting all digital signatures generated for the selected data components and / or areas in the annotation table, The steps include obtaining a verification key and verifying each of the extracted digital signatures, The method according to claim 2, further comprising the steps of presenting signature information and optionally providing, together with the signature information, the scope of application, the signer ID, and the date and time of signing.

11. The aforementioned genome data storage system further, for data acquisition, The steps include identifying any selected data components and / or regions within an encrypted annotation table, If any portion of the selected data component and / or area is encrypted, the steps include authenticating the user who requested the data retrieval and verifying whether the user has sufficient access rights. The steps include obtaining a decryption key and decrypting each of the encrypted data components and / or regions, Optionally, the step of performing data integrity verification, The method according to claim 3, further comprising the step of presenting the acquired data and any associated signature and / or verification results.

12. The aforementioned genome data storage system further, for data updating, The steps include identifying any previously encrypted and updated data components and / or regions, The steps include reapplying encryption to previously encrypted and updated data, The steps include compressing the updated data components and / or payload blocks as needed, The method according to claim 1, further comprising the step of storing the updated data in the annotation table.

13. The aforementioned genome data storage system further, for data updating, The steps include identifying any previously signed and updated data components and / or regions, The steps include: generating a new digital signature for the updated data and replacing the old digital signature; The steps include compressing the updated data components and / or payload blocks as needed, The method according to claim 2, further comprising the step of storing the updated digital signature in the annotation table.

14. The method according to claim 13, further comprising the step of locking selected data components and payload blocks protected by digital signatures so that only authenticated users with sufficient access rights can update the protected data.

15. A system for storing genome data in a data structure including a file structure, wherein the system A genome dataset containing one or more genomic data from multiple fields or attributes of different data types, Data structures for storing genome data, Data compression algorithms, A system comprising: (i) generating a protective metadata structure for the genome dataset, which includes a specification for the selective encryption of one or more data components and regions of the genome data in an annotation table; (ii) compressing the genome data and the protective metadata structure using the data compression algorithm to generate a compressed genome dataset and a compressed protective metadata structure; and (iii) storing the compressed genome dataset and the compressed protective metadata structure in the data structure.

16. A system for storing genome data in a data structure including a file structure, wherein the system A genome dataset containing one or more genomic data from multiple fields or attributes of different data types, Data structures for storing genome data, Data compression algorithms, A system comprising: (i) generating a protective metadata structure for the genome dataset, which includes a specification for the selective signing of one or more data components and regions of the genome data in an annotation table; (ii) compressing the genome data and the protective metadata structure using the data compression algorithm to generate a compressed genome dataset and a compressed protective metadata structure; and (iii) storing the compressed genome dataset and the compressed protective metadata structure in the data structure.

17. A system for storing genome data in a data structure including a file structure, wherein the system A genome dataset containing one or more genomic data from multiple fields or attributes of different data types, Data structures for storing genome data, Data compression algorithms, A system comprising: (i) generating a protective metadata structure for the genome dataset, which includes user key information for verifying whether a user has access rights to one or more data components and regions of the genome data in an annotation table; (ii) compressing the genome data and the protective metadata structure using the data compression algorithm to generate a compressed genome dataset and a compressed protective metadata structure; and (iii) storing the compressed genome dataset and the compressed protective metadata structure in the data structure.

18. A system for storing genome data in a data structure including a file structure, wherein the system A genome dataset containing one or more genomic data from multiple fields or attributes of different data types, Data structures for storing genome data, Data compression algorithms, A system comprising: (i) generating a protective metadata structure for the genome dataset, which includes rules for access control applied to one or more data components and regions of the genome data in an annotation table; (ii) compressing the genome data and the protective metadata structure using the data compression algorithm to generate a compressed genome dataset and a compressed protective metadata structure; and (iii) storing the compressed genome dataset and the compressed protective metadata structure in the data structure.

19. The system according to claim 15, wherein the processor further encrypts or decrypts individual data components and payload blocks of the genome data, and optionally compresses or decompresses them to facilitate random access.

20. The system according to claim 15, wherein the processor further receives a selection result of one or more data components or payload blocks of a specific region of the genomic data in an annotation table, which includes one or more identification information from among data component IDs for applying encryption, a range of matrix indices, a range of genomic coordinates, and a sample ID.

21. The system according to claim 16, wherein the processor further receives a selection result for one or more data components or payload blocks of a specific region of the genomic data in an annotation table, which includes one or more identification information from a data component ID, a matrix index range, a genomic coordinate range, and a sample ID for applying a digital signature.

22. The system according to claim 16, wherein the processor further extracts all digital signatures generated for the selected data components and / or areas in the annotation table, obtains a verification key, verifies each of the extracted digital signatures, presents the signature information, and optionally provides the scope, signer ID, and signing date and time together with the signature information.

23. The system according to claim 17, wherein the processor further identifies any selected data components and / or regions in an encrypted annotation table, authenticates a user requesting data retrieval if any portion of the selected data components and / or regions is encrypted, verifies whether the user has sufficient access rights, obtains a decryption key, decrypts each of the encrypted data components and / or regions, optionally performs data integrity verification, and presents the retrieved data and any associated signatures and / or verification results.

24. The system according to claim 15, wherein the processor further identifies any previously encrypted updated data components and / or regions, reapplies encryption to the previously encrypted updated data, compresses the updated data components and / or payload blocks as necessary, and stores the updated data in the annotation table.

25. The system according to claim 16, wherein the processor further identifies any previously signed updated data components and / or regions, generates a new digital signature for the updated data to replace the old digital signature, compresses the updated data components and / or payload blocks as necessary, and stores the updated digital signature in the annotation table.

Images