A method, system, and electronic device for encryption of a multi-modal database

By filtering, compressing, and encrypting sensitive data in the DolphinDB system, the problem of low data encryption efficiency is solved, achieving fine-grained encryption and efficient adaptation to multimodal storage engines, thus improving the efficiency and adaptability of data encryption.

CN120162802BActive Publication Date: 2026-01-23DOLPHINDB INC (CN)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510141767.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-02-08
Publication Date
2026-01-23
Estimated Expiration
2045-02-08

AI Technical Summary

Technical Problem

The DolphinDB system suffers from low data encryption efficiency. Existing technologies encrypt both sensitive and ordinary data, impacting system efficiency.

Method used

By filtering sensitive data in the data blocks to be processed, only sensitive data is compressed and encrypted. The compressor is modified to adapt to the multimodal storage engine under the compression and encryption method, reducing the encryption operation on non-sensitive data.

Benefits of technology

It improves the precision and efficiency of data encryption, reduces the performance loss caused by encryption, enhances adaptability to multimodal storage engines, and improves the data encryption efficiency of the DolphinDB system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120162802B_ABST
    Figure CN120162802B_ABST
Patent Text Reader

Abstract

The application relates to a multi-modal database encryption method, system and electronic equipment, wherein the multi-modal database encryption method comprises the following steps: acquiring a to-be-processed data block, screening sensitive data in the to-be-processed data block, and taking the sensitive data as a to-be-encrypted data block; compressing the to-be-encrypted data block through a compressor to obtain binary stream data; and encrypting the binary stream data, and writing the encrypted binary stream into a disk to realize encryption of the to-be-encrypted data block. Through the application, the problem of low data encryption efficiency in a DolphinDB system is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of information security, and in particular to an encryption method, system, and electronic device for a multimodal database. Background Technology

[0002] With the rapid development of computer technology, a large amount of data is stored in databases, and sensitive data in these databases needs to be encrypted to prevent data leakage.

[0003] DolphinDB is a distributed database system that supports multimodal storage engines. While it uses LSM-Tree encryption, this approach encrypts both sensitive and general data, impacting encryption efficiency. Therefore, improving the efficiency of data encryption in DolphinDB is a pressing issue. Summary of the Invention

[0004] This application provides an encryption method, system, and electronic device for a multimodal database, which at least solves the problem of low data encryption efficiency in the DolphinDB system in related technologies.

[0005] In a first aspect, embodiments of this application provide an encryption method for a multimodal database, comprising:

[0006] The method is applied to the DolphinDB system, which includes a compressor. The method includes:

[0007] Obtain the data block to be processed, filter out the sensitive data in the data block to be processed, and use the sensitive data as the data block to be encrypted.

[0008] The data block to be encrypted is compressed using a compressor to obtain a binary stream of data;

[0009] Encrypt binary stream data by writing the encrypted binary stream to disk to encrypt the data block to be encrypted.

[0010] In one embodiment, filtering sensitive data from the data to be processed includes:

[0011] The data blocks to be processed are stored in a mixed row and column storage format to obtain a data table, and the sensitive columns are determined in the data table. The data table includes several rows of data.

[0012] In the data table, rows that meet the preset number of rows are used as intermediate data blocks until all data in the data table is obtained. Then, rows of sensitive columns in all intermediate data blocks are used as sensitive data.

[0013] In one embodiment, acquiring the data to be processed includes:

[0014] Receive several files to be processed. For any one file to be processed, divide the data in the file into several data blocks according to a preset data size, and use the data blocks as the data blocks to be processed.

[0015] In one embodiment, the method further includes:

[0016] Identify the faulty node in the DolphinDB system;

[0017] When a failed node comes back online, the data is encrypted based on the database in the DolphinDB system, and the encrypted data is transmitted to the normal node. The data of the failed node is then restored through online incremental recovery.

[0018] In one embodiment, the DolphinDB system further includes a multi-model storage engine and a decompressor, and the method further includes:

[0019] The data to be decrypted is obtained through a multimodal storage engine;

[0020] The data to be decrypted is decrypted using a decompressor to obtain the decrypted data.

[0021] The decrypted data is cached in the memory of the DolphinDB system to make the decrypted data compatible with the multimodal storage engine.

[0022] In one embodiment, the DolphinDB system further includes a query engine, and the method further includes:

[0023] The decrypted data is cached in memory;

[0024] The decrypted data is retrieved from memory using a query engine.

[0025] In one embodiment, the method further includes:

[0026] Obtain several encrypted files, decrypt and decompress any one of the encrypted files, and read the data blocks in the encrypted file to obtain the first data block;

[0027] According to the preset requirements, all first data blocks will be merged;

[0028] If the first data block is merged, the second data block is obtained iteratively in the encrypted file and merged, until all data blocks in the encrypted files are merged.

[0029] Secondly, embodiments of this application provide an encryption system for a multimodal database, characterized in that the system includes a compressor, and the system includes:

[0030] The acquisition module is used to acquire data blocks to be processed, filter sensitive data in the data blocks to be processed, and use the sensitive data as data blocks to be encrypted.

[0031] The compression module is used to compress the data block to be encrypted through a compressor to obtain binary stream data;

[0032] The encryption module is used to encrypt binary stream data and write the encrypted binary stream to disk to encrypt the data block to be encrypted.

[0033] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the encryption method for the multimodal database as described in the first aspect above.

[0034] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the encryption method for the multimodal database as described in the first aspect above.

[0035] Compared to related technologies, the encryption method, system, and electronic device for multimodal databases provided in this application use data blocks as processing units and obtain intermediate data blocks from the data blocks to be processed. The encrypted binary stream data is then encrypted using a compressor to encrypt sensitive data. By encrypting sensitive data blocks, non-sensitive data does not require encryption, thereby improving the precision of data encryption and reducing performance loss caused by encryption. Furthermore, the method of encrypting sensitive data after compression allows for modifications to the compressor to adapt the compressed data to different multimodal storage engines in the DolphinDB system, improving the adaptability of encrypted data to different multimodal storage engines and thus increasing the efficiency of data encryption in the DolphinDB system. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a flowchart illustrating an encryption method for a multimodal database according to an exemplary embodiment;

[0038] Figure 2 This is a schematic diagram illustrating the filtering of sensitive columns according to an exemplary embodiment;

[0039] Figure 3 This is a block diagram illustrating an encryption system for a multimodal database according to an exemplary embodiment;

[0040] Figure 4 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0042] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0043] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0044] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0045] Firstly, embodiments of this application provide an encryption method for a multimodal database. Figure 1 This is a flowchart illustrating an encryption method for a multimodal database according to an exemplary embodiment, such as... Figure 1 As shown, the encryption methods for multimodal databases include:

[0046] Step S101: Obtain the data block to be processed, filter the sensitive data in the data block to be processed, and use the sensitive data as the data block to be encrypted.

[0047] Obtain the data block to be processed. In the data block, the data is stored in a mixed row and column storage format. Identify the sensitive columns in the data block to be processed, and use the sensitive data in the sensitive columns as the data block to be encrypted, so that subsequent data encryption will only be performed on the sensitive columns.

[0048] Obtain the data block to be processed, specifically including:

[0049] Receive several files to be processed. For any one file to be processed, divide the data in the file into several data blocks according to a preset data size, and use the data blocks as the data blocks to be processed.

[0050] The files to be processed contain various types of data. After receiving several files, the data from these files is merged. For any given file, the data is divided into several data blocks according to a preset data size. When reading a file, these data blocks are read sequentially as the data blocks to be processed for encryption.

[0051] By using data blocks as the processing unit, the amount of data that needs to be decrypted when reading data is reduced, and memory usage is also reduced when merging data. The specific implementation method will be explained in detail later.

[0052] In the DolphinDB system, data within data blocks is stored in a mixed row-column format, with column data stored contiguously within the data block. Therefore, sensitive columns can be encrypted only during the data filtering process. The specific steps for filtering sensitive data within a data block are as follows:

[0053] Step S111: Store the data in the data block to be processed in a mixed row and column storage format to obtain a data table, and determine the sensitive column in the data table. The data table includes several rows of data.

[0054] The data in the data block to be processed is stored in a mixed row-column storage format to obtain a data table. The data table contains several rows and columns of data, with data in the same column being consecutive. Sensitive columns are identified in the data table to provide a basis for subsequent encryption of the data in these sensitive columns.

[0055] Step S113: In the data table, the row data that meets the preset number of rows is taken as intermediate data blocks in sequence until all data in the data table is obtained, and the row data of the sensitive column in all the intermediate data blocks is taken as sensitive data.

[0056] Based on a preset number of rows, each row of data in each column of the data table is sequentially divided into several intermediate data blocks until all data in the data table is divided into several intermediate data blocks. In each intermediate data block, the row data corresponding to the sensitive column is determined, and the row data of the sensitive column is regarded as sensitive data. Optionally, the preset number of rows includes 3, 4, and 5 rows. The preset number of rows is not specifically limited and is determined according to the actual application scenario.

[0057] Figure 2 This is a schematic diagram illustrating the filtering of sensitive columns according to an exemplary embodiment, such as... Figure 2As shown, the data table has three columns and six rows. Column 1 is the sensitive column. Rows 0, 1, and 2 are used as the first intermediate data block, and rows 3, 4, and 5 are used as the second intermediate data block. In the first intermediate data block, the row data of column 1 is used as the sensitive data, and in the second intermediate data block, the row data of column 1 is used as the sensitive data.

[0058] In multimodal storage, sorting columns and primary key columns are involved in operations such as querying and merging, but they are often not sensitive columns. By filtering out sensitive columns using the method described above, ordinary columns such as sorting columns and primary key columns are no longer affected by encryption. This provides a foundation for subsequent encryption of data in sensitive columns, thereby improving the granularity of data encryption and reducing the performance and cost losses caused by encryption.

[0059] Step S102: Compress the data block to be encrypted using a compressor to obtain binary stream data.

[0060] In the DolphinDB system, the multi-model storage engine writes the data blocks to be encrypted to a cache in memory during the disk write process. When the written data blocks reach a preset size in the buffer, the buffer is frozen, and the data blocks to be encrypted are compressed by a compressor to obtain a binary stream.

[0061] By compressing the data block to be encrypted using a compressor to obtain a binary stream, it becomes possible to obtain data compatible with the multimodal storage engine in the DolphinDB system by modifying the compressor, thus improving the compatibility with multimodal storage engines.

[0062] Step S103: Encrypt binary stream data and write the encrypted binary stream to disk to encrypt the data block to be encrypted.

[0063] The compressed binary data stream is encrypted, and the encrypted binary data stream is written to the disk. Since the binary data stream is obtained by the compressor encrypting the data block to be encrypted, the encryption method in this embodiment can adapt to different storage engines by modifying the compressor to obtain the binary stream and then encrypting the binary stream.

[0064] After the encrypted data is written to disk, it must be decrypted. The DolphinDB system also includes a multi-model storage engine and a decompressor. Decryption involves the following steps:

[0065] Step S201: Obtain the data to be decrypted through the multimodal storage engine.

[0066] Step S202: Decrypt the data to be decrypted using a decompressor to obtain the decrypted data.

[0067] Step S203: Cache the decrypted data in the memory of the DolphinDB system so that the decrypted data is compatible with the multimodal storage engine.

[0068] The DolphinDB multi-modal storage engine retrieves the data to be decrypted from the disk. During data reading, the DolphinDB multi-modal storage engine distributes the data across memory and disk. Data in memory is unencrypted and can be read directly. For data on disk, the DolphinDB multi-modal storage engine uses the index to locate the file and offset to be read, and then reads the corresponding file data into memory. A decompressor then decrypts the data in memory to obtain the decrypted data. The decrypted and decompressed data is then cached in the DolphinDB system's memory, and the query engine retrieves the decrypted data from memory.

[0069] Using the method described above, only the decompressor needs to be modified to make the decrypted data compatible with DolphinDB's multimodal storage engine. By first decrypting in memory, obtaining the decrypted data, and then decompressing and buffering the decrypted data in memory, direct querying in memory is possible, thus achieving efficient decryption and ensuring that the decrypted data is compatible with DolphinDB's multimodal storage engine.

[0070] The encryption method in steps S101 to S103, which involves compressing the data to be encrypted before encryption, and the decryption method in steps S201 to S203, which involves decrypting before decompression, can achieve unified support for multimodal storage engines by modifying the compressor and decompressor. On the other hand...

[0071] In one embodiment, the query performance of DolphinDB's multimodal storage engine is affected after decryption is introduced, requiring further improvement in the DolphinDB system's query performance. Data queries exhibit locality of reference, meaning data accessed in the past may be accessed again in the future. Data queries also exhibit hotspot characteristics, meaning the query content focuses on a small amount of data. Therefore, caching the accessed decrypted and decompressed data in memory allows the system to directly read data from memory during subsequent queries if the data is already in the memory cache. This avoids reading data from disk and performing encryption and decompression, reducing the amount of data that needs to be decrypted during queries and thus improving the system's query performance when encryption and decryption are supported.

[0072] In another embodiment, where encryption is supported, data merging, or file merging, is also required. When processing each encrypted file, the DolphinDB system reads the data blocks that were partitioned during the write process, decompresses and decrypts the first data block, and performs a data merging operation. If the current data block is merged and the operation is completed, the second data block is read from the file, and the above steps are repeated until the entire encrypted file is consumed. For example, with three files, data blocks are read from each encrypted file to obtain three data blocks, which are then merged. This involves sorting and merging the three data blocks according to preset requirements. Once the merge is successful, another data block is read from the file, and the same merging process is repeated until all three files are consumed, ultimately merging them into one file. File consumption refers to writing data from the input file to the output file. This data merging method reduces the operating overhead of the DolphinDB system and improves merging efficiency.

[0073] In another embodiment, due to node failure or network issues, one or more nodes in the DolphinDB system may go offline; these offline nodes are designated as faulty nodes. When a faulty node comes back online, it needs to synchronize data with other nodes. The DolphinDB system encrypts the data in the database before sending it to other nodes and uses an online incremental approach to restore the faulty node, ensuring the security of the entire system.

[0074] In summary, the encryption method, system, and electronic device for multimodal databases provided in this application achieve refined encryption by encrypting and decrypting data in sensitive columns, while reducing the performance and cost losses associated with encryption. By encrypting after compression and decrypting before decompression, the encrypted and decrypted data are generally compatible with multimodal storage engines, enabling the encryption method to fully support distributed database encryption. By processing data blocks as the unit, the amount of data that needs to be decrypted when reading data and the memory usage of data during data merging are reduced, thereby improving the efficiency of data encryption in the DolphinDB system.

[0075] Secondly, embodiments of this application provide an encryption system for a multimodal database. Figure 3 This is a block diagram illustrating an encryption system for a multimodal database according to an exemplary embodiment. For example... Figure 3 As shown, the encryption system of the multimodal database includes: an acquisition module, used to acquire a data block to be processed, filter sensitive data in the data block to be processed, and use the sensitive data as a data block to be encrypted;

[0076] A compression module is used to compress the data block to be encrypted through the compressor to obtain binary stream data;

[0077] An encryption module is used to encrypt the binary stream data and write the encrypted binary stream to a disk to encrypt the data block to be encrypted.

[0078] In summary, the encryption system, method, system, and electronic device for multimodal databases provided in this application use data blocks as processing units, obtain intermediate data blocks from the data blocks to be processed, and encrypt the binary stream data compressed by a compressor, thereby achieving data encryption. By encrypting sensitive data, non-sensitive data does not need to be encrypted, thus improving the fineness of data encryption and reducing the performance loss caused by encryption. Furthermore, by using compression before encryption for sensitive data, only the compressor needs to be modified to adapt the compressed data to different multimodal storage engines in the DolphinDB system, improving the adaptability of encrypted data to different multimodal storage engines, and thus improving the efficiency of data encryption in the DolphinDB system.

[0079] It should be noted that the encryption system for the multimodal database provided in this embodiment is used to implement the above-described embodiments, and details already described will not be repeated. As used above, the terms "module," "unit," and "subunit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the system described in the above embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0080] Thirdly, embodiments of this application provide an electronic device, Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. (e.g.) Figure 4 As shown, the electronic device may include a processor 81 and a memory 82 storing computer program instructions.

[0081] Specifically, the processor 81 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0082] The memory 82 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include removable or non-removable (or fixed) media. Where appropriate, the memory 82 may be internal or external to a data processing device. In a particular embodiment, the memory 82 is non-volatile memory. In a particular embodiment, the memory 82 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only ROM (PROM), an erasable programmable read-only ROM (EPROM), an electrically erasable programmable read-only ROM (EEPROM), an electrically alterable read-only ROM (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0083] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 81.

[0084] The processor 81 implements any of the encryption methods for the multimodal database in the above embodiments by reading and executing computer program instructions stored in the memory 82.

[0085] In one embodiment, the encryption device for the multimodal database may further include a communication interface 83 and a bus 80. Wherein, as Figure 4 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.

[0086] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication port 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0087] Bus 80 includes hardware, software, or both, that couples the components of the encryption device for the multimodal database together. Bus 80 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0088] Fourthly, embodiments of this application provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the encryption method for the multimodal database provided in the first aspect.

[0089] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0090] In a possible implementation, the present invention can also be implemented as a program product comprising program code, which, when the program product is run on a terminal device, is used to cause the terminal device to perform steps implementing the encryption method for the multimodal database provided in the first aspect.

[0091] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An encryption method for a multimodal database, characterized in that, The method is applied to the DolphinDB system, which includes a compressor, and the method includes: Obtain a data block to be processed, filter out sensitive data in the data block to be processed, and use the sensitive data as a data block to be encrypted. The compressor is used to compress the data block to be encrypted to obtain binary stream data. The binary stream data is encrypted, and the encrypted binary stream is written to the disk to encrypt the data block to be encrypted. The filtering of sensitive data in the data block to be processed includes: The data block to be processed is stored in a mixed row and column storage format to obtain a data table, and a sensitive column is determined in the data table, which includes several rows of data. In the data table, the rows of data that meet the preset number of rows are sequentially used as intermediate data blocks until all data in the data table is obtained, and the rows of sensitive columns in all the intermediate data blocks are used as sensitive data. The acquisition of data to be processed includes: Receive several files to be processed. For any one of the files to be processed, divide the data in the file to be processed into several data blocks according to a preset data size, and use the data blocks as data blocks to be processed.

2. The encryption method for a multimodal database according to claim 1, characterized in that, The method further includes: Identify the faulty node in the DolphinDB system; When the faulty node comes back online, the data is encrypted based on the data in the database of the DolphinDB system, and the encrypted data is transmitted to the normal node. The data of the faulty node is then restored through online incremental recovery.

3. The encryption method for a multimodal database according to claim 1 or 2, characterized in that, The DolphinDB system also includes a multi-model storage engine and a decompressor, and the method further includes: The data to be decrypted is obtained through the multimodal storage engine. The decompressor is used to decrypt the data to be decrypted, and the decrypted data is obtained. The decrypted data is cached in the memory of the DolphinDB system to make the decrypted data compatible with the multimodal storage engine.

4. The encryption method for a multimodal database according to claim 3, characterized in that, The DolphinDB system also includes a query engine, and the method further includes: The decrypted data is cached in the memory location; The decrypted data is retrieved from the memory using the query engine.

5. The encryption method for the multimodal database according to claim 3, the method further comprising: Obtain several encrypted files, decrypt and decompress any one of the encrypted files, and read the data blocks in the encrypted file to obtain the first data block; According to the preset requirements, all the first data blocks are merged; If the first data block is merged, the second data block is obtained iteratively in the encrypted file and merged, until all data blocks in the encrypted file are merged.

6. An encryption system for a multimodal database, characterized in that, The system includes a compressor, the system comprising: The acquisition module is used to acquire a data block to be processed, filter sensitive data in the data block to be processed, and use the sensitive data as a data block to be encrypted. A compression module is used to compress the data block to be encrypted through the compressor to obtain binary stream data; An encryption module is used to encrypt the binary stream data and write the encrypted binary stream to a disk to encrypt the data block to be encrypted. The step of filtering sensitive data in the data block to be processed includes: storing the data block to be processed in a row-column mixed storage format to obtain a data table, and determining sensitive columns in the data table, wherein the data table includes several rows of data; in the data table, sequentially taking the rows of data that meet a preset number of rows as intermediate data blocks until all data in the data table is obtained, and taking the rows of sensitive columns in all the intermediate data blocks as sensitive data; The process of acquiring data to be processed includes: receiving a number of files to be processed; for any one of the files to be processed, dividing the data in the file to be processed into a number of data blocks according to a preset data size, and using the data blocks as data blocks to be processed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the encryption method for the multimodal database as described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the encryption method for the multimodal database as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Data transmission method and device, electronic equipment and storage medium

    CN111586072A

  • Information encryption method, information decryption method and device, electronic equipment and storage medium

    CN113886850A