Rapid retrieval method and device based on row-column hybrid encrypted storage

Abstract

The invention relates to the technical field of data processing, in particular to a rapid retrieval method based on row-column hybrid encrypted storage, which comprises the following steps: storing acquired data information according to a preset data storage principle, the data information comprising real data and data in a data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries initial position information and data block index information corresponding to each data block; and in response to a received retrieval instruction, filtering the data attribute block based on the data block index information to obtain a target data block, and retrieving in the real data to obtain target data based on the initial position information carried in the target data block. By applying the method, the data retrieval speed can be increased under the conditions of saving the data storage space and ensuring the data security.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a fast retrieval method and device based on row-column hybrid encrypted storage. Background Art

[0002] With the continuous development of social economy, the Internet of Things technology and artificial intelligence technology are also developing rapidly, which has given rise to the advent of the big data era, bringing great convenience to people from all walks of life. However, the existence of a large amount of data has also brought a lot of inconvenience to people, such as data storage problems and data retrieval problems.

[0003] Currently, common data storage methods include row storage and column storage. Among them, row storage is a storage method that organizes data by row. In row storage, the entire row of data is stored as a basic unit on disk or in memory. This method is similar to the storage method in traditional relational databases, where each row represents a record and contains all the fields or attributes of the record. Row storage tables support complete addition, deletion, modification and query. It is suitable for scenarios where data needs to be updated frequently. Column storage is a storage method that organizes data by column. In column storage, each column of the table is stored separately on disk, which means that all values ​​of a given column are stored together. This storage method is suitable for scenarios with batch data insertion, less updates, and statistical analysis based on queries. Column storage tables are not suitable for point queries, and the performance of inserting a single record is poor.

[0004] In view of this, how to provide a fast retrieval method based on row-column hybrid encrypted storage to improve data retrieval speed while saving data storage space and ensuring data security has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a fast retrieval method based on row-column hybrid encrypted storage, a fast retrieval device based on row-column hybrid encrypted storage, an electronic device and a computer storage medium, which are used to solve the problem of how to improve data retrieval speed while saving data storage space and ensuring data security.

[0006] In a first aspect of an embodiment of the present application, a fast retrieval method based on row-column hybrid encrypted storage is provided, comprising:

[0007] According to the preset data storage principle, the acquired data information is stored, wherein the data information includes real data and data in the data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries the starting position information and data block index information corresponding to each data block;

[0008] In response to receiving a search instruction, the data attribute block is filtered based on the data block index information to obtain a target data block, and the target data is retrieved from the real data based on the starting position information carried in the target data block.

[0009] In a second aspect of an embodiment of the present application, a fast retrieval device based on row-column hybrid encryption storage is provided, comprising:

[0010] The storage module is configured to store the acquired data information according to a preset data storage principle, wherein the data information includes real data and data in a data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries the starting position information and data block index information corresponding to each data block;

[0011] The retrieval module is configured to filter the data attribute block based on the data block index information in response to receiving a retrieval instruction to obtain a target data block, and retrieve the target data from the real data based on the starting position information carried in the target data block.

[0012] In a third aspect of an embodiment of the present application, a computing device is provided, including:

[0013] Memory and processor;

[0014] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned fast retrieval method based on row-column hybrid encryption storage are implemented.

[0015] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned fast retrieval method based on row-column hybrid encryption storage are implemented.

[0016] The present application provides a fast retrieval method based on row-column hybrid encryption storage, comprising: first, storing the acquired data information according to a preset data storage principle, wherein the data information includes real data and data in a data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries starting position information and data block index information corresponding to each data block; then, in response to receiving a retrieval instruction, filtering the data attribute block based on the data block index information to obtain a target data block, and retrieving the target data from the real data based on the starting position information carried in the target data block.

[0017] The fast retrieval method based on row-column hybrid encrypted storage provided in the embodiment of the present application can improve data retrieval speed while saving data storage space and ensuring data security.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, which can be implemented in accordance with the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0020] Figure 1 A flowchart of a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of the structure of the storage format in a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the storage structure of the tail in a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a process of writing data encryption in a fast retrieval method based on row-column hybrid encryption storage provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of a process of writing data encryption in a fast retrieval method based on row-column hybrid encryption storage provided in an embodiment of the present application;

[0025] Figure 6 A timing diagram of a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of a fast retrieval device based on row-column hybrid encrypted storage provided in an embodiment of the present application;

[0027] Figure 8 A structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0029] See also Figure 1 , Figure 1 The flowchart of a fast retrieval method based on row-column hybrid encrypted storage provided by the embodiment of the present application is as follows. Figure 1 As shown, the specific steps include the following steps.

[0030] Step S102: According to a preset data storage principle, the acquired data information is stored, wherein the data information includes real data and data in a data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries the starting position information and data block index information corresponding to each data block.

[0031] In an embodiment of the present application, the storage of the acquired data information includes: normal storage and encrypted storage; wherein, the storage of the acquired data information according to a preset data storage principle includes: acquiring data information, dividing the data information into at least one logical horizontal partition according to rows, wherein each of the logical horizontal partitions is composed of column blocks arranged in columns; when the amount of data in each of the column blocks meets a preset data amount threshold, determining the maximum data value information and the minimum data value information corresponding to each column block, and encrypting the data except the maximum data value information and the minimum data value information; performing normal storage on the maximum data value information and the minimum data value information, and performing encrypted storage on the data except the maximum data value information and the minimum data value information.

[0032] In an embodiment of the present application, the storage of the acquired data information includes: row storage and column storage; wherein, the storage of the acquired data information according to a preset data storage principle includes: determining the update frequency of each target data in the data information, and in response to the frequent update of the target data, row storage of the target data; determining the insertion frequency of each target data in the data information, and in response to the frequent insertion of the target data, column storage of the target data; in response to the infrequent insertion of the target data, row storage of the target data; determining the number of table columns corresponding to each target data in the data information, and in response to the number of table columns corresponding to the target data exceeding a first preset table column number threshold, column storage of the target data; determining the number of columns of the target data corresponding to the retrieval instruction based on the historical retrieval instruction, and in response to the target data corresponding to the retrieval instruction, the number of table columns corresponding to the data information is less than a second preset table column number threshold, and column storage of the target data.

[0033] In actual applications, the selection principles of row storage and column storage are as follows: 1. Update frequency: If data is frequently updated, choose row storage table; 2. Insert frequency: If frequent small-scale inserts are made, choose row storage table. If a large amount of data is inserted at one time, choose column storage table; 3. Number of columns in the table: If the table has many columns, choose column storage table; 4. Number of query columns: If each query involves only a few columns (less than 50% of the total number of columns), choose column storage table; 5. Compression rate: Column storage table has a higher compression rate than row storage table. However, a high compression rate consumes more CPU resources.

[0034] In the embodiment of the present application, storing the acquired data information according to a preset data storage principle includes: storing the acquired data information according to actual needs of the user.

[0035] In practical applications, the embodiment of the present application uses this technology in special file types, and can freely choose whether to encrypt data and data attribute blocks during storage, and when only data is encrypted but not indexes, it can ensure both the security of stored data and the efficiency of queries.

[0036] In an embodiment of the present application, a method for determining data block index information corresponding to each data block carried in attribute information corresponding to a data attribute block includes: when writing a data block, indexing the column storage data of each data block, and unencryptingly storing the data maximum value information and data minimum value information carried in the index, wherein the index carries the data maximum value information, the data minimum value information, the column length information before encryption, the column length information after encryption, and the storage starting position information after encryption.

[0037] In an embodiment of the present application, structured data is stored according to a preset data storage principle to generate a storage file in a target storage format, wherein the target storage format is a JFILE format, and the JFILE format carries a special identifier; the data in the storage file is read through a storage and computing engine.

[0038] It should be noted that the fast retrieval method based on row-column hybrid encrypted storage provided in the embodiment of the present application has better query efficiency when applied in the OLAP scenario, and realizes a new storage format JFILE, in which data is stored based on columns.

[0039] JFILE is a file storage format implemented at the bottom layer of JUPITER. It stores structured data according to certain rules and generates a storage file of type JFILE. There are special identifiers in this file, so the data can only be read using our storage engine. At the same time, JUPITER supports data compression and encryption for JFILE content. When we use JFILE, we can make corresponding settings. In this file, we use column storage to store file content. When storing data, you can use the specified compression algorithm and encryption algorithm to compress and encrypt the data, which not only saves storage space but also ensures data security.

[0040] See also Figure 2 , Figure 2 A structural schematic diagram of the storage format in a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application.

[0041] like Figure 2 As shown, the storage format carries Header (including JUP1 and JUPE), INDEX, FOOTER, Rowgroup, column chunk and page.

[0042] Specifically, the Header mainly records the storage type, which corresponds to JUP1 and JUPE. JUP1 represents a normal JFILE file, and JUPE represents an encrypted JFILE file. There is an identifier corresponding to the header at the end of the footer.

[0043] Data is data information.

[0044] Index is mainly used for filtering, storing the maximum and minimum values ​​in the page, so that blocks can be filtered.

[0045] The footer contains source data information such as schema, row group size and offset, column chunk size and offset, etc. We can locate the specific column chunk for reading based on the source data information. The last four bits are also JUP1 / JUPE, corresponding to the header.

[0046] Row Group divides data into multiple logical horizontal partitions according to rows. A row group consists of a column chunk for each column.

[0047] Column Chunk: A column block is distributed in row groups and is guaranteed to be continuous in the file.

[0048] A page block is divided into multiple pages, and a page is the smallest basic unit.

[0049] When storing, the embodiment of the present application not only stores the real data, but also stores the attribute information of some data blocks of the data, including the starting position of each data block, simple index information, etc. Based on this information, we can quickly filter the database.

[0050] The attribute information of the data block is stored separately from the real data. The advantage of this is that when we perform conditional retrieval, we only need to read the attribute information of the data block without accessing the real data. The length, type and simple index information of each file block are stored in the data block. When we filter, we only need to filter in the data attribute block first, find the required data block, and then directly locate the specific location based on the stored information to obtain the data.

[0051] The main positioning data information is stored at the end of the file, see Figure 3 , Figure 3 A schematic diagram of the storage structure of the tail portion of a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application.

[0052] like Figure 3 As shown, the storage structure at the end includes the following information about the location data:

[0053] Extra: Stores file extension information, you can write custom information;

[0054] Uncompress size: the size of the uncompressed data;

[0055] Compress size: the size of the compressed data;

[0056] Offset ofindex: The offset of the index, which can quickly locate the specified index;

[0057] Offset ofdatapage: the offset of the page, which can quickly locate the specific page;

[0058] Encode: encoding method;

[0059] Path: The path where the file is stored.

[0060] By using the information of positioning data provided by the embodiment of the present application, a specified page in a specified row group can be quickly positioned.

[0061] For example: if there is a data block with data 1, 2, 3, 4, 5, then the maximum value recorded in our data block attributes is 5 and the minimum value is 1. When performing a select query and the where condition contains size comparison, we judge the comparison condition. If the data taken by this condition is greater than or equal to 1 or less than or equal to 5, it will be matched according to the conditions in the data block attributes, and then the specific data will be found in the corresponding data block.

[0062] When storing, in order to ensure the security of the data, the stored data will be encrypted. Although this ensures security, every time we need to obtain data, we need to decrypt all the data, which greatly wastes the query time. In order to improve the query efficiency, we have indexed the encrypted data, which greatly improves the query efficiency.

[0063] Specific process: When writing a data block, an index is created for the column data in each data block, which contains the maximum and minimum value information, the length of each column before and after encryption, and the starting position of the encrypted storage. It should be noted that the starting position of the encrypted storage here can directly jump to the column and decrypt the required column; on the other hand, it can also directly skip the unnecessary data, thereby improving the query efficiency.

[0064] Step S104: In response to receiving the search instruction, the data attribute block is filtered based on the data block index information to obtain a target data block, and the target data is retrieved from the real data based on the starting position information carried in the target data block.

[0065] In an embodiment of the present application, the data attribute block is filtered based on the data block index information to obtain a target data block, and based on the starting position information carried in the target data block, the target data is retrieved from the real data, including: determining the retrieval data corresponding to the retrieval instruction, determining the target data range corresponding to the data attribute block corresponding to the retrieval data in the stored data information based on the data block index information to obtain the target data block, and filtering other data attribute blocks except the target data block; based on a preset operation instruction, determining the target starting position information corresponding to the retrieval data in the target data block, based on the target starting position information, jumping to the column block corresponding to the target data block, and performing decryption processing to retrieve the target data from the real data, wherein the preset operation instruction includes an equal value operation and a comparison operation.

[0066] See also Figure 4 , Figure 4 A schematic diagram of the process of writing data encryption in a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application.

[0067] like Figure 4 As shown, the specific steps include: first, start writing the jfile file; then, when the preset size is reached, start writing the data of each column; encrypt the data of each column, and obtain the min and max before encryption; write the starting position of each column of data in the file into the footer, and write the min and max; write in a loop until a file is written; finally, after a file is written, end the process.

[0068] Depend on Figure 4 It can be seen that when writing each column of data, the maximum value (max) and minimum value (min) in this column are actually written. We do not encrypt these data. The advantage of this is that comparison can be performed directly during query.

[0069] When performing query operations, if it is an equality operation, comparison operation, or range operation, data can be filtered based on index information, skipping unnecessary data and only decrypting the required data packets, which greatly improves query efficiency.

[0070] Specific process: execute the select statement to query; push down the query conditions; according to the query conditions, make a judgment in the added index. If there is data in the corresponding rowgroup, decrypt the corresponding encrypted data block. Otherwise, skip the query of the encrypted block and continue to filter the index of the next block.

[0071] By applying this method, when performing equal value queries and size comparison queries, a filter can be performed to filter out the entire database that does not meet the conditions, greatly improving the query performance.

[0072] See also Figure 5 , Figure 5 A schematic diagram of the process of writing data encryption in a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application.

[0073] like Figure 5 As shown, the specific steps include: first, select c2, c3 from tab where c1=1, that is, when c1=1, select c2 and c3 from the table; then, perform conditional push based on c1=1; secondly, obtain the attributes of each c1 in jfile; thirdly, determine whether c1 is between min and max in the data block; if so, take the c1 data between to determine whether it exists, and if c1 exists, obtain c2 and c3 for decryption, and obtain c2 and c3 according to the position; if c1 does not exist, skip the data block and continue to process the next attribute block; if not, skip the data block directly and continue to process the next attribute block.

[0074] Depend on Figure 5 It can be seen that we can quickly filter out unnecessary data blocks during query, saving the work of decryption and data matching for these databases. If the amount of data is too large, this operation can greatly save time.

[0075] See also Figure 6 , Figure 6 A timing diagram of a fast retrieval method based on row-column hybrid encrypted storage provided in an embodiment of the present application.

[0076] The fast retrieval method based on row-column hybrid encrypted storage provided in the embodiment of the present application can improve data retrieval speed while saving data storage space and ensuring data security.

[0077] Corresponding to the above method embodiment, this specification also provides a fast retrieval device embodiment based on row-column hybrid encryption storage, Figure 7 A schematic diagram of a fast retrieval device based on row-column hybrid encrypted storage provided in an embodiment of the present application. Figure 7 As shown, the device comprises:

[0078] The storage module 702 is configured to store the acquired data information according to a preset data storage principle, wherein the data information includes real data and data in a data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries the starting position information and data block index information corresponding to each data block;

[0079] The retrieval module 704 is configured to filter the data attribute block based on the data block index information in response to receiving a retrieval instruction to obtain a target data block, and retrieve the target data from the real data based on the starting position information carried in the target data block.

[0080] In an optional embodiment, the storage module 702 is further configured to:

[0081] Acquire data information, and divide the data information into at least one logical horizontal partition according to rows, wherein each of the logical horizontal partitions is composed of column blocks arranged in columns;

[0082] When the amount of data in each of the column blocks meets a preset data amount threshold, determining maximum data value information and minimum data value information corresponding to each of the column blocks, and encrypting data except the maximum data value information and the minimum data value information;

[0083] The data maximum value information and the data minimum value information are stored normally, and data other than the data maximum value information and the data minimum value information are stored encrypted.

[0084] In an optional embodiment, the storage module 702 is further configured to:

[0085] Determining the update frequency of each target data in the data information, and in response to the target data being frequently updated, performing row storage on the target data;

[0086] Determine the frequency of insertion of each target data in the data information, and in response to the target data being frequently inserted, store the target data in columns; in response to the target data being infrequently inserted, store the target data in rows;

[0087] Determine the number of table columns corresponding to each target data in the data information, and in response to the number of table columns corresponding to the target data exceeding a first preset table column number threshold, perform column storage on the target data;

[0088] According to the historical search instructions, the number of columns of the target data corresponding to the search instruction is determined. In response to the target data corresponding to the search instruction, if the number of corresponding table columns in the data information is less than a second preset table column number threshold, the target data is stored in columns.

[0089] In an optional embodiment, the storage module 702 is further configured to:

[0090] When writing a data block, the column storage data of each data block is indexed, and the maximum data value information and the minimum data value information carried in the index are stored in an unencrypted manner, wherein the index carries the maximum data value information, the minimum data value information, the column length information before encryption, the column length information after encryption, and the storage starting position information after encryption.

[0091] In an optional embodiment, the retrieval module 704 is further configured to:

[0092] Determine the search data corresponding to the search instruction, determine the target data range corresponding to the data attribute block corresponding to the search data in the stored data information based on the data block index information, obtain the target data block, and filter other data attribute blocks except the target data block;

[0093] Based on preset operation instructions, determine the target starting position information corresponding to the retrieved data in the target data block, jump to the column block corresponding to the target data block based on the target starting position information, and perform decryption processing to retrieve the target data in the real data, wherein the preset operation instructions include equal value operations and comparison operations.

[0094] In an optional embodiment, the device is further configured to:

[0095] A generating module is configured to store the structured data according to a preset data storage principle and generate a storage file in a target storage format, wherein the target storage format is a JFILE format, and the JFILE format carries a special identifier;

[0096] The data in the storage file is read through the storage engine.

[0097] In an optional embodiment, the storage module 702 is further configured to:

[0098] According to the actual needs of users, the acquired data information is stored.

[0099] The fast retrieval device based on row-column hybrid encrypted storage provided by the embodiment of the present application can improve the data retrieval speed while saving data storage space and ensuring data security.

[0100] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the fast retrieval device based on row-column hybrid encryption storage, since it is basically similar to the fast retrieval method embodiment based on row-column hybrid encryption storage, the description is relatively simple, and the relevant parts can be referred to the partial description of the fast retrieval method embodiment based on row-column hybrid encryption storage.

[0101] Figure 8 This is a block diagram of a computing device provided in an embodiment of the present application. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0102] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0103] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0104] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 may also be a mobile or stationary server.

[0105] The processor 820 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned fast retrieval method based on row-column hybrid encryption storage.

[0106] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the fast retrieval method embodiment based on row-column hybrid encrypted storage, the description is relatively simple, and the relevant parts can be referred to the partial description of the fast retrieval method embodiment based on row-column hybrid encrypted storage.

[0107] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned fast retrieval method based on row-column hybrid encryption storage.

[0108] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the fast retrieval method embodiment based on row-column hybrid encrypted storage, the description is relatively simple, and the relevant parts can be referred to the partial description of the fast retrieval method embodiment based on row-column hybrid encrypted storage.

[0109] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned fast retrieval method based on row-column hybrid encryption storage.

[0110] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer program embodiment, since it is basically similar to the fast retrieval method embodiment based on row-column hybrid encryption storage, the description is relatively simple, and the relevant parts can be referred to the partial description of the fast retrieval method embodiment based on row-column hybrid encryption storage.

[0111] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0113] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present specification.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A fast retrieval method based on row-column hybrid encrypted storage, characterized in that: include: According to the preset data storage principle, the acquired data information is stored, wherein the data information includes real data and data in the data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries the starting position information and data block index information corresponding to each data block; In response to receiving a search instruction, the data attribute block is filtered based on the data block index information to obtain a target data block, and the target data is retrieved from the real data based on the starting position information carried in the target data block.

2. The method according to claim 1, characterized in that The storing of the acquired data information includes: ordinary storage and encrypted storage; The storing of the acquired data information according to the preset data storage principle includes: Acquire data information, and divide the data information into at least one logical horizontal partition according to rows, wherein each of the logical horizontal partitions is composed of column blocks arranged in columns; When the amount of data in each of the column blocks meets a preset data amount threshold, determining maximum data value information and minimum data value information corresponding to each of the column blocks, and encrypting data except the maximum data value information and the minimum data value information; The data maximum value information and the data minimum value information are stored normally, and data other than the data maximum value information and the data minimum value information are stored encrypted.

3. The method according to claim 1, characterized in that The storing of the acquired data information includes: row storage and column storage; The storing of the acquired data information according to the preset data storage principle includes: Determining the update frequency of each target data in the data information, and in response to the target data being frequently updated, performing row storage on the target data; Determine the frequency of insertion of each target data in the data information, and in response to the target data being frequently inserted, store the target data in columns; in response to the target data being infrequently inserted, store the target data in rows; Determine the number of table columns corresponding to each target data in the data information, and in response to the number of table columns corresponding to the target data exceeding a first preset table column number threshold, perform column storage on the target data; According to the historical search instructions, the number of columns of the target data corresponding to the search instruction is determined. In response to the target data corresponding to the search instruction, if the number of corresponding table columns in the data information is less than a second preset table column number threshold, the target data is stored in columns.

4. The method according to claim 1, characterized in that: The method for determining data block index information corresponding to each data block carried in attribute information corresponding to a data attribute block includes: When writing a data block, the column storage data of each data block is indexed, and the maximum data value information and the minimum data value information carried in the index are stored in an unencrypted manner, wherein the index carries the maximum data value information, the minimum data value information, the column length information before encryption, the column length information after encryption, and the storage starting position information after encryption.

5. The method according to claim 1, characterized in that The filtering process of the data attribute block based on the data block index information to obtain a target data block, and retrieving the target data from the real data based on the starting position information carried in the target data block, includes: Determine the search data corresponding to the search instruction, determine the target data range corresponding to the data attribute block corresponding to the search data in the stored data information based on the data block index information, obtain the target data block, and filter other data attribute blocks except the target data block; Based on preset operation instructions, determine the target starting position information corresponding to the retrieved data in the target data block, jump to the column block corresponding to the target data block based on the target starting position information, and perform decryption processing to retrieve the target data in the real data, wherein the preset operation instructions include equal value operations and comparison operations.

6. The method according to claim 1, characterized in that The method further comprises: The structured data is stored according to a preset data storage principle to generate a storage file in a target storage format, wherein the target storage format is a JFILE format, and the JFILE format carries a special identifier; The data in the storage file is read through the storage engine.

7. The method according to claim 1, characterized in that The step of storing the acquired data information according to a preset data storage principle includes: According to the actual needs of users, the acquired data information is stored.

8. A fast retrieval device based on row-column hybrid encrypted storage, characterized in that: include: The storage module is configured to store the acquired data information according to a preset data storage principle, wherein the data information includes real data and data in a data attribute block; the real data and the data in the data attribute block are stored separately; the attribute information corresponding to the data attribute block carries the starting position information and data block index information corresponding to each data block; The retrieval module is configured to filter the data attribute block based on the data block index information in response to receiving a retrieval instruction to obtain a target data block, and retrieve the target data from the real data based on the starting position information carried in the target data block.

9. A computer device, characterized in that: The computer device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images