Efficient encrypted compressed file processing method, device and system based on memory mapping

Abstract

The invention discloses an efficient encrypted compressed file processing method, device and system based on memory mapping. The method comprises the following steps: a client sends a data acquisition request to a server, and acquires a data block which corresponds to the data acquisition request and is in a compressed state from the server; and the client locates a target data block from the data blocks in the compressed state as required, and only decompresses the target data block. The technical problems of low decompression efficiency and slow response speed when a plurality of small files need to be accessed randomly in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of file processing, and more specifically, to a method, apparatus, and system for processing efficient encrypted and compressed files based on memory mapping. Background Technology

[0002] In online education, qualification examinations, and other scenarios, question bank systems need to handle massive amounts of small files (such as questions, images, and audio). Traditional solutions often use standard compression formats (such as ZIP) for data packaging and distribution. For example, a solution using the standard ZIP format for data packaging and decompression is widely documented in the public domain; for instance, the international standard ".ZIP File Format Specification" can be consulted. Its core process is as follows:

[0003] Compression / packaging process: Each file in the question bank (such as question_001.txt, image_005.png) is compressed one by one using algorithms such as DEFLATE, and then packaged together with their metadata (filename, size before and after compression, offset within the package, etc.) into a ZIP file.

[0004] Decompression / Access Process: When accessing a file within the ZIP file is required, the decompression program must first read the central directory area of ​​the ZIP file to locate the target file's metadata. Then, it locates the starting position of the compressed data block for that file based on the offset. Finally, it reads the entire compressed data block into memory for complete decompression to obtain the target file's content. This process involves multiple I / O operations and complete data block decompression.

[0005] However, this general approach suffers from problems such as low decompression efficiency, high memory consumption, and significant security risks when dealing with scenarios involving frequent and random access to files within compressed archives. As the question bank becomes increasingly rich in content and updated more frequently, these shortcomings become even more pronounced.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] This invention provides a method, apparatus, and system for efficient encrypted compressed file processing based on memory mapping, which at least solves the technical problems of low decompression efficiency and slow response speed in the prior art when multiple small files need to be randomly accessed.

[0008] According to one aspect of the present invention, a method for processing efficient encrypted compressed files based on memory mapping is provided, comprising: a client sending a data acquisition request to a server and acquiring a data block in a compressed state corresponding to the data acquisition request from the server; the client locating a target data block from the compressed data block as needed and decompressing only the target data block.

[0009] According to another aspect of the present invention, a method for efficient encrypted compressed file processing based on memory mapping is also provided, applied on a server. The method includes: the server responding to a data acquisition request sent by a client, locating a data block in a compressed state corresponding to the data acquisition request according to preset file header and index area information; and sending the data block in a compressed state to the client.

[0010] According to another aspect of the present invention, a method for efficient encrypted compressed file processing based on memory mapping is also provided, comprising: a client sending a data acquisition request to a server; the server responding to the data acquisition request, locating a data block in a compressed state corresponding to the data acquisition request according to preset file header and index area information, and sending the data block to the client; after receiving the data block in a compressed state, the client decrypting the data block in a compressed state using a key corresponding to a key ID; the client mapping the decrypted data block in a compressed state to a local memory address space without decompressing the entire decrypted data block in a compressed state; when the client needs to read page data, it accesses the memory address space, and in response to a page error generated by accessing the memory address space, it calls a decompressor, and uses the decompressor to decompress only the target data block corresponding to the currently requested page data from the compressed state.

[0011] According to another aspect of the present invention, a high-efficiency encrypted compressed file processing system based on memory mapping is also provided, including a client, a server, and a file layer. The server is configured to, in response to a data acquisition request sent by the client, locate a compressed data block corresponding to the data acquisition request from the file layer according to the file header and index area information preset in the file layer, and send the compressed data block to the client. The client is configured to: after receiving the compressed data block, decrypt the compressed data block using a key corresponding to a key ID; map the decrypted compressed data block to a local memory address space without decompressing the entire decrypted compressed data block; when page data needs to be read, access the memory address space, and in response to a page error generated by accessing the memory address space, call a decompressor, and use the decompressor to decompress only the target data block corresponding to the currently requested page data from the compressed state.

[0012] In this embodiment of the invention, the client sends a data retrieval request to the server and retrieves a compressed data block corresponding to the data retrieval request from the server; the client locates the target data block from the compressed data block and decompresses only the target data block. This method solves the technical problems of low decompression efficiency and slow response speed in the prior art when multiple small files need to be randomly accessed. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0014] Figure 1 This is a flowchart of an optional memory-mapped, high-efficiency encrypted compressed file processing method according to an embodiment of the present invention;

[0015] Figure 2 This is an operation flowchart of an optional memory-mapped high-efficiency encrypted compressed file processing system according to an embodiment of the present invention;

[0016] Figure 3 This is a flowchart of another optional memory-mapped efficient encrypted compressed file processing method according to an embodiment of the present invention;

[0017] Figure 4 This is a flowchart of an optional client-side memory mapping decompression method according to an embodiment of the present invention;

[0018] Figure 5This is a flowchart of an optional, efficient encrypted compressed file processing method based on encryption and decryption according to an embodiment of the present invention;

[0019] Figure 6 A schematic diagram of the structure of a computer device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] According to an embodiment of the present invention, a method embodiment of an efficient encrypted compressed file processing method based on memory mapping is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] Figure 1 This is an efficient encrypted compressed file processing method based on memory mapping according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0024] In step S102, the client sends a data acquisition request to the server and obtains a compressed data block corresponding to the data acquisition request from the server.

[0025] First, the client sends the data acquisition request to the server. The server locates the compressed data block corresponding to the data acquisition request based on the preset file header and index area information, and sends the compressed data block to the client. The client uses the key with the corresponding key ID obtained from the server to decrypt the compressed data block, obtaining a decrypted data block that is still in a compressed state.

[0026] Next, the client directly maps the compressed data block to its local memory address space without decompressing the entire compressed data block.

[0027] In step S104, the client locates the target data block from the compressed data block and decompresses only the target data block.

[0028] For example, when the client needs to read page data, it determines the pointer corresponding to the page data in the memory address space and accesses the memory address corresponding to the pointer. The client reads the target data block corresponding to the page data from the memory address corresponding to the pointer and uses a decompressor to decompress only the target data block from its compressed state. For example, the client accesses the memory address corresponding to the pointer and reads the target data block corresponding to the page data from the memory address corresponding to the pointer. For example, in response to a page fault caused by accessing the memory address corresponding to the pointer, the client calls the decompressor, uses the decompressor to decompress only the target data block from its compressed state, and loads it into physical memory.

[0029] Compared to traditional ZIP packages that require decompressing the entire data block, the memory-mapped on-demand decompression technology of this invention significantly reduces I / O and CPU computation, achieving near-instantaneous data access. Furthermore, because it's on-demand decompression, only the decompressed portion of the data actually needed currently exists in memory, resulting in extremely high memory efficiency. Moreover, through built-in fine-grained key management, different access permissions can be set for questions of varying difficulty and knowledge points in the question bank, providing security far superior to ZIP packages with a single password. Finally, the dedicated, hierarchical index structure provided in this embodiment enables extremely fast location of any question or resource, making it ideal for random access scenarios like question banks.

[0030] This application also provides a high-efficiency encrypted compressed file processing system based on memory mapping. This system comprises a three-layer architecture: a dedicated format file layer, a server-side data scheduling layer, and a client-side memory-mapped decompression layer. To address the problems of traditional ZIP compression, this application abandons the ZIP container approach and designs a dedicated file format and processing architecture that integrates layered indexing, memory-mapped decompression, and encrypted transmission. Its core is to pre-classify and structure the data, and utilize memory mapping technology to achieve "on-demand decompression and instant access." In practical applications, it provides a high-speed, secure processing system and method for massive amounts of small files, such as question banks, based on memory mapping and a dedicated format.

[0031] The dedicated format file layer provides the underlying data organization and storage structure, including the file header, key management area, table index area, file index area, table data area, and file data area. The specific data structure will be described below and will not be repeated here.

[0032] The server-side data scheduling layer is used for data location, reading, and encrypted transmission, serving as an intermediate scheduling layer connecting the file layer and the client. The server first parses the client request (such as retrieving the question ID), then uses file and table indexes to locate the data offset in memory, initiating precise I / O reads only for the requested data blocks. Finally, it encrypts different data blocks using fine-grained keys and transmits them via a TLS secure link.

[0033] The client-side memory-mapped decompression layer enables efficient data access and on-demand decompression on the client side. Specifically, the client uses operating system APIs (such as mmap or CreateFileMapping) to directly map the compressed data blocks returned by the server into memory. Upon access, the operating system triggers page-level decompression, decompressing only the currently accessed 4KB page. This allows for on-demand decompression and instant access, achieving a zero-latency experience.

[0034] The system has a three-level acceleration architecture, and its specific acceleration process is as follows: Figure 2 As shown, it includes the following steps:

[0035] Step S202: Read the file header.

[0036] First, the 128-byte header at the beginning of the file is read, and information such as the magic number, version number, and offsets and lengths of each data area (key management area, index area, and data area) is parsed. The header acts as a "map," providing the foundation for parsing the entire file structure. The header structure is designed to be 127 bytes, but in this embodiment, only 81 bytes are used; the remaining bytes are reserved for future upgrades.

[0037] Next, direct memory mapping. Using the memory mapping mechanism provided by the operating system (such as mmap or CreateFileMapping), the file's index area and / or data area are mapped to the process's virtual address space. After mapping, the program can access the file content directly as if it were accessing memory, without the need for frequent read / write system calls.

[0038] Step S204: Obtain the index position.

[0039] Based on the starting offset and length of the "table index area" and "file index area" recorded in the file header, locate the starting position of the index area.

[0040] Step S206: Load only the index area.

[0041] First, only the index areas (table indexes and file indexes) are loaded into memory, without loading the large data areas, thus reducing the initial memory footprint.

[0042] Then, Protobuf deserialization is performed. The data in the index area is typically stored using an efficient serialization format such as Protobuf. The index data is deserialized into in-memory structured objects (such as TableIndexEntry and FileIndexEntry).

[0043] Step S208, locate the data.

[0044] An index table is created in memory to record metadata such as the position (offset, length) of each table or file in the data area, encryption status, and initialization vector (IV).

[0045] Locate by offset. Based on the requested data ID (such as question ID or file ID), query the memory index table to obtain the offset and length of the target data in the data area.

[0046] Skip irrelevant data, locate the target data directly based on the offset, skip irrelevant data blocks, and avoid reading redundant data.

[0047] If the data is encrypted, it is decrypted using the corresponding key and initialization vector (IV) in a streaming manner (e.g., AES / CTR mode). The decrypted data remains compressed. Streaming decompression is then used, decompressing only the data segment currently needed (e.g., a 4KB page), rather than the entire data block. The final result is usable target data (e.g., question text, image binary stream) for application use.

[0048] Figure 3 This is another efficient encrypted compressed file processing method based on memory mapping according to an embodiment of this application, applied in the above-mentioned file processing system, such as... Figure 3 As shown, the method includes the following steps:

[0049] Step S302: The client requests access to a specific question.

[0050] Step S304: The server-side data scheduler receives the request and obtains the data.

[0051] After receiving a data retrieval request from the client (such as getQuestion(12345)), the server parses the client's data retrieval request (request to access a specific question) and opens a special format file in the file layer in memory. The overall structure of this special format file is shown in Table 1.

[0052] Table 1

[0053]

[0054] In Table 1, A = 1158 + table index area length, B = A + file index area length, and C = B + table data area length.

[0055] In Table 1, the "file header" is fixed at 128 bytes and is used to provide a "map" for parsing the entire file, allowing for quick location of any region by calculating the offset.

[0056] The specific file header structure is shown in Table 2:

[0057] Table 2

[0058]

[0059] The data length occupied by the "Key Management Area" in Table 1 is indicated in the file header, and actually reserves 1030 bytes. The key is mainly used to implement fine-grained encryption; different table data areas or file data areas can be encrypted with different key IDs. The server can decide which keys to transmit or which data blocks to decrypt based on user permissions. The specific structure of the "Key Management Area" in Table 1 is shown in Table 3.

[0060] Table 3

[0061]

[0062] The entire file is memory-mapped, including its table index area and file index area. The table index entries (TableIndexEntry) in the memory table index area have variable lengths and are primarily used for quickly locating the metadata table. Their specific structure is shown in Table 4.

[0063] Table 4

[0064]

[0065] The file index entry DataFileIndexEntry in the in-memory file index area structure has a variable length and is mainly used for quickly locating the position of a specific file resource. Its specific structure is shown in Table 5:

[0066] Table 5

[0067]

[0068] By querying the index, the offset [O1, L1] of the metadata record for the title, such as ID=12345, in the table data area, and the offset [O2, L2] of its accompanying image in the file data area, are found. The server initiates only two disk I / Os, reading a data block of length L1 starting at O1 and a data block of length L2 starting at O2. In some embodiments, the server itself can also utilize memory mapping and on-demand reading to optimize its performance.

[0069] Step S306: Transmit the required data blocks.

[0070] The server transmits these two data blocks to the client via a TLS secure link. The data blocks remain encrypted throughout the transmission process.

[0071] Step S308: Decompress the client memory mapping.

[0072] First, after receiving the data block, the client decrypts it using the key corresponding to the key ID securely obtained from the server, and obtains the data block that is still in a compressed state after decryption.

[0073] Next, the client calls operating system APIs (such as mmap in Linux and CreateFileMapping in Windows) to map the compressed data blocks into its process's memory address space. This operation is very fast and does not actually decompress the data.

[0074] When an application needs to read the metadata of a question or display an image, it directly accesses the mapped memory address via a pointer. When the CPU accesses this address, a page fault occurs, which the operating system catches. The operating system's page fault handler then calls a dedicated decompressor to decompress only the currently requested page (e.g., 4KB) of data from its compressed state and load it into physical memory. Afterward, the application can seamlessly access this decompressed data as if it had always been in memory.

[0075] Specifically, such as Figure 4 As shown, the client-side memory-mapped decompression method includes the following steps:

[0076] S3080, Application Layer Data Request.

[0077] When an Android application (i.e., an application on the client side) needs to access specific data (such as metadata or images from a question bank), it initiates a request by calling a JNI method provided by the Native layer. This request explicitly specifies the location and size of the required data.

[0078] Step S3082, JNI interface forwarding.

[0079] The JNI interface acts as a bridge between Java and Native, seamlessly forwarding this data request to the Native code module.

[0080] Step S3084, memory mapping of the Native layer.

[0081] Native code calls operating system APIs (such as mmap in Linux) to map the compressed data blocks returned by the server into the process's virtual memory address space.

[0082] In step S3086, the file system returns the mapped address.

[0083] The file system returns a pointer (ptr) to the starting address of the mapped region. This method establishes the mapping relationship without immediately decompressing the data, thus achieving extremely high efficiency.

[0084] Step S3087, On-demand computation and data access.

[0085] When a specific access request reaches the Native layer, it performs the following operations: 1) Calculate the offset: Based on the position and size parameters in the request, and combined with the preset data index structure, calculate the precise offset of the target data in the mapped memory area. 2) Direct memory access: Access the data at ptr[offset] directly through the memory pointer.

[0086] Step S3088: Unzip as needed.

[0087] Because the data pointed to by ptr[offset] is still compressed, a page fault is immediately triggered when the CPU attempts to access that memory address. The operating system's memory management unit catches this error and activates its page fault handler. This handler calls a dedicated decompressor corresponding to the compression algorithm. This decompressor only decompresses the compressed data page (e.g., a 4KB unit) touched by the current request and loads it into physical memory. After this, the application's access proceeds normally, and the native layer then returns the data pointer or copied data to the upper-layer Android application via the JNI interface.

[0088] Step S3089: The application layer uses the data.

[0089] The application receives the requested data and uses it for interface display or logical calculations, with no delay perceived by the user.

[0090] This embodiment breaks down the large amount of time-consuming decompression calculations into countless tiny, on-demand decompression operations that are efficiently scheduled by the underlying operating system, thereby achieving an "instant access" experience.

[0091] In summary, the embodiments of this application have the following beneficial effects: 1) Extremely high decompression speed and low latency: Compared with traditional ZIP which requires decompressing the entire data block, the memory-mapped on-demand decompression technology of this invention greatly reduces I / O and CPU computation, achieving near-instantaneous data access. 2) Extremely low memory usage: Due to on-demand decompression, only the decompressed form of a very small portion of the data that is actually needed to be accessed exists in memory, resulting in extremely high memory efficiency. 3) Enhanced security: Through built-in fine-grained key management, different access permissions can be set for questions of different difficulties and knowledge points in the question bank, making the security far higher than that of ZIP packages with a single password. 4) Excellent random access performance: The dedicated, hierarchical index structure makes it extremely fast to locate any question or resource, which is very suitable for random access scenarios such as question banks.

[0092] The following is a performance comparison between this embodiment and the traditional solution:

[0093]

[0094] The following is a comparison of the security of this embodiment with traditional solutions.

[0095]

[0096] Figure 5 This is a method for efficient encrypted and compressed file processing based on encryption and decryption according to an embodiment of this application, applied in the above-mentioned file processing system, such as... Figure 5 As shown, the method includes the following steps:

[0097] Step S502, Equipment Authentication.

[0098] The client (Receiver) sends device information to the server (sender), which includes encrypted device ID, nonce, and timestamp.

[0099] Step S504: Obtain the key and download address.

[0100] After verifying the information, the server locates the corresponding public or private key information through the device code, generates a session key for accessing the question bank package, and returns this key along with the corresponding temporary download link (OBS link) to the client. This temporary download link has an expiration date or is a one-time use feature, used to securely download the question bank package data from Object Storage Service (OBS), where OBS is the server.

[0101] Step S506: Request to download the question bank package.

[0102] The client sends a question bank package download request to OBS (File System) based on the received temporary download link.

[0103] Step S508: Send the question bank package data for segmented download.

[0104] OBS returns encrypted and compressed question bank data upon request. To improve transmission efficiency, the question bank data can be transmitted in segments (i.e., downloaded in chunks).

[0105] Step S510: Data verification and concurrent decompression.

[0106] 1) Decryption process.

[0107] Using the key obtained in the above steps, each segment of data is decrypted to obtain a segment of data block that is still in a compressed state.

[0108] 2) Memory mapping operations.

[0109] The client calls the memory mapping interface provided by the operating system (such as mmap() in Linux or CreateFileMapping() in Windows) to map the decrypted compressed data blocks into the virtual memory address space of the local process. This step does not actually decompress the data; it only establishes a logical mapping relationship, allowing subsequent access to be performed directly at the memory level.

[0110] 3) Unzip as needed.

[0111] When an application or question bank engine accesses the corresponding question or resource data in the mapped area, the CPU will trigger a Page Fault. The operating system catches the exception and calls a dedicated memory-mapped decompression engine to decompress only the currently needed pages (such as 4KB data blocks) in real time, and loads the decompression results into physical memory, thereby avoiding the decompression of the entire question bank package.

[0112] To improve processing efficiency, the client can establish memory-mapped regions for multiple data segments in parallel and execute the decryption and on-demand decompression processes in parallel through multi-threading or asynchronous I / O mechanisms, achieving efficient and low-latency access to the question bank package. Through this memory-mapped on-demand decompression technology, the client can achieve "instant access" without performing overall decompression, significantly reducing I / O load and memory consumption while maintaining the security and integrity of data transmission.

[0113] This application also provides a high-efficiency encrypted compressed file processing device based on memory mapping, set on a client side, including: a data acquisition module configured to send a data acquisition request to a server and acquire a compressed data block corresponding to the data acquisition request from the server; and a decompression module configured to locate a target data block from the compressed data block as needed and decompress only the target data block.

[0114] This application also provides a high-efficiency encrypted compressed file processing device based on memory mapping, set on the server side, including: a positioning module, configured to respond to a data acquisition request sent by the client, and locate the data block in a compressed state corresponding to the data acquisition request according to preset file header and index area information; and a sending module, configured to send the data block in a compressed state to the client.

[0115] This application also provides a high-efficiency encrypted compressed file processing system based on memory mapping, including a client, a server, and a file layer. The server is configured to respond to a data acquisition request sent by the client, locate the compressed data block corresponding to the data acquisition request from the file layer according to the file header and index area information preset in the file layer, and send the compressed data block to the client. The client is configured to: after receiving the compressed data block, decrypt the compressed data block using a key corresponding to the key ID; map the decrypted compressed data block to the local memory address space without decompressing the entire decrypted compressed data block; when page data needs to be read, access the memory address space, and in response to a page error caused by accessing the memory address space, call the decompressor, and use the decompressor to decompress only the target data block corresponding to the currently requested page data from the compressed state.

[0116] It should be noted that the memory-mapping-based high-efficiency encrypted compressed file processing device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the memory-mapping-based high-efficiency encrypted compressed file processing device and the method embodiment of the memory-mapping-based high-efficiency encrypted compressed file processing device provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0117] Figure 6 A schematic diagram of a computer device suitable for implementing embodiments of the present disclosure is shown. It should be noted that... Figure 6 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0118] like Figure 6 As shown, the computer device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0119] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0120] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A memory mapping based efficient encrypted compressed file processing method, applied to a client, characterized in that, The method comprises: The client sends a data acquisition request to the server and acquires a data block in a compressed state corresponding to the data acquisition request from the server; The client locates a target data block from the data block in the compressed state as needed and only decompresses the target data block.

2. The method of claim 1, wherein, After acquiring the data block in the compressed state corresponding to the data acquisition request from the server, the method further comprises that the client directly maps the data block in the compressed state to a local memory address space without overall decompressing the data block in the compressed state.

3. The method of claim 2, wherein, The client sends a data acquisition request to the server and acquires a data block in a compressed state corresponding to the data acquisition request from the server, comprising: The client sends the data acquisition request to the server, wherein the server locates a data block in a compressed state corresponding to the data acquisition request according to preset file header and index area information and sends the data block to the client; The client uses a key corresponding to a key ID acquired from the server to decrypt the data block in the compressed state to obtain a decrypted data block still in the compressed state.

4. The method of claim 2, wherein, The client locates a target data block from the data block in the compressed state as needed and only decompresses the target data block, comprising: The client determines a pointer corresponding to page data in the memory address space by querying index information when the page data needs to be read; The client accesses a memory address corresponding to the pointer and only decompresses the target data block from the compressed state by using a decompressor.

5. The method of claim 4, wherein, The client accesses a memory address corresponding to the pointer and only decompresses the target data block from the compressed state by using a decompressor, comprising: The client accesses a memory address corresponding to the pointer; The client calls a decompressor and only decompresses the target data block corresponding to the page data of the current request from the compressed state by using the decompressor and loads the target data block into a physical memory in response to a page error generated by accessing the memory address corresponding to the pointer.

6. A memory mapping-based efficient encrypted compressed file processing method, applied to a server, characterized in that, Comprising: The server locates a data block in a compressed state corresponding to a data acquisition request of a client according to preset file header and index area information in response to the data acquisition request of the client; The server sends the data block in the compressed state to the client.

7. An efficient encrypted compressed file processing method based on memory mapping, characterized in that The client sends a data acquisition request to the server; The server locates a data block in a compressed state corresponding to the data acquisition request according to preset file header and index area information in response to the data acquisition request and sends the data block in the compressed state to the client; The client receives the data block in the compressed state and uses a key corresponding to a key ID to decrypt the data block in the compressed state; The client maps the decrypted data block in compressed state to a local memory address space without decompressing the decrypted data block in compressed state as a whole; The client accesses the memory address space when page data is needed to be read, and in response to a page fault caused by accessing the memory address space, invokes a decompressor and uses the decompressor to only decompress the target data block corresponding to the page data currently requested from the compressed state.

8. A memory-mapping based efficient encrypted compressed file processing apparatus, disposed at a client, characterized in that, Comprise: a data acquisition module configured to send a data acquisition request to a server and acquire a data block in compressed state corresponding to the data acquisition request from the server; a decompression module configured to locate a target data block from the data block in compressed state on demand and only decompress the target data block.

9. A memory-mapping based efficient encrypted compressed file processing apparatus, disposed at a server, comprising: Comprise: a locating module configured to locate a data block in compressed state corresponding to a data acquisition request according to preset file header and index area information in response to the data acquisition request sent by a client; a sending module configured to send the data block in compressed state to the client.

10. An efficient encrypted compressed file processing system based on memory mapping, comprising a client, a server and a file layer, characterized in that the server is configured to locate a data block in compressed state corresponding to a data acquisition request from the file layer according to preset file header and index area information of the file layer in response to the data acquisition request sent by the client, and send the data block in compressed state to the client; the client is configured to: after receiving the data block in compressed state, use a key corresponding to a key ID to decrypt the data block in compressed state; map the decrypted data block in compressed state to a local memory address space without decompressing the decrypted data block in compressed state as a whole; when page data is needed to be read, access the memory address space, and in response to a page fault caused by accessing the memory address space, invoke a decompressor and use the decompressor to only decompress the target data block corresponding to the page data currently requested from the compressed state.

Images