Index-based question bank data on-demand access method and system

Abstract

The invention discloses an index-based question bank data on-demand access method and system. The method comprises the steps that a question bank package is obtained from a file layer storing question bank data in response to a question bank package obtaining request of a client side, and the file layer is a file system of a preset file format, a modern file system supporting transparent compression or a single-file embedded database; and the client locates a target data block from the question bank package as required to obtain a target file. According to the method and the device, the technical problem of low decompression efficiency of a traditional compressed file format in a scene needing efficient random access is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent education, and in particular, to an index-based question bank data on-demand access method and system. BACKGROUND

[0002] In online education, qualification examination, digital publishing and other scenarios, the question bank system usually needs to store and process a large number of small files, such as question texts, pictures, audios and analysis files, etc. The traditional file packaging and distribution method usually uses a general compression format (such as ZIP, RAR, etc.) to reduce the transmission volume and facilitate archive management.

[0003] The existing ZIP compression scheme generally has the following characteristics in use: in the compression phase, each file is compressed one by one by DEFLATE algorithm and packaged uniformly; in the decompression phase, any file in the package needs to read the central directory area first, parse the metadata and find the offset of the target file, and then read the corresponding compressed data block into the memory for complete decompression. Although this mechanism has clear structure and strong compatibility, it is not suitable for high-concurrency scenarios with frequent random access.

[0004] With the increasing size and faster update frequency of the question bank content, the limitations of this general solution become more and more prominent. First, when accessing any file in the package, the corresponding compressed block needs to be completely decompressed, resulting in low decompression efficiency and slow response speed, especially when multiple small files need to be randomly read, the I / O and CPU overhead increases exponentially. Second, the entire data block needs to be loaded into the memory before decompression, resulting in high memory occupation and making it difficult to run on terminal devices with limited memory. Third, the encryption mechanism provided by the standard ZIP format (such as ZIP 2.0 traditional encryption) has known security weaknesses, and even if AES encryption is used, a unified key is usually used, which cannot achieve fine-grained access control. In addition, the ZIP structure design itself is not optimized for random access, and reading a specific segment inside the compressed package must rely on the central directory for indexing, so the random access performance is poor.

[0005] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0006] The embodiments of the present application provide an index-based question bank data on-demand access method and system to at least solve the technical problem of low decompression efficiency of traditional compressed file format in scenarios requiring efficient random access.

[0007] According to an aspect of the embodiments of the present application, there is provided an index-based test library data on-demand access method, comprising: in response to a test library package acquisition request from a client, acquiring a test library package from a file layer storing test library data, wherein the file layer is a file system of a preset file format, a modern file system supporting transparent compression, or a single-file embedded database; and the client on-demand locates a target data block from the test library package to obtain a target file.

[0008] According to another aspect of the embodiments of the present application, there is also provided an index-based test library data on-demand access system, comprising: a test library acquisition module configured to acquire a test library package from a file layer storing test library data in response to a test library package acquisition request, wherein the file layer is a file system of a preset file format, a modern file system supporting transparent compression, or a single-file embedded database; and a file acquisition module configured to, in response to a file acquisition request, on-demand locate a target data block from the test library package by the client to obtain a target file.

[0009] In the embodiments of the present application, in response to a test library package acquisition request from a client, a test library package is acquired from a file layer storing test library data, wherein the file layer is a file system of a preset file format, a modern file system supporting transparent compression, or a single-file embedded database; and the client on-demand locates a target data block from the test library package to obtain a target file. Through the above scheme, the technical problem of low decompression efficiency of a traditional compressed file format in a scenario requiring efficient random access is solved. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0011] Figure 1 is a flowchart of an optional index-based test library data on-demand access method according to an embodiment of the present application;

[0012] Figure 2 is a flowchart of an optional memory mapping-based test library data on-demand access method according to an embodiment of the present application;

[0013] Figure 3 is a flowchart of another optional memory mapping-based test library data on-demand access method according to an embodiment of the present application;

[0014] Figure 4 is an optional test library data on-demand access method based on block compression and local index according to an embodiment of the present application;

[0015] Figure 5is a flow chart of a data on-demand access method based on modern file system characteristics according to an embodiment of the present application;

[0016] Figure 6 is a flow chart of a data on-demand access method based on a lightweight embedded database according to an embodiment of the present application;

[0017] Figure 7 is a structural schematic diagram of an optional index-based question bank data on-demand access system according to an embodiment of the present application;

[0018] Figure 8 A structural schematic diagram of a computer device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to make the technical personnel in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the present application.

[0020] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0021] According to an embodiment of the present application, a method embodiment of an index-based question bank data on-demand access method is provided. It should be noted that the steps shown in the flow chart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flow chart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0022] Figure 1 is an index-based question bank data on-demand access method according to an embodiment of the present application, as shown in Figure 1 the method comprises the following steps:

[0023] Step S102: In response to the question bank package retrieval request, retrieve the question bank package from the file layer storing the question bank data, wherein the file layer is a file system with a preset file format, a modern file system that supports transparent compression, or a single-file embedded database.

[0024] In step S104, in response to the file retrieval request, the client locates the target data block from the question bank package as needed and obtains the target file.

[0025] The method provided in this embodiment will be described in detail below for four different scenarios.

[0026] 1) The file layer is a file system with a preset file format, and the object being decompressed is a small fragment pushed by the server.

[0027] When the file layer is a file system with a preset file format, the question bank package is stored in a preset file format structure, wherein the preset file format structure includes: a file header, a key management area, a table index area, a file index area, a table data area, and a file data area, wherein the starting offset and length of the table index area, the file index area, the table data area, and the file data area are variable.

[0028] The client sends a device authentication request to the server. In response to the device authentication request, the server sends a temporary download link to the client. The client then sends a request to the file layer to obtain the question bank package corresponding to the temporary download link. In response to the question bank package acquisition request, the file system of the file layer sends the corresponding question bank package to the client.

[0029] The client receives the compressed data block and decrypts it using the key corresponding to the key ID; it then maps the decrypted compressed data block to the local memory address space without decompressing the entire decrypted compressed data block.

[0030] When page data needs to be read, the memory address space is accessed based on a preset file format. In response to a page error caused by accessing the memory address space, the decompressor is invoked. The decompressor is used to decompress only the target data block corresponding to the currently requested page data from the compressed state to obtain the target file corresponding to the file acquisition request.

[0031] 2) The file layer is a file system with a preset file format, and the decompression object is a data block in a local file.

[0032] When the file layer is a file system with a preset file format, the question bank package stores the question bank data in a preset file format structure, wherein the preset file format structure includes: a file header, a key management area, a table index area, a file index area, a table data area, and a file data area.

[0033] Unlike the above scenario, in this scenario, multiple files in the question bank package are stored in the table data area and / or the file data area in the form of multiple independently compressed data blocks of fixed size or logically grouped. The index information in the table index area and the file index area records the position of each file in the total data area formed by the table data area and the file data area, as well as the block ID, intra-block offset, and compressed data length of each file.

[0034] The client sends a device authentication request to the server. In response to the device authentication request, the server sends a temporary download link to the client. The client then sends a request to the file layer to obtain the question bank package corresponding to the temporary download link. The file layer then sends the question bank package to the client.

[0035] In response to the application's file retrieval request, the client locates the compressed data block in the question bank package corresponding to the file retrieval request based on the location of the total data area, block ID, intra-block offset, and compressed data length; reads the compressed data block into a memory buffer; and uses a decompression function to decompress the compressed data block in the memory buffer to obtain the target file corresponding to the file retrieval request.

[0036] 3) The file layer is a modern file system that supports transparent compression.

[0037] The question bank data is stored directly as uncompressed raw files on a disk partition formatted as APFS or ZFS, and the file system of the file layer automatically compresses the question bank data.

[0038] The client sends a file retrieval request to the file layer. In response to the file retrieval request, the file system of the file layer automatically locates the disk block where the target data block is located, reads the data on the disk block into memory, automatically decompresses the data on the disk block read into memory, and returns the decompressed data to the client. The client receives the decompressed data as the target file.

[0039] 4) The file layer is a single-file embedded database.

[0040] In the case where the file layer is a single-file embedded database, the question bank package is stored in the database as a BLOB, consisting of the metadata of each file and the compressed binary data itself.

[0041] The client sends a file retrieval request to the file layer. In response, the file layer reads data from the corresponding BLOB field in the database as the target data block and sends it to the client. The client decompresses the data in the BLOB field in memory to obtain the target file.

[0042] In the case where the file layer is a single-file embedded database, the question bank package is stored in the database as a BLOB, consisting of the metadata of each file and the compressed binary data itself.

[0043] Figure 2 This is an embodiment of the present invention regarding a method for on-demand access to question bank data based on memory mapping, such as... Figure 2 As shown, the method includes the following steps:

[0044] Step S202: Construct the file structure at the file layer.

[0045] The question bank package in this embodiment adopts a specific hierarchical file structure, mainly including the following six logical areas:

[0046] 1) File header

[0047] The file header records metadata such as the file format version number, the starting offset and length of each region, and global flag information, serving as the index entry point for the entire file structure. By reading the file header, the client can directly locate the position of any logical region without decompression.

[0048] 2) Key Management Area

[0049] The key management area stores the keys or key identifiers used for encryption of different data areas. Each data area can be encrypted using a different key, enabling fine-grained security control. The key area includes the key type (internal, general, or specific), algorithm identifier (such as AES / CTR mode), and the encrypted key content.

[0050] 3) Table index area

[0051] The table index area is used to record index information for table-type data (such as subject tables, knowledge point tables, and question tables) in the question bank. Each table index entry contains fields such as table type, number of records, data area offset, data length, encryption flag, and initialization vector.

[0052] 4) File Index Area

[0053] The file index area is used to record the location information of resource files (such as question stems, images, and parsing files) in the question bank. Each index entry corresponds to a file, including the file type, associated question ID, data offset, data length, encryption flag, and initialization vector.

[0054] 5) Table data area and file data area

[0055] The table data area and the file data area store compressed and encrypted table data and resource file data, respectively. Both are variable-length areas, and the data blocks within them are arranged in logical order, allowing for precise location via indexes.

[0056] With the above structure, the entire question bank package forms a self-describing binary container file, which supports cross-platform transmission and facilitates fast parsing and partial reading.

[0057] Based on the above data structure, compression algorithms (such as DEFLATE and ZSTD) are used to compress the question bank data. Then, according to the key system defined in the key management area, the compressed data is encrypted. Different data areas can use different keys to achieve hierarchical security control. Next, corresponding index records are generated based on the physical offset and length of the compressed data and written to the table index area and file index area. Finally, the file header, key management area, index area, and data area are written sequentially to form a complete question bank package file.

[0058] For details on the specific implementation of the data structure, please refer to the applicant's Chinese patent application No. 2025117177971, entitled "Method and System for On-Demand Access to Question Bank Data Based on Index", the entire contents of which are incorporated herein by reference and will not be repeated here.

[0059] Step S204: Obtain the question bank package.

[0060] There are two ways to obtain the question bank package: one is through a link, and the other is by having the server provide the data directly.

[0061] Method 1: During initialization, the client device sends an authentication request (i.e., a question bank retrieval request) to the server. After verifying the device's identity, the server generates a temporary download link or authorization token and distributes the temporary download link via a secure channel (such as TLS / HTTPS). The client then downloads the encrypted and compressed question bank package from the file layer based on the temporary download link.

[0062] Method 2: The client device sends an authentication request (i.e., a question bank retrieval request) to the server during initialization. After verifying the device's identity, the server retrieves the question bank package from the file layer and sends it to the client.

[0063] Step S206: File decryption and memory mapping.

[0064] After receiving the question bank package, the client first reads the offset information of the key management area from the file header, parses and loads the key list.

[0065] When accessing database data is required, the client uses the corresponding key ID to select the appropriate key to decrypt the encrypted data block, obtaining binary data that is still in a compressed state.

[0066] Subsequently, the client calls the memory mapping interface provided by the operating system (such as mmap() in Linux or CreateFileMapping() in Windows) to directly map the decrypted compressed data into the virtual address space of the local process. This process does not perform overall decompression; it only establishes the mapping relationship between memory addresses and file data.

[0067] Step S208: Locate and decompress locally as needed.

[0068] When an application requests access to a specific question or file (e.g., question ID=12345), the client determines the offset and length of the file in the data area (e.g., [O1, L1]) by querying the table index and file index. If the accessed page data has not yet been loaded when the program accesses the mapped address, the operating system will trigger a "Page Fault" event. At this time, the system calls the decompression module to perform partial decompression only on the compressed data block corresponding to the current page and loads the decompression result into physical memory.

[0069] The application can then directly read the decompressed page data as if the file had always existed in memory, thus achieving instant access.

[0070] During access, the client can decrypt and access only the authorized data based on the key ID and user permission policy in the key management area. Unauthorized portions remain encrypted to prevent unauthorized reading. Through this method, different users can access different ranges of question bank data according to their permission levels, achieving a dual guarantee of security and performance.

[0071] Figure 3 This is another method for on-demand access to question bank data based on memory mapping according to an embodiment of the present invention. The difference between this embodiment and the above embodiments lies in the server-side processing procedure, such as... Figure 3 As shown, the method includes the following steps:

[0072] Step S302: Construct the file structure at the file layer.

[0073] Similar to step S202, it will not be described again here.

[0074] Step S304: The server processes the data.

[0075] When the server receives a file retrieval request from the client (e.g., getQuestion(12345)), it parses the request, retrieves the question bank package in a specific format from the file layer, and maps its table index area and file index area in memory. By querying the index, it finds the offset [O1, L1] of the metadata record of question ID=12345 in the table data area and the offset [O2, L2] of its accompanying image in the file data area. The server initiates only two disk I / O operations, reading a data block of length L1 starting at O1 and a data block of length L2 starting at O2.

[0076] The server transmits these two data blocks to the client via a TLS secure link. The data blocks remain encrypted throughout the transmission process.

[0077] Step S306: The client performs the processing.

[0078] After receiving the data block, the client decrypts it using the key corresponding to the key ID obtained securely from the server, and obtains the data block that is still in a compressed state.

[0079] The client calls operating system APIs (such as mmap on Linux and CreateFileMapping on Windows) to map compressed data blocks into its process's memory address space. This operation is very fast and does not actually decompress the data. When the application needs to read the question's metadata or display an image, it directly accesses the mapped memory address via a pointer. When the CPU accesses this address, a page fault occurs.

[0080] The operating system's page fault handler invokes a dedicated decompressor to decompress only the currently requested page (e.g., 4KB) of data from its compressed state and load it into physical memory. Afterward, applications can seamlessly access this decompressed data as if it had always been in memory.

[0081] Figure 4 This invention discloses an on-demand access method for question bank data based on block compression and local indexing, according to an embodiment of the present invention. Unlike the embodiments described above, this embodiment employs a block compression and local indexing scheme, applying server-side scheduling from the online architecture to local files. Specifically, a dedicated file format is optimized. Instead of treating the entire file layer's data area as a single, continuous compressed block, this embodiment divides it into multiple independently compressed data blocks of fixed size or logically grouped (e.g., by knowledge point chapters). Simultaneously, the local index needs to record which data block each file is located in and its offset within that block.

[0082] like Figure 4As shown, the method includes the following steps:

[0083] Step S402, preprocessing.

[0084] During the data packaging stage, a large number of small files are grouped and packaged into multiple independently compressed data blocks (.dat.001, .dat.002, ...).

[0085] Step S404, index the record.

[0086] The local index (which is still the file index area in the above embodiment) not only needs to record the approximate location of the file in the total data area, but also needs to accurately record {block ID, offset within the block, and compressed data length}.

[0087] Step S406, Client access.

[0088] The application requests file A. It queries the local index and determines that file A is located in data block -003, at offset 0x500, and has a length of 50KB. The client directly opens the data block -003 file and uses a seek operation to locate position 0x500. Only this 50KB of compressed data is read into a memory buffer. Then, the decompression function is called to decompress this 50KB of data, obtaining the original content of file A.

[0089] With the above Figure 2 Compared to the corresponding embodiments, both embodiments achieve "on-demand decompression," avoiding the decompression of the entire large compressed file. The core idea of ​​both is "precise data location through fine-grained indexing." However, this embodiment differs from the above embodiments in that the decompression unit changes from "a small fragment pushed by the server" to "a data block in a local file." The performance bottleneck shifts from network I / O to local storage I / O. Although this embodiment completely abandons memory mapping and reverts to the traditional read-decompression process, it significantly reduces the amount of data that needs to be decompressed each time through block decompression. This results in the following advantages: simple implementation, good cross-platform compatibility, and flexible control of granularity.

[0090] However, this embodiment still requires explicit decompression of small data blocks, and its performance is inferior to memory mapping. It is suitable for game decompression packages with larger file types. At the same time, due to the splitting of files during the decompression process, file splicing needs to be handled, which results in slightly lower efficiency than the above embodiment.

[0091] Figure 5This is a data on-demand access method based on the characteristics of modern file systems according to an embodiment of the present invention. The difference between this embodiment and the previous embodiments is that this embodiment directly utilizes the characteristics of the operating system's file system to avoid decompression; that is, it no longer uses a custom compression container format, but directly uses a modern file system that supports transparent compression, such as APFS or ZFS. The question bank data is stored directly as uncompressed raw files, and the file system automatically performs compression / decompression in the background.

[0092] like Figure 5 As shown, the method includes the following steps:

[0093] Step S502, data deployment.

[0094] Copy the entire question bank (tens of thousands of files) directly to a disk partition formatted as APFS.

[0095] Step S504: Perform the access.

[0096] When the application reads / QuestionBank / image_12345.png through the standard file interface, the file system automatically locates the disk block where the file is located, reads it into memory, automatically decompresses it, and then returns it to the application.

[0097] and Figure 2 The similarities between the corresponding embodiments are that both are "on-demand decompression," transparent to the application, and feel like accessing a regular file. The difference lies in that this embodiment is a system-level solution, not an application-level one. This embodiment completely bypasses custom file formats and indexes. However, its disadvantages include poor cross-platform compatibility (Windows' NTFS compression does not support this feature), and encryption and permission management rely on the file system's own functionality, making it less flexible than custom solutions. Furthermore, while this embodiment is completely transparent to the application, requiring no additional logic development, it suffers from strong platform dependence, weak encryption and permission control, and, like traditional zip decompression methods, cannot perform concurrent and efficient reading.

[0098] Figure 6 This invention discloses a method for on-demand data access based on a lightweight embedded database. This embodiment transforms a resource management problem into a data management problem. Specifically, this embodiment uses a lightweight, single-file embedded database to manage all question bank resources. The metadata (filename, type, etc.) of each small file and the compressed binary data itself are stored as BLOBs in the database table.

[0099] like Figure 6 As shown, the method includes the following steps:

[0100] Step S602: The client requests file A.

[0101] When a client needs to access a target file A in the problem library package, it sends a file retrieval request to the system.

[0102] Step S604: Perform a query operation.

[0103] Based on a pre-built index structure (such as an offset table, segment table, directory tree, etc.), the location and length of the compressed data block corresponding to file A in memory are quickly located within the question bank package that has already been loaded or mapped into memory. This query process occurs in memory, reducing I / O overhead.

[0104] Step S606: Read data.

[0105] Using memory-mapped files or a one-time loading strategy, the system directly reads the target compressed data blocks from the data buffer that has been loaded into memory, according to the offset. After the data blocks are aggregated, the compressed file data is obtained.

[0106] Step S608: Decompress.

[0107] The client decompresses the target compressed data block in memory. Decompression is an explicitly invoked logical step, rather than being automatically triggered by the file system or database; decompression only applies to the target compressed data block, not the entire question bank package. This approach achieves "on-demand decompression," saving CPU resources and significantly reducing latency when dealing with large data volumes.

[0108] This embodiment and Figure 2 The examples shown share the commonality of achieving efficient random access and leveraging the indexing advantages of databases. The difference lies in that the data storage and indexing medium in this implementation changes from a custom binary format to a standard database file. The decompression action occurs after retrieving the BLOB from the database; it is an explicit step, rather than an implicit behavior triggered by the operating system. Furthermore, this embodiment is completely transparent to the application, requiring no additional logic development, but it suffers from strong platform dependency and weak encryption and access control.

[0109] This application also provides an index-based on-demand access system for question bank data, such as... Figure 7 As shown, it includes: a question bank acquisition module 72 and a file acquisition module 74.

[0110] The question bank acquisition module 72 is configured to, in response to a question bank package acquisition request, acquire a question bank package from a file layer storing question bank data. The file layer can be a file system with a preset file format, a modern file system supporting transparent compression, or a single-file embedded database. Specifically, in response to a question bank package loading request (question bank package acquisition request), the question bank acquisition module 72 loads the compressed file package (or encrypted compressed package) storing question bank data at once or maps it to memory via memory mapping. It also constructs or loads a corresponding index structure for quick file location. In this embodiment, the question bank package can be: a custom binary file format; a modern file system supporting transparent compression; or a memory-mapped container file with a single-file structure.

[0111] The file acquisition module 74 is configured to, in response to a file acquisition request, locate the target data block from the question bank package as needed and obtain the target file. Specifically, in response to the file acquisition request, it locates the target compressed block in the memory-mapped region according to the index; decompresses it in memory as needed; and finally obtains the target file content that can be used directly.

[0112] It should be noted that the index-based on-demand access system for question bank data provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the index-based on-demand access system for question bank data provided in the above embodiments and the index-based on-demand access method embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0113] Figure 8 A schematic diagram of a computer device suitable for implementing embodiments of the present disclosure is shown. In this embodiment, the memory portion (RAM) is used to store the mapped question bank package and index information; the CPU performs index location and memory decompression operations; and the storage portion is used to persistently store the question bank package file.

[0114] It should be noted that, Figure 8 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0115] like Figure 8As shown, the computer device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0116] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0117] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An index-based on-demand access method for item bank data, characterized by, The method comprises: in response to a question bank package acquisition request, acquiring a question bank package from a file layer storing question bank data, wherein the file layer is a file system in a preset file format, a modern file system supporting transparent compression, or a single-file embedded database; in response to a file acquisition request, a client acquires a target file by locating a target data block from the question bank package on demand.

2. The method of claim 1, wherein, In the case where the file layer is a file system having a preset file format, the question bank package is stored in a preset file format structure, wherein the preset file format structure comprises a file header, a key management area, a table index area, a file index area, a table data area, and a file data area, and the starting offset and length of the table index area, the file index area, the table data area, and the file data area are variable.

3. The method of claim 2, wherein, The client acquires a target file by locating a target data block from the question bank package on demand, comprising: The client receives the question bank package and decrypts the question bank package using a key corresponding to a key ID; The decrypted question bank package is mapped to a local memory address space without decompressing the decrypted question bank package as a whole; When page data needs to be read, the memory address space is accessed based on the preset file format, and a decompressor is called in response to a page error generated by accessing the memory address space, so that only a target data block corresponding to the page data currently requested is decompressed from a compressed state by using the decompressor, thereby obtaining the target file.

4. The method of claim 1, wherein, In the case where the file layer is a file system having a preset file format, the question bank package is stored in a preset file format structure, wherein the preset file format structure comprises a file header, a key management area, a table index area, a file index area, a table data area, and a file data area, and the question bank package is stored in the form of a plurality of files divided into a plurality of fixed-size or logically grouped, independently compressed data blocks in the table data area and / or the file data area, and the index information in the table index area and the file index area records the position of each file in the total data area composed of the table data area and the file data area, as well as the block ID, block offset, and compressed data length of each file.

5. The method of claim 4, wherein, The client acquires a target file by locating a target data block from the question bank package on demand, comprising: In response to a file acquisition request, the client locates a data block in a compressed state corresponding to the file acquisition request in the question bank package based on the position of the total data area, the block ID, the block offset, and the compressed data length; The data block in the compressed state is read into a memory buffer, and the data block in the compressed state in the memory buffer is decompressed by using a decompression function to obtain a target file corresponding to the file acquisition request.

6. The method of claim 1, wherein, In the case where the file layer is a modern file system supporting transparent compression, the question bank data is directly stored as uncompressed raw files on a disk partition formatted as an APFS format or a ZFS format, and the file system of the file layer automatically compresses the question bank data.

7. The method of claim 6, wherein, The client obtains a target file from the question bank package on demand by locating a target data block, comprising: the client sends the file acquisition request to the file layer, the file system of the file layer automatically locates to the disk block where the target data block is located in response to the file acquisition request, reads the data on the disk block into the memory, then automatically decompresses the data on the disk block read into the memory, and returns the decompressed data to the client; the client receives the decompressed data as the target file.

8. The method of claim 1, wherein, In the case that the file layer is a single-file embedded database, the question bank package is stored in the database in the form of BLOB, with the metadata of each file and the compressed binary data itself.

9. The method of claim 8, wherein, The client obtains a target file from the question bank package on demand by locating a target data block, comprising: the client sends the file acquisition request to the file layer, the file layer reads the data in the corresponding BLOB field from the database as the target data block in response to the file acquisition request and sends it to the client; the client decompresses the data in the BLOB field in the memory to obtain the target file.

10. An index-based on-demand access system for item bank data, comprising: Comprise: a question bank obtaining module configured to obtain a question bank package from a file layer storing question bank data in response to a question bank package acquisition request, wherein the file layer is a file system of a preset file format, a modern file system supporting transparent compression, or a single-file embedded database; a file obtaining module configured to locate a target data block from the question bank package on demand in response to a file acquisition request to obtain a target file.

Images