Archive file writing and reading method and device, terminal equipment and storage medium

Abstract

The application discloses a kind of filing file writing and reading method, device, terminal equipment and storage medium.The filing file writing method includes: receiving data file to be written;Index information corresponding to data file to be written is used to update the index file written in the tail of the archive file to obtain target index file;Target index file is written in the tail of the archive file, and data file to be written is appended to the data area of the archive file, when continuously adding data file, only data file is appended in the tail of the archive file, and the index file in the tail of the archive file is updated, without needing to modify the entire archive file, solve the problem that the mode of file archiving using ASAR archive file is not suitable for appending content in the archive file, provide a kind of support continuously appending write file archiving method, satisfy more extensive data archiving demand.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of file storage, and in particular to a file archiving writing and reading method and device, a terminal device and a storage medium. BACKGROUND

[0002] File archiving refers to the process of archiving multiple files into an archive file. Currently, the mainstream Node.js-based archive file is mainly an ASAR archive file.

[0003] The ASAR archive file has the advantages of supporting random access and using JSON to store index file information for easy parsing. However, the main disadvantage is that when appending content to the ASAR archive file, the index file stored in its file header needs to be modified, resulting in the need to modify the entire ASAR archive file whenever any changes are made. Therefore, the ASAR archive file is generally only suitable for one-time archiving scenarios.

[0004] Obviously, the file archiving method using the ASAR archive file has limited application scenarios and cannot meet the archiving requirements in scenarios where file data is constantly generated. SUMMARY

[0005] The present application provides a file archiving writing and reading method, device, terminal device and storage medium to solve the problem that the file archiving method using the ASAR archive file is not suitable for appending content to the archive file, and provides a method for continuously appending file archiving to meet more extensive data archiving requirements.

[0006] According to a first aspect of the present application, a file archiving writing method is provided, comprising:

[0007] receiving a data file to be written;

[0008] updating the index file written at the tail of the archive file based on the index information corresponding to the data file to be written to obtain a target index file;

[0009] appending the data file to be written to the data area of the archive file and writing the target index file to the tail of the archive file.

[0010] According to a second aspect of the present application, a file archiving reading method is provided, comprising:

[0011] obtaining a storage path of a data file to be read; wherein the data file to be read is written to the archive file using the file archiving writing method of any embodiment;

[0012] initializing the index file in the archive file according to the storage path;

[0013] According to the storage path, index information of the data file to be read is obtained from the index file;

[0014] The data file to be read is read from a data area of the archive file based on the index information.

[0015] According to a third aspect of the present application, a writing device of an archive file is provided, comprising:

[0016] a data file receiving module, configured to receive a data file to be written;

[0017] an index file updating module, configured to update an index file written at a tail of the archive file based on index information corresponding to the data file to be written to obtain a target index file;

[0018] a file writing module, configured to write the data file to be written to a data area of the archive file and write the target index file to the tail of the archive file.

[0019] According to a fourth aspect of the present application, a reading device of an archive file is provided, comprising:

[0020] a path obtaining module, configured to obtain a storage path of a data file to be read; wherein the data file to be read is written to the archive file by using the writing method of the archive file according to any one of the embodiments;

[0021] an initializing module, configured to initialize an index file in the archive file according to the storage path;

[0022] an information obtaining module, configured to obtain index information of the data file to be read from the index file according to the storage path;

[0023] a data file reading module, configured to read the data file to be read from a data area of the archive file based on the index information.

[0024] According to a fifth aspect of the present application, a terminal device is provided, comprising:

[0025] at least one processor; and

[0026] a memory connected with the at least one processor in communication; wherein,

[0027] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the writing method of the archive file according to any one of the embodiments of the present application or the reading method of the archive file according to any one of the embodiments of the present application.

[0028] According to a sixth aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for causing a processor to implement the writing method of the archive file according to any of the embodiments of the present application or the reading method of the archive file according to any of the embodiments of the present application when executed.

[0029] The technical solution of the embodiments of the present application comprises the following steps: receiving a data file to be written; updating an index file written at the tail of the archive file based on index information corresponding to the data file to be written to obtain a target index file; writing the target index file at the tail of the archive file; and appending the data file to be written to the data area of the archive file. When data files are continuously added, only the data files are appended at the tail of the archive file, and the index file at the tail of the archive file is updated, without the need to modify the entire archive file. The problem that the method of using the ASAR archive file to archive files is not applicable to appending contents in the archive file is solved, and a method of continuously appending and writing files to archive is provided, which meets more extensive data archiving requirements.

[0030] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0032] Figure 1 is a flowchart of a writing method of an archive file provided by the first embodiment of the present application;

[0033] Figure 2 is a schematic diagram of the data structure of an index file in the archive file;

[0034] Figure 3 is a flowchart of a writing method of an archive file provided by the second embodiment of the present application;

[0035] Figure 4 is a structural schematic diagram of the data structure of the archive file;

[0036] Figure 5 is a flowchart of a reading method of an archive file provided by the third embodiment of the present application;

[0037] Figure 6is a flow chart of a reading method of an archive file provided by the fourth embodiment of the present application;

[0038] Figure 7 is a structural schematic diagram of a writing device of an archive file provided by the third embodiment of the present application;

[0039] Figure 8 is a structural schematic diagram of a reading device of an archive file provided by the third embodiment of the present application;

[0040] Figure 9 is a structural schematic diagram of a terminal device for implementing the writing or reading method of an archive file. DETAILED DESCRIPTION

[0041] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0042] It should be noted that the terms “include” and “have” and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover the inclusions without exclusivity, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0043] Generally, in a file client (for example, a mail client), it is necessary to store the data file received from the server in the local disk file system for fast reading, and therefore the file system needs to support continuous and uninterrupted writing of newly received data files into an archive file. However, the ASAR archive file based on Node.js is generally only applicable to the scene of one-time archiving and does not support continuous and additional writing of the archive file.

[0044] Node.js is a JavaScript runtime based on Chrome V8 engine, based on event-driven, non-blocking I / O model, so that JavaScript can run on the server side, and also makes JavaScript can call the API of the operating system, which is used for the writing of cross-platform client software.

[0045] In order to solve the above problems, the embodiments of the present application provide a method which can support continuous and additional writing of an archive file.

[0046] Embodiment One

[0047] Figure 1 is a flowchart of a writing method of an archive file provided by Embodiment One of the present application. The present embodiment can be applied to the case of writing a data file to be written into an archive file. The method can be executed by a writing device of an archive file, which can be implemented in the form of hardware and / or software, and can be configured in a terminal device. As shown in the figure, the method comprises: Figure 1

[0048] S110, receiving a data file to be written.

[0049] The data file to be written can be understood as a data file waiting to be written into an archive file. The data file to be written can be a mail, a binary document file or a file in other formats, and the present embodiment does not limit the file format of the data file to be written.

[0050] For example, the terminal device receives the incoming data file to be written through a client, and thus archives the data file to be written in the local hard disk, i.e. writes the data file to be written into an archive file stored in the local hard disk.

[0051] S120, updating the index file written at the tail of the archive file based on the index information corresponding to the data file to be written to obtain a target index file.

[0052] The archive file can be understood as a file for archiving and storing a plurality of data files. In the present embodiment, an archive file can be generated based on the user identifier of each user, and the archive file is used to store a plurality of data files of the user. For example, for the archiving of mail data, a mail archive file is generated based on the user identifier locally, and a plurality of mail data of the same user is stored in the same mail archive file.

[0053] The archive file comprises a data area and an index area. The data area is used to store data files, and the index area is used to store index files corresponding to the data files. The index area is always located at the tail of the archive file. The index file refers to a file composed of index information corresponding to the data file. The target index file is an index file obtained after the index information of the index file in the archive file is updated.

[0054] Specifically, each time a data file to be written is written into an archive file, the index file written at the tail of the archive file needs to be updated based on the index information of the data file to be written to obtain a target index file.

[0055] For example, the index file can adopt the storage format of a JSON file and be stored in the index area of the archive file in a multi-way tree storage structure.​Figure 2 is a schematic diagram of a data structure of an index file in an archive file. As shown in Figure 2 , the key value of the index file can adopt a folder name or a file name, and since the same name of a folder or a file is not allowed to exist under each level of directory, the key value can be ensured to be unique.

[0056] S130, the data file to be written is appended to the data area of the archive file, and the target index file is written to the tail of the archive file.

[0057] Specifically, in the case of writing a data file into an archive file, the data file to be written is written to the tail of the archive file in an append mode, and the updated target index file is appended to the tail of the archive file.

[0058] In this embodiment, only the data file is appended to the tail of the archive file when the data file is continuously added, and the index file at the tail of the archive file is updated, without the need to modify the entire archive file, so that continuous append writing of file data can be supported, and more extensive data archiving requirements can be met.

[0059] The technical scheme of the embodiment of the application comprises the following steps: receiving a data file to be written; updating an index file written at the tail of an archive file based on index information corresponding to the data file to be written to obtain a target index file; and writing the target index file to the tail of the archive file and appending the data file to be written to a data area of the archive file. The method solves the problem that the ASAR archive file is not suitable for appending content in the archive file, provides a method for continuously appending and writing file archiving, and meets more extensive data archiving requirements.

[0060] Optionally, the index information comprises an offset of the data file, a file size, and a compressed storage identifier.

[0061] The compressed storage identifier is used to mark whether the data file in the archive file adopts a compressed storage mode, that is, the data file can support a compressed storage mode or a non-compressed storage mode when the data file is written into the archive file.

[0062] When the number of data files is large, the following problems can occur:

[0063] (1) When the directory structure level of the data file storage is deep, or the number of data files under a single directory is large, the performance of the operating system for reading and writing the data file will be significantly reduced; (2) the local disk file system itself cannot support compressed storage of these files, and the space occupied is large; (3) if the data files stored in the local disk file system need to be separately backed up and uploaded to the cloud storage, since the number of files and directories is too large, the files need to be packaged and compressed before being uploaded, and the use cost is high.

[0064] Therefore, the archive file provided by the embodiment of the present application supports two modes of compressed storage and uncompressed storage, and the compressed storage mode can solve the above problems.

[0065] Embodiment Two

[0066] Figure 3 is a flowchart of an archive file writing method provided by Embodiment Two of the present application. In the archive file writing method provided by Embodiment One, the data in the tail index file needs to be updated when each data file is written into the archive file, and when the number of data files written is too large, the data amount of the index file will also be very large, thereby causing the writing performance to decrease.

[0067] In order to solve the problem of the decrease in writing performance caused by the large data amount of the index file, the step S120 of the above embodiment is further refined as "the index file based on the index information corresponding to the to-be-written data file and written at the tail of the archive file is updated to obtain a target index file, including: when the size of the index file written at the tail of the archive file is greater than a preset data amount, the index file is split to generate a current index file; wherein the current index file is stored at the tail of the archive file and used to write the index information corresponding to the to-be-written data file; a link list pointer of the current index file is pointed to the index file before splitting to form a link list; and the index information stored in the current index file is updated based on the index information corresponding to the to-be-written data file to obtain the target index file".

[0068] As shown in Figure 3 , the method comprises:

[0069] S210, receiving a to-be-written data file.

[0070] S220, when the size of the index file written at the tail of the archive file is greater than a preset data amount, the index file is split to generate a current index file; wherein the current index file is stored at the tail of the archive file and used to write the index information corresponding to the to-be-written data file.

[0071] The preset data amount required by the index file splitting can be set according to the actual requirements such as the writing rate and the device nature, for example, 100K, and the embodiment of the present application does not limit this.

[0072] The index file is split to generate a current index file. The current index file is always stored at the tail of the archive file and used to write the index information corresponding to the to-be-written data file. The original index file writing the index information is stored at the original storage position when the index file is split as a common JSON file, and no modification is required for the original index file when the index file is updated.

[0073] Specifically, in order to reduce the influence of the large data volume of the index file on the writing performance, when the size of the index file written at the tail of the archive file is greater than the preset data volume, the index file written at the tail of the archive file is split to generate a new current index file.

[0074] In S230, a chain table pointer of the current index file is pointed to the archive index file to form a chain table.

[0075] In order to make the index information contained in the archive file complete, the index files stored in different positions are associated in a chain table manner, and a chain table pointer of the current index file is pointed to the original index file before splitting, so as to establish the chain table between the original index file and the current index file.

[0076] It can be understood that when the size of the index file written by the current index file is greater than the preset data volume, a new current index file is also generated by splitting, and the chain table pointer of the new current index file is pointed to the original index file before splitting, that is, the current index file, so as to associate the index files stored in different areas and different positions in the archive file. When reading the data file from the archive file, all the index files can be traversed based on the chain table to obtain complete index information.

[0077] In S240, the index information stored in the current index file is updated based on the index information corresponding to the data file to be written, to obtain a target index file.

[0078] Specifically, when updating the index file in the archive file, it is not necessary to update all the index files in the archive file, but only the current index file at the tail of the document file needs to be updated. Specifically, the index information (offset, file size) corresponding to the data file to be written is calculated, the index information stored in the current index file at the tail of the document file is updated based on the index information corresponding to the data file to be written, and a target index file is obtained.

[0079] In this step, when the data file to be written is written into the archive file, only the index information stored in the current index file at the tail of the document file is updated based on the index information corresponding to the data file to be written, and all the index files stored in the document file do not need to be updated, so that the influence of the large amount of writing of the index file on the writing performance of the archive file is reduced.

[0080] Optionally, the data structure of the archive file comprises: a head area, a data area, and an index area located at the tail of the archive file.

[0081] The index area is used for storing the current index file, the head area is used for storing the offset and file length of the current index file, and the data area is used for storing the data file to be written and the index file.

[0082] Specifically, Figure 4 is a structural diagram of a data structure of an archive file. As shown in the figure, the data structure of the archive file includes a header area, a data area, and an index area located at the tail of the archive file. The index area is located at the tail of the archive file and stores a current index file. The data area stores the offset and file length of the current index file, and when the index file is queried from a specified file, the current index file can be located based on the offset and file length of the current index file stored in the header area. The data area is used to store data files and index files to be written. Among them, the index file in the data area and the current index file in the index area are associated in the form of a linked list. Figure 4

[0083] For example, the header area of the archive file can include 32 bytes, the first 16 bytes are the offset of the current index file at the tail of the archive file, and the 16th to 32nd bytes are the size of the current index file.

[0084] S250, reserving a blank storage area at the tail of the data area of the archive file, the size of the blank storage area being determined according to the size of the data file to be written.

[0085] If an abnormal situation such as sudden power failure, program crash, etc. occurs when writing an index file to an archive file, the index file may not be completely written, thereby causing the entire archive file to be unreadable. In order to avoid the problem that the archive file cannot be read, the embodiment of the application adopts a reverse order writing method, that is, the index file is written first, and then the data file is written. Since the index file is stored in blocks, when writing the file, only the current index file at the tail of the archive file is modified, and the data amount of a single index file is relatively small (for example, less than 100k), and compared with the data file with a large data amount, the probability of writing failure will be greatly reduced. Avoiding the situation that after the data file is successfully written, the index file fails to be written, resulting in that the data file needs to be re-written.

[0086] And when the index file is written first and the data file is written at the tail of the data file, the written index file will be overwritten. Therefore, a blank storage area needs to be reserved at the tail of the data area of the archive file according to the size of the data file to be written, for writing the data file in the blank storage area after writing the index file. In this way, it can not only ensure that the index file is written first and then the data file is written, but also ensure that the index file written first is always located at the tail of the archive file.

[0087] S260, moving a write pointer pointing to the tail of the archive file to the end of the blank storage area, and writing the target index file at the tail of the archive file.

[0088] ​Specifically, after reserving a blank storage area at the end of the data area of the archive file, the write pointer is moved from the end of the archive file to the end of the blank storage area, the blank storage area is skipped, and the target index file is written at the end of the archive file.

[0089] Optionally, the method further comprises:

[0090] The index information of the data file to be written is backed up to a log file, wherein the log file and the archive file are in the same directory.

[0091] Specifically, the writing to the archive file can also be damaged, resulting in the entire archive file being unreadable, and therefore the index information of the writing can also be backed up through the log file. Specifically, a hidden log file is generated in the same directory as the archive file. When the data file to be written needs to be written, the index information of the data file to be written is written to the log file while the target index file is written to the archive file.

[0092] Since the log file saves the backup index information of all index files, when the index file is read from the archive file due to program exception or power failure writing failure, the backup index information in the log file can be used to completely restore the index file.

[0093] S270, moving the write pointer to the start of the blank storage area, and writing the data file to be written in the blank storage area.

[0094] Specifically, after the writing of the target index file is completed, the write pointer pointing to the end of the archive file is moved to the start of the reserved blank storage area, and the data file to be written is written in the blank storage area.

[0095] The technical scheme of the embodiment of the application comprises the following steps: receiving a data file to be written; when the size of the index file written at the tail of the archive file is greater than a preset data amount, splitting the index file to generate a current index file; the current index file is stored at the tail of the archive file and is used for writing index information corresponding to the data file to be written; a link list pointer of the current index file is pointed to the index file before splitting to form a link list; the index information stored in the current index file is updated based on the index information corresponding to the data file to be written to obtain a target index file; a blank storage area is reserved at the tail of the data area of the archive file, and the size of the blank storage area is determined according to the size of the data file to be written; a write pointer pointing to the tail of the archive file is moved to the end of the blank storage area, and the target index file is written at the tail of the archive file; the write pointer is moved to the start of the blank storage area, and the data file to be written is written in the blank storage area; based on the block storage mode of the index file, the influence of the large amount of writing of the index file on the writing performance of the archive file is reduced; based on the mechanism of writing the index file first and the mechanism of backing up the index information in the log, the probability of the entire archive file being unable to be read due to the index file being unable to be completely written is reduced.

[0096] In a specific example, when writing the data file to be written into the archive file, the size of the data file to be written is obtained by calling the fs module of Node.js. If the remaining space of the disk where the archive file is located is greater than the size of the data file to be written, the index file in the archive file is updated, the offset and the file size of the data file to be written are written into the archive file, and then the data writing operation is started at the tail of the data area of the archive file.

[0097] For example, when a data file to be written with a size of 1.5 MB is to be written at the tail of the data area of the archive file, the write pointer is moved backward by 1.5 MB from the tail of the archive file to reserve a blank storage area, the index file is written after the blank storage area, the index information contained in the index file is backed up in the log file, after the writing of the index file and the backup are successful, the write pointer is moved forward by the sum of 1.5 MB and the size of the tail index file, and the data file to be written is written in the previously reserved blank storage area.

[0098] Embodiment three

[0099] Figure 5 A flowchart of an archive file reading method is provided for the third embodiment of the application. The embodiment can be applied to the case of reading a data file to be read from an archive file. The method can be executed by an archive file reading device. The archive file reading device can be implemented in the form of hardware and / or software and can be configured in a terminal device. As shown in the figure, the method comprises the following steps. Figure 5 ​

[0100] S310, acquire a storage path of a data file to be read; wherein the data file to be read is written into the archive file by using an archive file writing method.

[0101] The storage path of the data file to be read can be composed of two parts. The first part is a storage path of the archive file in a local disk of the terminal device. The second part is a storage path of the data file to be read in the archive file.

[0102] For example, the storage path of the data file to be read is / mailbox / 2022 / 0101 / 1.eml. The storage path of the archive file is / mailbox / 2022 / 0101. The storage path of the data file to be read in the archive file is 0101 / 1.eml.

[0103] S320, initialize an index file in the archive file according to the storage path. The index information of the data file to be read can include an offset of the data file to be read, a file size, and a compression storage identifier. The index information can also include a deletion identifier for marking whether the data file to be read is deleted.

[0104] Specifically, according to the storage path of the archive file corresponding to the data file to be read, the archive file in which the data file to be read is located is determined, and the index file contained in the archive file is searched to initialize the index file.

[0105] For example, the initialization of the index file in the archive file can be performed by acquiring an offset and a size of the index file at the tail end of the archive file from a head region of the archive file, sequentially acquiring the next index file according to a linked list relationship of pointers of the index file at the tail end of the archive file, and merging the acquired index files to complete the initialization of the index file in the archive file.

[0106] S330, acquire index information of the data file to be read from the index file according to the storage path.

[0107] Specifically, since the index file in the archive file is stored in a multi-way tree structure, and the file name of the index file with uniqueness can be used as a key value, the index information of the data file to be read can be acquired from the index file by directly acquiring the index file of any directory through the key value without looping, thereby greatly reducing the time complexity of file searching.

[0108] Exemplary, for the storage path / a / b / c of the data file to be read, the path can be mapped into the index file object of the array form of indexFile[“a”][“b”][“c”] using the “ / ” separator, so that the index information of the index file object of any subdirectory can be easily obtained by accessing the index file object step by step. The index information of the index file object can include: the offset, size, whether to use compressed storage, and other information of the data file in the archive file. To find the 1.eml data file in the / mailbox / 2022 / 0101 directory, only indexFile[“mailbox”][“2022”][“0101”][“files”][“1.eml”] is needed to directly obtain the index information of the data file 1.eml from the index file initialized from the archive file files.

[0109] S340, reading the data file to be read from the data area of the archive file based on the index information.

[0110] Specifically, the method of reading the data file to be read from the data area of the archive file according to the index information of the data file to be read can be: if the deletion identifier of the data file to be read indicates that the data file to be read is deleted, returning a result that the data file to be read cannot be read; otherwise, reading the data file from the data area of the archive file according to the offset and the offset size of the data file to be read. If the data file to be read is compressed storage, the read data file needs to be decompressed to obtain the data file to be read; otherwise, the read data file is the data file to be read.

[0111] The technical scheme of the embodiment of the application comprises: obtaining the storage path of the data file to be read; wherein the data file to be read is written into the archive file using an archive file writing method; initializing the index file in the archive file according to the storage path, obtaining the index information of the data file to be read from the index file according to the storage path; and reading the data file to be read from the data area of the archive file based on the index information. The data file stored in the archive file can be randomly accessed and read, which is convenient and easy to parse.

[0112] Embodiment Four

[0113] Figure 6 A flowchart of an archive file reading method provided by the fourth embodiment of the application is further detailed for step S320 of the third embodiment described above. As shown in Figure 6 The method comprises:

[0114] S410, acquire a storage path of a data file to be read; wherein the data file to be read is written into the archive file by using an archive file writing method.

[0115] S420, read head information stored in a head area of the archive file according to the storage path.

[0116] Specifically, the archive file stored in the terminal device is acquired based on the storage path of the archive file according to the storage path of the archive file parsed from the storage path of the data file to be read, and the head information stored in the head area is parsed from the archive file. The head information can include an offset and a size of an index file at the tail end of the archive file.

[0117] S430, move a file reading pointer according to an offset of a current index file contained in the head information, and read an index file object in the current index file.

[0118] The current index file is an index file at the tail end of the archive file. The pointer of the current index file is associated with the previous index file through a linked list, so that all data files corresponding to the index files contained in the archive file are connected to each other.

[0119] Specifically, the reading pointer is moved from the head area to the current index file according to the offset of the current index file contained in the head information, the current index file is read based on the file size of the current index file contained in the head information, and the index file object in the current index file is parsed by using a JSON parser.

[0120] S440, if the index file object contains a linked list pointer pointing to a next index file, move the file reading pointer based on the linked list pointer, and read a next index file object in the next index file in the archive file.

[0121] Specifically, if the index file object contains a linked list pointer pointing to a next index file, it indicates that there are multiple index files in the archive file. The next index file is found by using the offset and the file size of the linked list pointer contained in the attribute of the index file object, and all index file indexFile objects of the index files are found in turn by using the data structure of the linked list, and are stored in the memory in the form of an array.

[0122] S450, combine each index file object parsed to obtain an index file in the archive file.

[0123] Specifically, the complete index file of the archive file can be obtained by combining each index file object stored in the memory, and the initialization of the index file is completed.

[0124] Optionally, further comprising: in the process of initializing the index file in the archive file according to the storage path of the archive file, if the index file object cannot be loaded, reading a log file to obtain the index file.

[0125] Specifically, in the process of initializing the index file in the archive file according to the storage path of the archive file, if the index file object cannot be loaded due to index file damage and the like, the index file can be restored through a log file, because the log file contains complete information of all index file writing, log information is read from the backup log file, and the index file can be parsed and restored based on the log information.

[0126] Illustratively, the specific way of reading the log file to obtain the index file can be reading the log file line by line, one file information per line, the file information can be separated by a “|” symbol, and the file information can be split and parsed into a file path, an offset, a file size, and whether compression, and then the file path is split by a “ / ” separator, such as / a / b / c, to generate a multi-level index file object of indexFile[“a”][“b”][“c”].

[0127] S460, obtaining index information of the to-be-read data file from the index file.

[0128] S470, reading the to-be-read data file from a data area of the archive file based on the index information.

[0129] The technical scheme of the embodiment of the application, by obtaining the storage path of the to-be-read data file, wherein the to-be-read data file is written into the archive file by using an archive file writing method, according to the storage path, reading header information stored in a header area of the archive file, moving a file reading pointer according to an offset of a current index file contained in the header information, reading an index file object in the current index file, wherein the current index file is an index file at the tail of the archive file, if the index file object contains a linked list pointer pointing to a next index file, moving the file reading pointer based on the linked list pointer, reading a next index file object in a next index file in the archive file, merging each index file object parsed to obtain an index file in the archive file, obtaining index information of the to-be-read data file from the index file according to the storage path of the to-be-read data file, and reading the to-be-read data file from a data area of the archive file based on the index information, can randomly access and read the data file stored in the archive file, reading is convenient, easy to parse, and when the index file is damaged, the index file can be restored from the log file, avoiding the problem that the entire archive file cannot be read due to index file damage.

[0130] Embodiment five

[0131] Figure 7 Figure 1 is a structural schematic diagram of an archiving file writing device according to an embodiment of the present application. As shown in the figure, the device comprises a data file receiving module 510, an index file updating module 520 and a file writing module 530. Figure 7

[0132] The data file receiving module 510 is configured to receive a data file to be written.

[0133] The index file updating module 520 is configured to update an index file written at the tail of an archiving file based on index information corresponding to the data file to be written to obtain a target index file.

[0134] The file writing module 530 is configured to append the data file to be written to a data area of the archiving file and write the target index file at the tail of the archiving file.

[0135] Optionally, the index file updating module 520 is specifically configured to:

[0136] when the size of the index file written at the tail of the archiving file is greater than a preset data amount, split the index file to obtain a current index file; wherein the current index file is stored at the tail of the archiving file and is used to write the index information corresponding to the data file to be written.

[0137] point a link list pointer of the current index file to the index file before splitting to form a link list.

[0138] update the index information stored in the current index file based on the index information corresponding to the data file to be written to obtain a target index file.

[0139] Optionally, the data structure of the archiving file comprises a head area, a data area and an index area at the tail of the archiving file.

[0140] wherein the index area is used to store the current index file; the head area is used to store the offset and file length of the current index file; and the data area is used to store the data file to be written and the index file.

[0141] Optionally, the file writing module 530 is specifically configured to:

[0142] reserve a blank storage area at the tail of the data area of the archiving file, and the size of the blank storage area is determined according to the size of the data file to be written.

[0143] move a writing pointer pointing to the tail of the archiving file to the end of the blank storage area and write the target index file at the tail of the archiving file.​

[0144] moving the write pointer to the start of the blank storage area, and writing the data file to be written into the blank storage area.

[0145] Optionally, the method further comprises:

[0146] an index backup module, configured to backup index information of the data file to be written into a log file; wherein the log file and the archive file are in the same directory.

[0147] The archive file writing device provided by the embodiment of the application can execute the archive file writing method provided by any of the embodiments of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0148] Embodiment six

[0149] Figure 8 Fig. 6 is a structural schematic diagram of an archive file writing device provided by the embodiment five of the application. As shown in the figure, the device comprises a path acquisition module 610, an initialization module 620, an information acquisition module 630 and a data file reading module 640. Figure 8

[0150] The path acquisition module 610 is configured to acquire a storage path of a data file to be read in an archive file and a storage path of the archive file; wherein the data file to be read is written into the archive file by using an archive file writing method.

[0151] The initialization module 620 is configured to initialize an index file in the archive file according to the storage path of the archive file.

[0152] The information acquisition module 630 is configured to acquire index information of the data file to be read from the index file according to the storage path.

[0153] The data file reading module 640 is configured to read the data file to be read from a data area of the archive file based on the index information.

[0154] Optionally, the initialization module 620 is specifically configured to:

[0155] read header information stored in a header area of the archive file according to the storage path of the archive file;

[0156] move a file reading pointer according to an offset of a current index file contained in the header information, and read an index file object in the current index file; wherein the current index file is an index file at the tail of the archive file.

[0157] ​If the index file object contains a chain table pointer pointing to the next index file, the file reading pointer is moved based on the chain table pointer, and a next index file object in a next index file in the archive file is read;

[0158] The parsed index file objects are merged to obtain the index file in the archive file.

[0159] Optionally, the method further comprises:

[0160] The log file reading module is configured to, in the process of initializing the index file in the archive file according to the storage path of the archive file, if the index file object cannot be loaded, read a log file to obtain the index file.

[0161] The archive file reading device provided by the embodiments of the present application can perform the archive file reading method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of performing the method.

[0162] Embodiment seven

[0163] Figure 9 A structural schematic diagram of a terminal device 10 that can be used to implement embodiments of the present application is shown. The terminal device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The terminal device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headgear, eyewear, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0164] As shown in Figure 9 The terminal device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the terminal device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0165] A plurality of components in the terminal device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, and the like; an output unit 17, such as various types of displays, speakers, and the like; a storage unit 18, such as a magnetic disk, an optical disk, and the like; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 19 allows the terminal device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0166] The processor 11 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as the write method of an archive file or the read method of an archive file.

[0167] In some embodiments, the write method of an archive file or the read method of an archive file can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the terminal device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the write method of an archive file or the read method of an archive file described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the write method of an archive file or the read method of an archive file by any other appropriate means, such as by means of firmware.

[0168] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0169] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, enables the functions / acts specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package and partially on a remote machine or entirely on a remote machine or server.

[0170] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] To provide for interaction with a user, the systems and techniques described here can be implemented on a terminal device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the terminal device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0172] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0173] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0174] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, and the present disclosure is not limited herein as long as the desired results of the technical solutions of the present disclosure can be achieved.

[0175] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of writing an archive file, characterized by, The method comprises the following steps: receiving a data file to be written; updating an index file written at the tail of an archive file based on index information corresponding to the data file to be written to obtain a target index file; appending the data file to be written to a data area of the archive file and writing the target index file to the tail of the archive file; the step of updating the index file written at the tail of the archive file based on the index information corresponding to the data file to be written to obtain the target index file comprises the following steps: when the size of the index file written at the tail of the archive file is greater than a preset data amount, splitting the index file to obtain a current index file; wherein the current index file is stored at the tail of the archive file and is used for writing the index information corresponding to the data file to be written; pointing a linked list pointer of the current index file to a linked list formed by the index file before splitting; updating the index information stored in the current index file based on the index information corresponding to the data file to be written to obtain the target index file; the data structure of the archive file comprises a head area, a data area and an index area at the tail of the archive file; wherein the index area is used for storing the current index file; the head area is used for storing the offset and file length of the current index file; and the data area is used for storing the data file to be written and the index file.

2. The method of claim 1, wherein, the step of appending the data file to be written to the data area of the archive file and writing the target index file to the tail of the archive file comprises the following steps: reserving a blank storage area at the tail of the data area of the archive file, and the size of the blank storage area is determined according to the size of the data file to be written; moving a write pointer pointing to the tail of the archive file to the end of the blank storage area and writing the target index file at the tail of the archive file; moving the write pointer to the start of the blank storage area and writing the data file to be written in the blank storage area.

3. The method of claim 2, wherein, The method further comprises the following steps: backing up the index information of the data file to be written to a log file; wherein the log file and the archive file are in the same directory.

4. The method of claim 1, wherein, The index information comprises the offset, file size and compressed storage identifier of the data file.

5. A method of reading an archived file, characterized by, The method comprises the following steps: obtaining a storage path of a data file to be read; wherein the data file to be read is written into an archive file by using the archive file writing method in any one of claims 1 to 4; initializing an index file in the archive file according to the storage path; obtaining index information of the data file to be read from the index file according to the storage path; reading the data file to be read from a data area of the archive file based on the index information.

6. The method of claim 5, wherein, The step of initializing the index file in the archive file according to the storage path comprises the following steps: reading head information stored in a head area of the archive file according to the storage path; moving a file reading pointer according to the offset of a current index file contained in the head information to read an index file object in the current index file; wherein the current index file is an index file at the tail of the archive file. If the index file object contains a chain table pointer pointing to a next index file, the file reading pointer is moved based on the chain table pointer to read a next index file object in a next index file in the archive file; The index file objects obtained by parsing are combined to obtain an index file in the archive file.

7. The method of claim 6, wherein, Further comprising: In the process of initializing the index file in the archive file according to the storage path, if the index file object cannot be loaded, the index file is obtained by reading a log file.

8. An apparatus for writing an archive file, characterized by Comprising: A data file receiving module, configured to receive a data file to be written; An index file updating module, configured to update an index file written at the tail of the archive file based on index information corresponding to the data file to be written to obtain a target index file; A file writing module, configured to append the data file to be written to a data area of the archive file and write the target index file to the tail of the archive file. The index file updating module is specifically configured to: When the size of the index file written at the tail of the archive file is greater than a preset data amount, split the index file to generate a current index file; wherein the current index file is stored at the tail of the archive file and is used to write the index information corresponding to the data file to be written; Point a chain table pointer of the current index file to the index file before splitting to form a chain table; Update the index information stored in the current index file based on the index information corresponding to the data file to be written to obtain a target index file; The data structure of the archive file comprises: a head area, a data area and an index area at the tail of the archive file; wherein the index area is used to store the current index file; the head area is used to store the offset and file length of the current index file; and the data area is used to store the data file to be written and the index file.

9. An apparatus for reading an archived file, characterized by Comprising: A path obtaining module, configured to obtain a storage path of a data file to be read; wherein the data file to be read is written into an archive file by using the archive file writing method in any one of claims 1-4; An initialization module, configured to initialize an index file in the archive file according to the storage path; An information obtaining module, configured to obtain index information of the data file to be read from the index file according to the storage path; A data file reading module, configured to read the data file to be read from a data area of the archive file based on the index information.

10. A terminal device, comprising: The terminal device comprises: At least one processor; and A memory connected with the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the archive file writing method in any one of claims 1-4 or the archive file reading method in any one of claims 5-7.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the writing method of the archive file of any one of claims 1-4 or the reading method of the archive file of any one of claims 5-7 when executed.

Images