Data encapsulation and extraction method based on custom mark, electronic equipment and medium

Abstract

The invention relates to a data encapsulation and extraction method based on a custom mark, electronic equipment and a medium. The method comprises the following steps: opening or creating a target file and positioning to the end of the file; sequentially writing to-be-stored to-be-packaged files at the tail of the target file, wherein each to-be-packaged file comprises a preset start mark, a length identifier and a data entity which are sequentially written into the target file; the start mark is composed of a beginning 0xFF and a plurality of ASCII characters written behind the 0xFF; the length identifier is used for representing the byte length of the to-be-packaged file, adopts an integer with a fixed length and is written according to a certain byte order; the data entity is the data content of the to-be-packaged file; meanwhile, the invention discloses an extraction method for extracting the file, electronic equipment formed according to the method and a medium. According to the method, heterogeneous data can be closely and efficiently stored in a single file in an associated mode, and any part of data in the heterogeneous data can be rapidly and independently extracted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer data storage and processing, and in particular to a data packaging and extraction method based on custom markers, an electronic device and a medium. BACKGROUND

[0002] In modern computer systems, managing multiple types of associated data (for example, a picture and its textual description) is a common requirement. Existing conventional solutions usually adopt the following ways: 1) Separate storage: binary files (such as pictures, videos) and text files (such as JSON, XML descriptions) are stored separately in different locations of the file system; this approach has obvious defects: an additional mechanism (such as a database or an index file) is needed to maintain the association between files, increasing the complexity of the system; when moving, copying or deleting files, it is easy to lose associated files due to negligence, damaging the integrity of the data; when sharing or backing up data, multiple files must be handled simultaneously, which is easy to overlook and inconvenient to transfer.

[0003] 2) Simple packaging: use general compression formats such as ZIP, TAR to package multiple files; this approach, although it combines data into one file, is actually a collection of separate files, and also has the following disadvantages: to read any file in the package, the entire package or the directory structure within the package must be decompressed first, and then the individual file inside can be read, which is inefficient and cumbersome, especially when only metadata or media files are needed; general compression formats include a lot of redundant header information and directory structure that is not related to the specific business, which is not lightweight and efficient for specific application scenarios. SUMMARY

[0004] To address the existing deficiencies, the present application provides a data packaging and extraction method based on custom markers, an electronic device and a medium.

[0005] The technical solution adopted by the present application to solve its technical problems is: a data packaging method based on custom markers, comprising the following steps: Opening or creating a target file and positioning to the end of the file; Writing the to-be-stored to-be-packaged files one by one at the end of the target file, each to-be-packaged file including a preset starting marker, a length identifier and a data entity written into the target file in sequence, and the starting markers of the multiple to-be-packaged files being different; The start mark is composed of 0xFF at the beginning and a plurality of ASCII characters written after 0xFF, and the same type of file to be packaged has the same number of ASCII characters; the length identifier is used to represent the byte length of the file to be packaged, which uses a fixed-length integer and is written into the target file immediately after the start mark in a certain byte sequence, and the length identifiers of a plurality of files to be packaged are written in the same byte sequence; the data entity is the data content of the file to be packaged.

[0006] As preferred, the length identifier is written into the target file immediately after the start mark in little-endian sequence.

[0007] As preferred, when the file to be packaged is a binary source file, the data entity is directly written into the target file; when the file to be packaged is a text file, the data entity is converted into a byte stream according to character set encoding and written into the target file.

[0008] As preferred, the ASCII characters in the start mark are letter characters, and the start mark contains 2-10 letter characters.

[0009] A data extraction method based on a custom mark, the steps are as follows: Scan for the start mark in the file to be processed and record its location; Jump to the location of the start mark found according to the start mark, read the length identifier after the start mark and parse the byte length; Read the corresponding number of bytes after the length identifier according to the parsed byte length; Process the read bytes in a manner corresponding to the file type corresponding to the start mark according to the file type corresponding to the start mark; Among them, the start mark and the length identifier are the start mark and the length identifier in any one of the data packaging methods described above.

[0010] As preferred, when the file type is a binary source file, the read bytes are saved as an independent binary file and then output; when the file type is a text file, the read bytes are decoded into a text string using a character set and then output.

[0011] As preferred, the scanning is performed by a sliding window buffer.

[0012] As preferred, when the start mark is not found, the entire file to be processed is extracted or the empty text content is returned.

[0013] An electronic device includes: a processor and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement a data encapsulation method based on custom tags as described in any of the preceding claims and / or a data extraction method based on custom tags as described in any of the preceding claims.

[0014] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data encapsulation method based on custom tags as described in the preceding claim and / or the data extraction method based on custom tags as described in the preceding claim.

[0015] The beneficial effects of this invention are as follows: it can physically encapsulate multiple logically related data items into a single file, facilitating moving, copying, backup, and sharing operations, and eliminating the risk of losing associations; through fixed tags and length prefixes, specific data segments in the file to be processed can be randomly accessed, allowing for quick location and extraction of the required parts without parsing the entire file, greatly improving reading efficiency; the file itself describes its internal structure through tags, possessing self-descriptive properties; even if one data block is lost or corrupted during extraction, another data block can still be successfully extracted; this invention also allows data to be appended to the end of any existing file without modifying the original file content, exhibiting good backward compatibility. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the encapsulation method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the internal structure of the composite file formed by encapsulation in an embodiment of the present invention; Figure 3 The flowchart of the extraction method in this embodiment of the invention is shown in the diagram. Detailed Implementation

[0017] To more clearly illustrate the objectives, technical solutions, and advantages of the embodiments of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It is clear and complete that the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0018] Examples of embodiments of the present invention Figure 1 and Figure 2 As shown, a data encapsulation method based on custom tags includes the following steps: Open or create the target file and position it at the end of the file. The target file can be in any existing format. The files to be packaged are sequentially written to the end of the target file. These files can be of the same type or different types, such as binary source files like audio, video, and 3D models, and text files in formats like XML and JSON. After writing one file to be packaged to the end of the target file, another file to be packaged is written after that file, and so on, until multiple files to be packaged are written to the target file. At this point, the software corresponding to the target file can read the contents of the target file normally, but the files to be packaged cannot be read. Each file to be packaged includes a preset start marker, a length identifier, and a data entity that are sequentially written to the target file. The start markers of multiple files to be packaged are different. By using different start markers, the data to be extracted can be located more accurately. The start marker consists of a 0xFF at the beginning followed by multiple ASCII characters. Files of the same type to be packaged have the same number of ASCII characters, but the individual characters are different. This makes it easy to determine the type of file being extracted, and the different characters distinguish files of the same type. Using 0xFF as the start marker allows for quick location of the beginning of a data block. Defining the data block type using ASCII characters increases the uniqueness and readability of the identifier, and also facilitates debugging. The length marker indicates the byte length of the file to be packaged. It is a fixed-length integer written immediately after the start marker in a specific byte order. Multiple files to be packaged must have their length markers written in the same byte order, such as little-endian or big-endian. For example, if the file to be packaged is a 1000-byte image, a fixed length of 4 bytes is used. The hexadecimal representation of 1000 is 0x000003E8, and the 4 bytes written in little-endian order correspond to... In other words, the length identifier is The length identifier written in big-endian order is... The preferred length identifier is written to the target file in little-endian order immediately following the start marker, ensuring consistency in data parsing. The data entity is the data content of the file to be packaged. When the file to be packaged is a binary source file, its data entity is directly written to the target file. When the file to be packaged is a text file, its data entity is converted into a byte stream according to the character set encoding and written to the target file.

[0019] Example 1: To append a binary source file (containing a 30,000-byte image) and a JSON-formatted text file (containing 156 bytes) to a target file, the specific steps are as follows: 1) Prepare the target file: Open or create the target file and position yourself at the end of the file; 2) Write the starting marker for the binary source file: 0xFF,'I','M','G','S','R','C'; 3) Write the length identifier to the binary source file: Immediately following the start marker, write the length identifier in little-endian order. ; 4) Write data entity: Write all the data content of the binary source file to the target file; 5) Write the starting marker to the text file: 0xFF,'D','A','T','A'; 6) Immediately following the start marker of the text file, write the length identifier in little-endian order. ; 7) Convert the text content (JSON string) of the text file into a byte stream using UTF-8 encoding, and then write it to the target file; The above steps encapsulate the binary source file and JSON text into a composite file within the target file.

[0020] Further improvements include using ASCII characters in the start mark, which are alphabetic characters. The start mark contains 2-10 alphabetic characters, which are highly readable, easy to debug and edit, highly compatible, and less prone to misunderstanding. The 2-10 alphabetic characters ensure that the mark is not too short, avoiding conflicts, and not too long, saving space and simplifying parsing, thus providing flexibility.

[0021] A data extraction method based on custom tags, such as Figure 3 As shown, the steps are as follows: The process involves scanning the file to be processed to find the start marker and recording its position. This scanning is performed using a sliding window buffer, where the window size is the byte length of the start marker. The window slides across the entire data stream of the file, avoiding buffer boundary restrictions and preventing the start marker from being truncated. This ensures accuracy by ensuring no marker is missed. If the start marker is not found, the entire file is extracted or an empty text file is returned. If no start marker is found, the entire file is extracted, or an empty text file is returned. If no start marker is found, the entire file is extracted, meaning it wasn't encapsulated using the aforementioned data encapsulation method. If one or more start markers are found, but one is missing, the corresponding file doesn't exist, and an empty text file is returned. For other files with found start markers, the corresponding files are processed according to the file type associated with that start marker. After finding the start marker, jump to the location of the start marker, read the length identifier immediately following the start marker, and parse out the byte length; Read the corresponding number of bytes immediately following the length identifier based on the parsed byte length; The read bytes are processed according to the file type corresponding to the start marker. When the file type is a binary source file, the read bytes are saved as a separate binary file and then output; when the file type is a text file, the read bytes are decoded into a text string using a character set and then output. The starting marker and length identifier described in the above extraction method are the same as the starting marker and length identifier mentioned in the aforementioned data encapsulation method.

[0022] Example 2: 1) Locating Data Blocks: The starting markers in the file to be processed are located. There are two starting markers: the first is 0xFF,'I','M','G','S','R','C', corresponding to a binary source file; the second is 0xFF,'D','A','T','A', corresponding to a text file. The file is scanned starting from the beginning using a sliding window buffer. Initially, the sliding window size is 7 bytes to search for the first starting marker; if found, its position is recorded. Then, the sliding window size is 5 bytes to search for the second starting marker; if found, its position is recorded. 2) Extracting Data Blocks: First, extract the data block corresponding to the first start marker. This involves jumping to the position immediately following the first start marker. Based on the fixed length of the length identifier and the sorting method, read and parse the corresponding number of bytes at this position. For example, if the fixed length of the length identifier is 4 bytes, the four bytes read are... Since the length identifier is written in little-endian order, we can deduce from this that the file to be extracted is a binary source file containing 30,000 bytes by working backwards. The backwards process involves first reversing the order of the bytes... Then, convert it to hexadecimal 0x00007530, and then to decimal, which is 30000. After parsing that the file to be extracted is a binary source file containing 30000 bytes, read 30000 bytes after the length marker, save the read bytes as a separate binary file, and then output it. Next, extract the data block corresponding to the second start marker, that is, jump to the position after the found second start marker, and according to the fixed length and sorting method of the length marker, read four bytes. By working backwards, we can deduce that the file to be extracted is a text file containing 156 bytes. After the length identifier, we read 156 bytes, encode and decode the read bytes into a text string using the character set (UTF-8), and then output it.

[0023] An electronic device includes: a processor and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement a data encapsulation method based on custom tags as described in any of the preceding claims and / or a data extraction method based on custom tags as described in any of the preceding claims.

[0024] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data encapsulation method based on custom tags as described in the preceding claim and / or the data extraction method based on custom tags as described in the preceding claim.

[0025] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A data encapsulation method based on custom tags, characterized in that, Includes the following steps: Open or create the target file and position it at the end of the file; The files to be packaged are written sequentially at the end of the target file. Each file to be packaged includes a preset start marker, length identifier and data entity that are written sequentially to the target file. The start markers of the multiple files to be packaged are different. The start marker consists of a 0xFF at the beginning and multiple ASCII characters written after 0xFF, and files of the same type to be packaged have the same number of ASCII characters; the length identifier is used to indicate the byte length of the file to be packaged, and it is written to the target file in a fixed-length integer and in a certain byte order immediately following the start marker, and the length identifiers of multiple files to be packaged are written in the same byte order; the data entity is the data content of the file to be packaged.

2. The data encapsulation method based on custom tags according to claim 1, characterized in that, The length identifier is written to the target file in little-endian order immediately following the start marker.

3. The data encapsulation method based on custom tags according to claim 1, characterized in that, When the file to be packaged is a binary source file, its data entity is directly written to the target file; when the file to be packaged is a text file, its data entity is converted into a byte stream according to the character set encoding and written to the target file.

4. The data encapsulation method based on custom tags according to claim 1, characterized in that, The ASCII characters in the start mark are alphabetic characters, and the start mark contains 2-10 alphabetic characters.

5. A data extraction method based on custom tags, characterized in that, The steps are as follows: Scan the file to be processed to find the starting marker and record its location; Jump to the location of the found start marker, read the length identifier after the start marker, and parse out the byte length; Read the corresponding number of bytes after the length identifier based on the parsed byte length; The bytes read are processed in a manner corresponding to the file type, based on the file type corresponding to the start marker. The start marker and length identifier are the start marker and length identifier described in any one of claims 1-4 in the data encapsulation method.

6. The data extraction method based on custom tags according to claim 5, characterized in that, When the file type is a binary source file, the read bytes are saved as an independent binary file and then output; when the file type is a text file, the read bytes are decoded into a text string using a character set and then output.

7. The data extraction method based on custom tags according to claim 5, characterized in that, The scanning search is performed using a sliding window buffer.

8. The data extraction method based on custom tags according to claim 5, characterized in that, If the starting marker is not found, the entire file to be processed will be extracted or empty text content will be returned.

9. An electronic device, characterized in that, include: A processor, and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement the data encapsulation method based on custom tags as described in any one of claims 1 to 4 and / or the data extraction method based on custom tags as described in any one of claims 5 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data encapsulation method based on custom tags as described in any one of claims 1 to 4 and / or the data extraction method based on custom tags as described in any one of claims 5 to 8.

Images