File parsing method, device, equipment and storage medium

Abstract

The application discloses a file analysis method, device, equipment and storage medium, and belongs to the technical field of automobiles. The method specifically comprises the following steps: acquiring an embedded firmware file to be processed, wherein the embedded firmware file comprises a file size and a file type; in the case that the file size of the embedded firmware file is greater than a preset file size threshold, performing a streaming analysis processing on the embedded firmware file to obtain metadata corresponding to at least one data record; based on the metadata corresponding to each data record, a first layer file index table is constructed by using a first construction strategy; based on the first layer file index table, a second layer address range tree is constructed by using a second construction strategy; based on a preset byte size, a third layer block-level cache is constructed by using a third construction strategy; and based on the second layer address range tree, a data block list corresponding to the embedded firmware file is obtained, so that the data block list is output through a display interface.

Description

Technical Field

[0001] This application relates to the field of automotive technology, specifically to vehicle testing technology, vehicle data processing technology, and other technical fields, and particularly to a method, apparatus, device, and storage medium for file parsing. Background Technology

[0002] Currently, embedded firmware files, such as Motorola S-Record / S19 and Intel HEX formats, are widely used in automotive electronics, industrial control, and consumer electronics.

[0003] Typically, file parsing schemes for embedded firmware files in related technologies include full-load parsing, block-based reading parsing, and general large file index parsing. However, these schemes suffer from problems such as high memory consumption, long parsing latency, and failure to consider the format-specific attributes of embedded firmware files. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for file parsing, which can solve the problems of high memory consumption and long parsing delay during the file parsing process of embedded firmware files. The technical solution is as follows: Firstly, a method for file parsing is provided, the method comprising: Obtain the embedded firmware file to be processed, the embedded firmware file including file size and file type; If the file size of the embedded firmware file is determined to be greater than a preset file size threshold, the embedded firmware file is subjected to streaming parsing to obtain metadata corresponding to at least one data record. Based on the metadata corresponding to each data record, the first-level file index table is constructed using the first construction strategy; Based on the first-level file index table, a second-level address range tree is constructed using the second construction strategy; Based on the preset byte size, a third-level block cache is constructed using a third construction strategy; Based on the second-level address range tree, a list of data blocks corresponding to the embedded firmware file is obtained, and the list of data blocks is output through the display interface.

[0005] In one possible implementation, the streaming parsing of the embedded firmware file to obtain metadata corresponding to at least one data record includes: The embedded firmware file is read line by line in a streaming manner to obtain at least one data record; Based on the file type of the embedded firmware file, record header parsing is performed on at least one data record to obtain key data corresponding to at least one data record. Based on the key data corresponding to at least one data record, obtain the metadata corresponding to at least one data record.

[0006] In one possible implementation, the step of constructing a first-level file index table based on the metadata corresponding to each data record and utilizing a first construction strategy includes: Based on the metadata corresponding to each data record, determine at least one tuple; Construct a first-level file index table based on at least one tuple.

[0007] In one possible implementation, constructing a second-level address range tree based on the first-level file index table and utilizing a second construction strategy includes: Determine the address range of each data record in the first-level file index table; Based on each data record and its address range in the first-level file index table, a second-level address range tree is constructed.

[0008] In one possible implementation, obtaining the data block list corresponding to the embedded firmware file based on the second-level address range tree includes: Based on the second-level address range tree, determine the address range of the nodes in the second-level address range tree; The address ranges of the nodes in the second-level address range tree are merged according to the order of the node address ranges to obtain the data block list corresponding to the embedded firmware file.

[0009] In one possible implementation, after outputting the list of data blocks through the display interface, the process includes: In response to the selection operation of the first data block in the data block list, the cached data corresponding to the first data block is queried in the third-level block cache; If the cached data corresponding to the first data block is not found in the third-level block cache, the metadata corresponding to the first data block in the first-level file index table is determined based on the first data block and the second-level address range tree. Based on the metadata corresponding to the first data block, obtain the memory data corresponding to the first data block; The memory data corresponding to the first data block is exported according to the specified format.

[0010] Secondly, a file parsing apparatus is provided, the apparatus comprising: The first acquisition unit is used to acquire the embedded firmware file to be processed, wherein the embedded firmware file includes file size and file type; The first parsing unit is used to perform streaming parsing processing on the embedded firmware file when it is determined that the file size of the embedded firmware file is greater than a preset file size threshold, so as to obtain metadata corresponding to at least one data record. The first construction unit is used to construct the first-level file index table based on the metadata corresponding to each data record and using the first construction strategy. The second construction unit is used to construct a second-level address range tree based on the first-level file index table and using a second construction strategy. The third building unit is used to build a third-level block cache based on a preset byte size and using a third building strategy. The first display unit is used to obtain a list of data blocks corresponding to the embedded firmware file based on the second-level address range tree, so as to output the list of data blocks through the display interface.

[0011] Thirdly, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the aspects and any possible implementations described above.

[0012] Fourthly, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0013] The beneficial effects of the technical solution provided in this application include at least the following: As can be seen from the above technical solution, the embodiments of this application can obtain the embedded firmware file to be processed, which includes file size and file type. Then, when it is determined that the file size of the embedded firmware file is greater than a preset file size threshold, the embedded firmware file can be streamed and parsed to obtain metadata corresponding to at least one data record. Based on the metadata corresponding to each data record, a first-level file index table is constructed using a first construction strategy. Based on the first-level file index table, a second-level address range tree is constructed using a second construction strategy. Based on a preset byte size, a third-level block-level cache is constructed using a third construction strategy. Based on the second-level address range tree, a list of data blocks corresponding to the embedded firmware file is obtained, and the list of data blocks is output through a display interface. Since a lightweight hierarchical index can be constructed based only on the metadata of the data records during file parsing, there is no need to load the actual data to be parsed. Memory usage is controlled through block-level caching, achieving low memory usage, fast startup, and visual parsing of large-capacity embedded firmware files.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a file parsing method provided in one embodiment of this application; Figure 2 This is a schematic diagram of a hierarchical sparse index architecture provided in one embodiment of this application; Figure 3 This is a structural block diagram of a document parsing apparatus provided in another embodiment of this application; Figure 4 This is a block diagram of an electronic device used to implement the file parsing method of the embodiments of this application. Detailed Implementation

[0017] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] Obviously, the described embodiments are only some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0019] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0020] Please refer to Figure 1 This illustration shows a flowchart of a file parsing method provided in one embodiment of this application. Specifically, the file parsing method may include: Step 101: Obtain the embedded firmware file to be processed, which includes file size and file type.

[0021] Step 102: If the file size of the embedded firmware file is determined to be greater than a preset file size threshold, the embedded firmware file is subjected to streaming parsing processing to obtain metadata corresponding to at least one data record.

[0022] Step 103: Based on the metadata corresponding to each data record, construct the first-level file index table using the first construction strategy.

[0023] Step 104: Based on the first-level file index table, construct the second-level address range tree using the second construction strategy.

[0024] Step 105: Based on the preset byte size, construct the third-level block cache using the third construction strategy.

[0025] Step 106: Based on the second-level address range tree, obtain the data block list corresponding to the embedded firmware file, and output the data block list through the display interface.

[0026] It should be noted that embedded firmware files can include firmware files in formats such as S19, Intel HEX, Hex-ASCII, Binary, C-Array, CANape, MIME, GMM, and Ford-VBF.

[0027] It should be noted that streaming parsing can be file parsing based on the Hierarchical Sparse Index Algorithm (HSIA).

[0028] It should be noted that, when the file size of the embedded firmware file is determined to be less than or equal to a preset file size threshold, the embedded firmware file is subjected to full parsing to obtain a list of data blocks corresponding to the embedded firmware file, and the list of data blocks is output through the display interface.

[0029] It should be noted that, preferably, the preset file size threshold can be 100MB.

[0030] It should be noted that the display interface may include the human-computer interaction display interface of the device running the file parsing tool.

[0031] In this way, by acquiring the embedded firmware file to be processed, which includes file size and file type, and then determining that the file size of the embedded firmware file is greater than a preset file size threshold, streaming parsing processing can be performed on the embedded firmware file to obtain metadata corresponding to at least one data record. Based on the metadata corresponding to each data record, a first-level file index table is constructed using a first construction strategy. Based on the first-level file index table, a second-level address range tree is constructed using a second construction strategy. Based on a preset byte size, a third-level block-level cache is constructed using a third construction strategy. Based on the second-level address range tree, a list of data blocks corresponding to the embedded firmware file is obtained, and the list of data blocks is output through a display interface. Since only a lightweight hierarchical index is constructed during file parsing without loading actual data, and memory usage is controlled through block-level caching, low-memory, fast-start, and on-demand loading visual parsing of large-capacity embedded firmware files is achieved.

[0032] Optionally, in one possible implementation of this embodiment, in step 102, firstly, the embedded firmware file can be read line by line in a streaming manner to obtain at least one data record. Secondly, based on the file type of the embedded firmware file, the header of the at least one data record can be parsed to obtain key data corresponding to the at least one data record. Thirdly, based on the key data corresponding to the at least one data record, metadata corresponding to the at least one data record can be obtained.

[0033] In a specific implementation of this method, firstly, each data record can be filtered based on the file type of the embedded firmware file to obtain valid data records. Secondly, for any given data record, the header information of the valid data record can be extracted to obtain the key data corresponding to the data record. In this implementation, key data may include byte offset (file_offset), line number (line_num), starting address (address), data length (length), and raw hexadecimal data string (data_hex).

[0034] Here, line numbers can be used for debugging and location. The starting address can be a processed extended address. For IntelHEX files, the context state of the current extended linear address (type 04) or extended segment address (type 02) can be maintained. For S19 files, the address field is correctly resolved according to the record type (24-bit address for S2, 32-bit address for S3) to achieve extended address processing. The data length can be the number of bytes. The raw hexadecimal data string can be used for subsequent on-demand parsing.

[0035] In another specific implementation of this approach, the type of each data record can be identified based on the file type of the embedded firmware file to obtain the record type corresponding to each data record. Based on the record type corresponding to each data record, valid data records are obtained.

[0036] In this implementation, the record type can include S19 and Intel HEX, etc. For example, it can include S0 / S1 / S2 / S3 / S9 of S19, and 00 / 01 / 02 / 03 / 04 / 05 of Intel HEX.

[0037] In this way, the embedded firmware file can be read line by line in a streaming manner to obtain at least one data record. Then, based on the file type of the embedded firmware file, the record header of each data record can be parsed to obtain more accurate and effective metadata corresponding to each data record, so as to ensure the accuracy of the first-level file index table subsequently constructed.

[0038] Optionally, in one possible implementation of this embodiment, in step 103, at least one tuple can be determined based on the metadata corresponding to each data record, and then a first-level file index table can be constructed based on the at least one tuple.

[0039] In a specific implementation of this approach, for any given data record's metadata, the metadata can be treated as a tuple, which can then serve as an index entry in the first-level file index table. The first-level file index table can be constructed based on each tuple.

[0040] In this implementation, for example, the tuple, i.e., the index entry of the first-level file index table, is (file_offset, line_num, address, length, data_hex). The index entry of the first-level file index table may also include the identifier of the index entry, file_refs, for example, the identifier may be represented as l1_idx.

[0041] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the foregoing implementation methods to implement the file parsing method of this embodiment. Detailed descriptions can be found in the relevant content of the foregoing implementation methods, and will not be repeated here.

[0042] Optionally, in one possible implementation of this embodiment, in step 104, firstly, the address range of each data record in the first-level file index table can be determined. Secondly, a second-level address range tree can be constructed based on each data record in the first-level file index table and the address range of each data record.

[0043] In this implementation, the second-level address range tree can be an address range tree based on a variant of the B+ tree. This tree structure can be optimized for range queries, supporting range overlap detection, adjacent range merging, and dynamic splitting.

[0044] In a specific implementation of this approach, for any given data record, the address range of that data record can be calculated based on its metadata.

[0045] One specific implementation involves calculating the complete address range corresponding to a data record based on its starting address and length.

[0046] Here, the address range of a data record can be represented as [address, address + length - 1]. In one specific implementation of this approach, the address range of each data record and the metadata of each data record in the first-level file index table can be inserted into the nodes of the second-level address range tree to obtain the second-level address range tree.

[0047] Here, the first-level file index table can include metadata of data records, namely the index entries of the first-level file index table and the identifiers of the index entries corresponding to the data records. The address range of each data record and the identifier of the index entry corresponding to each data record can be inserted into each node of the second-level address range tree.

[0048] One implementation method involves constructing each node of a second-level address range tree based on the address range of each data record and the identifier of the corresponding index entry, thus obtaining the second-level address range tree.

[0049] For example, the information in a node of the second-level address range tree can be represented as [address, end_address, l1_idx]. Understandably, when a newly inserted address range overlaps with an existing node's address range, the tree structure can automatically handle the index associations of the overlapping areas. If the new address range is contiguous with an existing address range, it can be merged into a larger range node to optimize storage efficiency. When the number of ranges stored in a node exceeds a threshold, it automatically splits into child nodes to maintain the tree's balance.

[0050] In this way, a second-level address range tree can be constructed based on the address range of each data record in the first-level file index table, so as to ensure the efficiency and reliability of subsequent data queries.

[0051] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the foregoing implementation methods to implement the file parsing method of this embodiment. Detailed descriptions can be found in the relevant content of the foregoing implementation methods, and will not be repeated here.

[0052] Optionally, in one possible implementation of this embodiment, in step 106, firstly, the address range of the nodes in the second-level address range tree can be determined based on the second-level address range tree. Secondly, the address ranges of the nodes in the second-level address range tree can be merged according to the order of their address ranges to obtain the data block list corresponding to the embedded firmware file.

[0053] In this implementation, the nodes of the second-level address range tree can include a root node and child nodes. The nodes of the second-level address range tree can be all nodes in the second-level address range tree.

[0054] In a specific implementation of this method, the starting address of the address range of the nodes in the second-level address range tree can be determined based on the address range of the nodes in the second-level address range tree. Based on the starting address of the address range of the nodes in the second-level address range tree, the order of the address ranges of the nodes is obtained. According to the order of the address ranges of the nodes, the address ranges of the overlapping nodes in the second-level address range tree are merged to obtain multiple data blocks. Based on the multiple data blocks, a list of data blocks corresponding to the embedded firmware file is obtained.

[0055] For example, if the starting address of the current range is less than or equal to the ending address of the previous range plus 1, then they are merged into a larger contiguous range. The new ending address of the merged range is the maximum of the two ending addresses.

[0056] In this implementation, a data block may include a start address (start), an end address (end), and a length (length = end - start + 1).

[0057] In this implementation, the list of data blocks can be represented as: Block 1: 0x08000000 - 0x0801FFFF (128KB); Block 2: 0x08020000 - 0x0803FFFF (128KB); ... and so on.

[0058] Understandably, the list of data blocks can be returned to the upper-layer application, such as a GUI interface, and displayed as selectable contiguous address ranges. In this case, all operations involve only index metadata, without loading any actual data load, ensuring rapid interface response and low memory consumption.

[0059] In this way, the list of data blocks corresponding to the embedded firmware file can be determined by the second-level address range tree. This allows for querying based on a list of consecutive address ranges of valid data blocks without triggering actual data loading, which further reduces memory usage and response time.

[0060] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the foregoing implementation methods to implement the file parsing method of this embodiment. Detailed descriptions can be found in the relevant content of the foregoing implementation methods, and will not be repeated here.

[0061] Optionally, in one possible implementation of this embodiment, after step 101, further, in response to the selection operation of the first data block in the data block list, the cache data corresponding to the first data block can be queried in the third-level block-level cache. If the cache data corresponding to the first data block is not found in the third-level block-level cache, the metadata corresponding to the first data block in the first-level file index table is determined based on the first data block and the second-level address range tree. Based on the metadata corresponding to the first data block, the memory data corresponding to the first data block is obtained, and the memory data corresponding to the first data block is exported according to a specified format.

[0062] In this implementation, the third-level block cache can be a Least Recently Used (LRU) cache container. The cache key can include an address range tuple (start_addr, end_addr) that identifies the contiguous address range covered by the cache block. The cache value can include a contiguous array of bytes within that address range, i.e., the actual firmware data.

[0063] Here, the preset byte size can be determined based on the actual business scenario. An LRU cache container with byte-limited cache is possible. When caching a new data block, if the capacity threshold of the third-level block cache is exceeded, the least recently used data block can be deleted to cache the new data block.

[0064] Understandably, the cache of the third-level block cache can be queried based on the first data block using `get_block(start_addr, end_addr)`. If a match is found, the data is returned and the access time is updated; if a miss is found, the data is loaded from the file and stored in the cache. Since address range queries typically exhibit spatial locality (access to contiguous addresses), the LRU cache can significantly reduce the number of disk I / O operations.

[0065] In a specific implementation of this method, the metadata corresponding to the first data block in the first-level file index table can be located based on the byte offset in the metadata corresponding to the first data block. The original hexadecimal data string in the metadata corresponding to the first data block in the first-level file index table is parsed and processed. Based on the result of the parsing and processing, the memory data corresponding to the first data block is obtained.

[0066] Here, the result of the parsing process can include data information of points that have a mapping relationship with the first data block, which can be represented as {address: byte}.

[0067] In another specific implementation of this approach, the memory data corresponding to the first data block can be stored in a third-level block cache.

[0068] In another specific implementation of this method, the memory data corresponding to the first data block can be formatted as a Hex Dump for output to the display interface.

[0069] In this implementation, the specified format can be determined according to the original equipment manufacturer's requirements. For example, the specified format may include Ford IHEX / VBF, GMM, Fiat, GAC, etc.

[0070] Understandably, uniform memory data {address: byte} can be exported to various specified formats.

[0071] In another specific implementation of this approach, the source data can be merged into the current data based on the source file's selected address range, the current data block's address range, and a specified target offset, thus overwriting the target address range.

[0072] In another specific implementation of this approach, the memory data file can be compared, and the difference bytes in the comparison results can be highlighted.

[0073] Understandably, the first data block can be determined based on user requirements. Users can select the appropriate data block to obtain the corresponding memory data based on their specific needs.

[0074] In this implementation, Figure 2 A schematic diagram of a hierarchical sparse index architecture provided in one embodiment of this application is shown, as follows: Figure 2As shown. The HSIA three-level index structure for firmware formats such as S19 / Intel HEX includes an L1 file location index, an L2 address range tree, and an L3 block cache. The L1 file location index can be a first-level file index table, the L2 address range tree can be a second-level address range tree, and the L3 block cache can be a third-level block cache. The storage metadata structure of the L1 file location index is: offset1, line1, addr1, len1, hex1. Here, only metadata is stored, not the parsed byte data. Metadata example [(0x100,1,0x8000000,16,'S123…'), (0x200,1,0x8000010,32,'S223…')]. The L2 address range tree is constructed by passing file references via file_ref. For example, the structure of an L2 address range tree may include a root node [0x8000000, 0x800FFFF], left child nodes [0x8000000, 0x8007FFF], refref: [1, 2, 3], and right child nodes [0x8008000, 0x800FFFF], refref: [4, 5]. The query method for an L2 address range tree can be `query_range(start, end)`. The query interface can implement the address range query method `query_range(start_addr, end_addr)`, which returns a list of all L1 index entries covering the specified address range. For example, `query_range(start, end) → List[l1_index]`. Here, the insertion time complexity of an L2 address range tree is O(log n), and the query time complexity is O(log n + k), where k is the number of result records. An L2 address range tree can be a tree structure based on a B+ variant. Address range queries can be performed using `query_range(start, end)`. During the query, each child node can be recursively queried, and all query results are merged and deduplicated to obtain the final list of L1 index entry references, which are the corresponding index entry identifiers in the first-level file index table. The L3 block-level LRU cache has a key-value structure: Key:(start_addr, end_addr); Value: bytes. The caching strategy is based on a limit on the total number of bytes; exceeding the limit evicts the least recently used block. Example: (0x8000000, 0x800FFFF) → [0x12, 0x34, 0x56, ...].

[0075] For example, in this instance, after obtaining the firmware file to be processed, firstly, it can be determined whether the file size of the firmware file is greater than 100MB. If not, the firmware file can be fully parsed to obtain the corresponding data block list. If so, HSIA mode can be started to perform streaming parsing of the firmware file, constructing an L1 index and storing metadata for data records, then constructing an L2 address range tree, determining the tree nodes including address ranges and L1 index identifiers, and initializing the L3 cache to obtain the corresponding data block list. The data block list includes metadata but does not load the parsed memory data. Secondly, the output data block list is displayed. The user selects the required data block and can query the L3 cache. If a match is found, the returned data can be obtained directly. If no match is found, the L1 index identifier can be determined through the L2 address range tree, the metadata corresponding to the L1 index identifier can be determined, the data record can be read according to the byte offset, and the hex of the data record can be parsed to obtain the memory data {address, bytes}, and the memory data can be written to the L3 cache. Finally, the memory data corresponding to the data block required by the user can be displayed, i.e., the parsed data.

[0076] In this way, through the hierarchical sparse index architecture L1, L2, and L3 and their collaborative work, the total memory usage for file parsing can be kept under control, enabling the processing of firmware files in the hundreds of MB range. Index building is a single-pass streaming scan, eliminating the need to parse and store all data. Large files can be indexed and a list of data blocks can be displayed within tens of seconds, significantly reducing startup time.

[0077] Furthermore, the solution in this embodiment can automatically merge adjacent blocks by storing only the address range containing data in the L2 tree, thus avoiding the storage of empty addresses.

[0078] Furthermore, the solution in this embodiment can achieve efficient on-demand loading by address range through more efficient address range-based loading. Users select data blocks by address, and the system queries and loads them by address range, without needing to concern itself with the internal line structure of the file.

[0079] Furthermore, the solution adopted in this embodiment can automatically select full parsing or HSIA mode based on the file size, balancing the performance of small files with the processability of large files, and enabling dual-mode switching for firmware file parsing.

[0080] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the foregoing implementation methods to implement the file parsing method of this embodiment. Detailed descriptions can be found in the relevant content of the foregoing implementation methods, and will not be repeated here.

[0081] Figure 3 A structural block diagram of a file parsing apparatus according to an embodiment of this application is shown, as follows: Figure 3As shown. The file parsing apparatus 300 of this embodiment may include a first acquisition unit 301, a first parsing unit 302, a first construction unit 303, a second construction unit 304, a third construction unit 305, and a first display unit 306. Specifically, the first acquisition unit 301 is used to acquire the embedded firmware file to be processed, the embedded firmware file including file size and file type; the first parsing unit 302 is used to perform streaming parsing processing on the embedded firmware file when the file size of the embedded firmware file is determined to be greater than a preset file size threshold, to obtain metadata corresponding to at least one data record; the first construction unit 303 is used to construct a first-level file index table based on the metadata corresponding to each data record using a first construction strategy; the second construction unit 304 is used to construct a second-level address range tree based on the first-level file index table using a second construction strategy; the third construction unit 305 is used to construct a third-level block-level buffer based on a preset byte size using a third construction strategy; and the first display unit 306 is used to obtain a list of data blocks corresponding to the embedded firmware file based on the second-level address range tree, and output the list of data blocks through a display interface.

[0082] Optionally, in one possible implementation of this embodiment, the first parsing unit 302 is used to perform line-by-line reading processing on the embedded firmware file in a streaming manner to obtain at least one data record; based on the file type of the embedded firmware file, perform record header parsing processing on the at least one data record to obtain key data corresponding to the at least one data record; and based on the key data corresponding to the at least one data record, obtain metadata corresponding to the at least one data record.

[0083] Optionally, in one possible implementation of this embodiment, the first construction unit 303 is used to determine at least one tuple based on the metadata corresponding to each data record; and to construct a first-level file index table based on the at least one tuple.

[0084] Optionally, in one possible implementation of this embodiment, the second construction unit 304 is used to determine the address range of each data record in the first-level file index table; and to construct a second-level address range tree based on each data record in the first-level file index table and the address range of each data record.

[0085] Optionally, in one possible implementation of this embodiment, the first display unit 306 is used to determine the address range of the nodes of the second-level address range tree based on the second-level address range tree; and to merge the address ranges of the nodes of the second-level address range tree according to the order of the address ranges of the nodes to obtain the data block list corresponding to the embedded firmware file.

[0086] Optionally, in one possible implementation of this embodiment, the first display unit 306 is configured to, in response to a selection operation on a first data block in the data block list, query the cached data corresponding to the first data block in the third-level block-level cache; if the cached data corresponding to the first data block is not found in the third-level block-level cache, determine the metadata corresponding to the first data block in the first-level file index table based on the first data block and the second-level address range tree; obtain the memory data corresponding to the first data block based on the metadata corresponding to the first data block; and export the memory data corresponding to the first data block according to a specified format.

[0087] In this embodiment, the embedded firmware file to be processed can be obtained by the first acquisition unit. The embedded firmware file includes file size and file type. When the first parsing unit determines that the file size of the embedded firmware file is greater than a preset file size threshold, it performs streaming parsing processing on the embedded firmware file to obtain metadata corresponding to at least one data record. Based on the metadata corresponding to each data record, the first construction unit constructs a first-level file index table using a first construction strategy. Based on the first-level file index table, the second construction unit constructs a second-level address range tree using a second construction strategy. Based on a preset byte size, the third construction unit constructs a third-level block-level cache using a third construction strategy. Based on the second-level address range tree, the first display unit obtains the data block list corresponding to the embedded firmware file and outputs the data block list through the display interface. Since a lightweight hierarchical index can be constructed based only on the metadata of the data records during file parsing, there is no need to load the actual data to be parsed. Memory usage is controlled through block-level caching, achieving low memory usage, fast startup, and visual parsing of large-capacity embedded firmware files.

[0088] The technical solution of this application involves the collection, storage, use, processing, transmission, provision, and disclosure of user personal information, such as user image and attribute data, which comply with relevant laws and regulations and do not violate public order and good morals.

[0089] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0090] Figure 4A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0091] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0092] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0093] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the file parsing method. For example, in some embodiments, the file parsing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the file parsing method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the file parsing method by any other suitable means (e.g., by means of firmware).

[0094] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0098] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0099] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0100] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0101] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of file parsing, characterized by, The method includes: Obtain the embedded firmware file to be processed, the embedded firmware file including file size and file type; If the file size of the embedded firmware file is determined to be greater than a preset file size threshold, the embedded firmware file is subjected to streaming parsing to obtain metadata corresponding to at least one data record. Based on the metadata corresponding to each data record, the first-level file index table is constructed using the first construction strategy; Based on the first-level file index table, a second-level address range tree is constructed using the second construction strategy; Based on the preset byte size, a third-level block cache is constructed using a third construction strategy; Based on the second-level address range tree, a list of data blocks corresponding to the embedded firmware file is obtained, and the list of data blocks is output through the display interface.

2. The method of claim 1, wherein, The step of performing streaming parsing on the embedded firmware file to obtain metadata corresponding to at least one data record includes: The embedded firmware file is read line by line in a streaming manner to obtain at least one data record; Based on the file type of the embedded firmware file, record header parsing is performed on at least one data record to obtain key data corresponding to at least one data record. Based on the key data corresponding to at least one data record, obtain the metadata corresponding to at least one data record.

3. The method of claim 1, wherein, The construction of a first-level file index table based on the metadata corresponding to each data record, using a first construction strategy, includes: Based on the metadata corresponding to each data record, determine at least one tuple; Construct a first-level file index table based on at least one tuple.

4. The method of claim 1, wherein, The construction of a second-level address range tree based on the first-level file index table and using a second construction strategy includes: Determine the address range of each data record in the first-level file index table; Based on each data record and its address range in the first-level file index table, a second-level address range tree is constructed.

5. The method of claim 1, wherein, The step of obtaining the data block list corresponding to the embedded firmware file based on the second-level address range tree includes: Based on the second-level address range tree, determine the address range of the nodes in the second-level address range tree; The address ranges of the nodes in the second-level address range tree are merged according to the order of the node address ranges to obtain the data block list corresponding to the embedded firmware file.

6. The method according to claim 1, characterized in that, After outputting the data block list through the display interface, the process includes: In response to the selection operation of the first data block in the data block list, the cached data corresponding to the first data block is queried in the third-level block cache; If the cached data corresponding to the first data block is not found in the third-level block cache, the metadata corresponding to the first data block in the first-level file index table is determined based on the first data block and the second-level address range tree. Based on the metadata corresponding to the first data block, obtain the memory data corresponding to the first data block; The memory data corresponding to the first data block is exported according to the specified format.

7. A file parsing apparatus, characterized in that, The device includes: The first acquisition unit is used to acquire the embedded firmware file to be processed, wherein the embedded firmware file includes file size and file type; The first parsing unit is used to perform streaming parsing processing on the embedded firmware file when it is determined that the file size of the embedded firmware file is greater than a preset file size threshold, so as to obtain metadata corresponding to at least one data record. The first construction unit is used to construct the first-level file index table based on the metadata corresponding to each data record and using the first construction strategy. The second construction unit is used to construct a second-level address range tree based on the first-level file index table and using a second construction strategy. The third building unit is used to build a third-level block cache based on a preset byte size and using a third building strategy. The first display unit is used to obtain a list of data blocks corresponding to the embedded firmware file based on the second-level address range tree, so as to output the list of data blocks through the display interface.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images