Method and system for firmware upgrade management of charging pile terminal for two-wheel electric vehicle

Abstract

The application provides a firmware upgrade management method and system for a two-wheel electric vehicle charging pile terminal, which comprises splicing old firmware and new firmware of the charging pile into a source data stream in the order of old firmware first and new firmware last, segmenting based on fixed-length instruction codes and data boundary symbols, and converting into a variable-length semantic block symbol sequence; a multi-pass merge sort is used to construct a suffix array for the external storage symbol sequence, a composite sorting key is composed of a semantic block prefix, a type identifier and an original offset, the original offset is used as a source positioning and stable sorting auxiliary field, and the prefix comparison length is adjusted with the number of merge passes; a length threshold is calculated based on the longest common prefix distribution of an ordered subsequence; in the merging stage, adjacent suffixes sharing an ultra-long common prefix are aggregated into a strongly related suffix cluster and a cluster prefix digest is generated, the suffix array and the digest are used to accelerate retrieval, a greedy algorithm is used to match reusable sequences in the old firmware offset interval, and a differential upgrade package containing copy and addition instructions is output.

Description

Technical Field

[0001] This application belongs to the field of charging piles for two-wheeled electric vehicles, and particularly relates to a firmware upgrade management method and system for charging pile terminals for two-wheeled electric vehicles. Background Technology

[0002] In recent years, with the popularization of green travel concepts, the number of two-wheeled electric vehicles has continued to grow, and the deployment of charging piles for these vehicles, as an important part of urban infrastructure, has also gradually expanded. These widely distributed charging pile terminals, as important nodes in the Internet of Things (IoT) system, require firmware upgrades to fix potential system vulnerabilities, optimize charging control logic, and expand intelligent business functions during long-term operation. Currently, OTA (Over-The-Air) remote upgrades are a common method for updating terminal firmware. However, two-wheeled electric vehicle charging pile terminals are typically deployed in complex outdoor network environments, mainly relying on cellular networks for communication, which suffers from limited network bandwidth, insufficient signal stability, and high data traffic costs. Simultaneously, the terminal hardware typically uses low-cost microcontrollers, with limited computing power and internal storage resources. Upgrading by distributing a complete new firmware not only consumes a large amount of network traffic and prolongs transmission time but is also prone to download interruptions due to network fluctuations, affecting device availability. Therefore, differential upgrade technology, which extracts the differences between the old and new firmware to generate smaller differential upgrade packages, has become an important means to reduce the bandwidth consumption of charging pile terminal firmware upgrades and improve the upgrade success rate.

[0003] In the differential upgrade package generation process, the key lies in finding identical or highly similar data sequences between the old and new firmware, thereby extracting reusable code segments and data sequences as much as possible. Suffix arrays, as an efficient string indexing structure, are commonly used for massive data matching and difference calculation. However, traditional suffix array construction algorithms are mostly based on pure in-memory computation, typically requiring the entire data to be processed to be loaded into memory. For systems with limited hardware resources, or when dealing with ever-increasing firmware data, memory capacity constraints often necessitate the use of external storage for suffix array construction. Conventional external storage suffix array construction algorithms tend to generate frequent external storage I / O read / write operations when processing binary files like firmware that contain variable instructions and a large number of similar code segments, leading to reduced construction efficiency. Furthermore, existing algorithms often lack semantic awareness of the underlying firmware instruction structure, making it difficult to effectively cluster and quickly skip large numbers of data segments with long common prefixes, resulting in redundant comparisons and invalid searches. Therefore, how to optimize the construction process of the suffix array based on external memory, reduce the complexity of sorting and matching massive firmware data, and generate high-compression differential upgrade packages for two-wheeled electric vehicle charging pile terminals is an urgent problem to be solved in the current field of IoT device operation and maintenance. Summary of the Invention

[0004] According to one aspect of the present invention, a firmware upgrade management method for a two-wheeled electric vehicle charging pile terminal is proposed to solve the problem of how to optimize the construction process of the suffix array based on external memory, reduce the complexity of sorting and matching massive amounts of data, and generate a high-compression differential upgrade package for the two-wheeled electric vehicle charging pile terminal, comprising the following steps: The old firmware and new firmware of the two-wheeled electric vehicle charging pile are concatenated into a continuous source data stream according to a preset order; wherein, the preset order is old firmware first and new firmware second, so that the offset range of the source data stream file corresponding to the old firmware is a continuous range from 0 to the total byte length of the old firmware; the source data stream is segmented based on fixed-length instruction codes and preset data boundary symbols, and the source data stream is converted into a symbol sequence composed of multiple variable-length semantic blocks; A suffix array is constructed from the symbol sequence in external storage. The construction process of the suffix array includes multiple passes of merge sorting to establish a composite sorting key consisting of semantic block prefixes, type identifiers, and original offsets. In each pass of merge sorting, the composite sorting key is used for comparison, and the length of the semantic block prefixes participating in the comparison in the composite sorting key is adjusted according to the current pass number. A length threshold is calculated based on the statistical distribution of the longest common prefix value of the generated ordered subsequences. During the merging phase of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated. Based on the suffix array and the strongly correlated suffix clusters, the cluster prefix digest is used to accelerate the search. A greedy algorithm is used to traverse the new firmware symbol sequence, and the reusable sequence of the old firmware is matched within the original offset interval corresponding to the old firmware to generate a differential upgrade package containing copy instructions and add instructions.

[0005] Optionally, the step of segmenting the source data stream based on fixed-length instruction codes and preset data boundary symbols, and converting the source data stream into a symbol sequence composed of multiple variable-length semantic blocks, includes: The memory mapping boundary parameters of code segments and data segments are extracted from the configuration files of the old firmware and the new firmware, and the actual boundary parameters in the source data stream are calculated by combining them with the file offset increment caused by concatenation. Within the actual boundary range of the code segment, data is read sequentially according to the preset fixed-length instruction code step size, the continuous instructions are divided into code semantic blocks, and the first type identifier is written into the symbol record corresponding to the code semantic block; Within the actual boundary range of the data segment, the data is segmented by scanning a pre-defined data boundary symbol to obtain a data semantic block, and a second type of identifier is written into the symbol record corresponding to the data semantic block; The code semantic blocks and data semantic blocks are arranged in order of their original offsets in the source data stream file to generate the symbol sequence.

[0006] Optionally, establishing a composite sorting key consisting of a semantic block prefix, a type identifier, and the original offset includes: Allocate a variable-length structure in memory as storage space for the composite sort key; Using each semantic block in the symbol sequence as a suffix starting point, the first byte sequence is continuously read and extracted sequentially along the subsequent semantic blocks in the symbol sequence, and used as the prefix of the semantic block; Extract the type identifier of the current semantic block; Read the starting offset of the current semantic block in the source data stream file as the original offset; The extracted semantic block prefix, the type identifier, and the original offset are concatenated sequentially to generate a composite sorting key for the corresponding semantic block and stored in the storage space. The original offset serves as a suffix source location field and a stable sorting auxiliary field, and does not replace the extended comparison of subsequent semantic block prefixes when the semantic block prefix and type identifier are not completely distinguished.

[0007] Optionally, the step of using the composite sort key for comparison in each merge sort pass, and adjusting the length of the semantic block prefixes participating in the comparison in the composite sort key according to the current pass number, includes: Set the maximum number of execution passes for merge sort, and initialize the base value of the semantic block prefix length participating in the comparison to a preset fixed length; After the current round of merging and sorting is completed, the current round number is extracted and multiplied by the preset length growth coefficient to obtain the length update step size; Add the base value to the length update step size to obtain the expected prefix length for the next sorting pass; The expected prefix length is compared with the actual remaining byte length of the corresponding suffix. If it exceeds the actual remaining byte length, the prefix length of the semantic block in the next pass is truncated to the actual remaining byte length. In the comparison phase of the next sorting pass, the semantic block prefix substrings of the specified length in the two composite sort keys are first extracted and compared lexicographically. If the semantic block prefix substrings are the same, the type identifier is compared. For suffix equivalence intervals that are still not distinguished, the semantic block prefix length is extended in subsequent passes. The original offset is only used as an auxiliary field when a precise comparison is completed or when it is necessary to maintain the determinism of the output order.

[0008] Optionally, the step of calculating the length threshold based on the statistical distribution of the longest common prefix value of the generated ordered subsequences includes: Create a length record array to store the data from the merge sort process; When comparing adjacent suffixes in each merge sorted subsequence, calculate and obtain the longest common prefix value between them, and store the longest common prefix value into the length record array. Before proceeding to the final merging stage, calculate the arithmetic mean and standard deviation of all record values ​​in the length record array; The arithmetic mean is added to the product of a preset proportional coefficient and the standard deviation to obtain the benchmark value; The length threshold is derived by limiting the benchmark value to a pre-configured range of minimum and maximum allowable thresholds.

[0009] Optionally, in the merging stage of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated, including: Calculate the exact longest common prefix value of adjacent suffixes in the final merging stage; When the exact longest common prefix value of at least one pair of adjacent suffixes is not less than the length threshold, or when the exact longest common prefix value of adjacent suffixes in a continuous suffix interval is not less than the length threshold, the common prefix byte sequence shared by the corresponding suffixes is extracted. Determine the index prefix length, wherein the index prefix length is not greater than the actual byte length of the common prefix byte sequence; Extract an index prefix byte sequence with a length equal to the index prefix length from the common prefix byte sequence; The index prefix byte sequence is calculated using a preset hash function to obtain the index hash value; Calculate the actual byte length of the common prefix byte sequence; The index hash value, the index prefix length, and the actual byte length are associated and stored to generate a cluster prefix digest of the strongly related suffix cluster.

[0010] According to another aspect of the present invention, a firmware upgrade management system for a charging pile terminal for two-wheeled electric vehicles is proposed, comprising the following modules: The conversion module is used to concatenate the old firmware and new firmware of the two-wheeled electric vehicle charging pile into a continuous source data stream according to a preset order; wherein, the preset order is old firmware first and new firmware second, so that the offset range of the source data stream file corresponding to the old firmware is a continuous range from 0 to the total byte length of the old firmware; the source data stream is segmented based on fixed-length instruction codes and preset data boundary symbols, and the source data stream is converted into a symbol sequence composed of multiple variable-length semantic blocks; The construction module is used to construct a suffix array from the symbol sequence in external memory. The construction process of the suffix array includes multiple merge sorts to establish a composite sorting key consisting of semantic block prefixes, type identifiers, and original offsets. In each merge sort, the composite sorting key is used for comparison, and the length of the semantic block prefixes participating in the comparison in the composite sorting key is adjusted according to the current number of passes. The length threshold is calculated based on the statistical distribution of the longest common prefix value of the generated ordered subsequences. The generation module is used during the merging stage of suffix array construction. When the longest common prefix value of adjacent suffixes is not less than the length threshold, it summarizes consecutive suffixes sharing a long common prefix into a strongly correlated suffix cluster and generates a cluster prefix digest containing hash value and byte length. Based on the suffix array and the strongly correlated suffix clusters, it uses the cluster prefix digest to accelerate the search, traverses the new firmware symbol sequence using a greedy algorithm, matches the reusable sequence of the old firmware within the original offset interval corresponding to the old firmware, and generates a differential upgrade package containing copy instructions and add instructions.

[0011] Preferably, the step of segmenting the source data stream based on fixed-length instruction codes and preset data boundary symbols, and converting the source data stream into a symbol sequence composed of multiple variable-length semantic blocks, includes: The memory mapping boundary parameters of code segments and data segments are extracted from the configuration files of the old firmware and the new firmware, and the actual boundary parameters in the source data stream are calculated by combining them with the file offset increment caused by concatenation. Within the actual boundary range of the code segment, data is read sequentially according to the preset fixed-length instruction code step size, the continuous instructions are divided into code semantic blocks, and the first type identifier is written into the symbol record corresponding to the code semantic block; Within the actual boundary range of the data segment, the data is segmented by scanning a pre-defined data boundary symbol to obtain a data semantic block, and a second type of identifier is written into the symbol record corresponding to the data semantic block; The code semantic blocks and data semantic blocks are arranged in order of their original offsets in the source data stream file to generate the symbol sequence.

[0012] Preferably, establishing a composite sorting key consisting of a semantic block prefix, a type identifier, and the original offset includes: Allocate a variable-length structure in memory as storage space for the composite sort key; Using each semantic block in the symbol sequence as a suffix starting point, the first byte sequence is continuously read and extracted sequentially along the subsequent semantic blocks in the symbol sequence, and used as the prefix of the semantic block; Extract the type identifier of the current semantic block; Read the starting offset of the current semantic block in the source data stream file as the original offset; The extracted semantic block prefix, the type identifier, and the original offset are concatenated sequentially to generate a composite sorting key for the corresponding semantic block and stored in the storage space. The original offset serves as a suffix source location field and a stable sorting auxiliary field, and does not replace the extended comparison of subsequent semantic block prefixes when the semantic block prefix and type identifier are not completely distinguished.

[0013] Preferably, the step of using the composite sort key for comparison in each merge sort pass, and adjusting the length of the semantic block prefix participating in the comparison in the composite sort key according to the current pass number, includes: Set the maximum number of execution passes for merge sort, and initialize the base value of the semantic block prefix length participating in the comparison to a preset fixed length; After the current round of merging and sorting is completed, the current round number is extracted and multiplied by the preset length growth coefficient to obtain the length update step size; Add the base value to the length update step size to obtain the expected prefix length for the next sorting pass; The expected prefix length is compared with the actual remaining byte length of the corresponding suffix. If it exceeds the actual remaining byte length, the prefix length of the semantic block in the next pass is truncated to the actual remaining byte length. In the comparison phase of the next sorting pass, the semantic block prefix substrings of the specified length in the two composite sort keys are first extracted and compared lexicographically. If the semantic block prefix substrings are the same, the type identifier is compared. For suffix equivalence intervals that are still not distinguished, the semantic block prefix length is extended in subsequent passes. The original offset is only used as an auxiliary field when a precise comparison is completed or when it is necessary to maintain the determinism of the output order.

[0014] Preferably, the step of calculating the length threshold based on the statistical distribution of the longest common prefix value of the generated ordered subsequences includes: Create a length record array to store the data from the merge sort process; When comparing adjacent suffixes in each merge sorted subsequence, calculate and obtain the longest common prefix value between them, and store the longest common prefix value into the length record array. Before proceeding to the final merging stage, calculate the arithmetic mean and standard deviation of all record values ​​in the length record array; The arithmetic mean is added to the product of a preset proportional coefficient and the standard deviation to obtain the benchmark value; The length threshold is derived by limiting the benchmark value to a pre-configured range of minimum and maximum allowable thresholds.

[0015] Preferably, in the merging stage of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated, including: Calculate the exact longest common prefix value of adjacent suffixes in the final merging stage; When the exact longest common prefix value of at least one pair of adjacent suffixes is not less than the length threshold, or when the exact longest common prefix value of adjacent suffixes in a continuous suffix interval is not less than the length threshold, the common prefix byte sequence shared by the corresponding suffixes is extracted. Determine the index prefix length, wherein the index prefix length is not greater than the actual byte length of the common prefix byte sequence; Extract an index prefix byte sequence with a length equal to the index prefix length from the common prefix byte sequence; The index prefix byte sequence is calculated using a preset hash function to obtain the index hash value; Calculate the actual byte length of the common prefix byte sequence; The index hash value, the index prefix length, and the actual byte length are associated and stored to generate a cluster prefix digest of the strongly related suffix cluster.

[0016] This invention concatenates old and new firmware in order of old firmware first, followed by new firmware, and segments the source data stream based on fixed-length instruction codes and data boundary symbols, converting it into a variable-length semantic block symbol sequence, thus preserving the inherent correlation between firmware instructions and data structures. By storing mapping symbols, semantic block original byte content indexes, semantic block lengths, type identifiers, and source data stream file offsets in the symbol sequence, subsequent semantic block prefix extraction, longest common prefix calculation, and old firmware offset interval filtering all have a clear data source. When constructing the external storage suffix array, a composite sorting key containing semantic block prefixes, type identifiers, and original offsets is established, and the prefix lengths participating in comparison are dynamically adjusted according to the number of merge passes, reducing redundant reads and writes and invalid comparisons during the external storage sorting process. A length threshold is calculated based on the statistical distribution of the longest common prefix value, and adjacent suffixes exceeding the threshold are grouped into strongly correlated suffix clusters, and cluster prefix summaries are extracted for candidate sequence location. The cluster prefix summaries are used to accelerate the search process, and a greedy algorithm is used to quickly match reusable sequences within the old firmware offset interval, improving the efficiency of duplicate data retrieval. The generated differential upgrade package can reduce file size, reduce network transmission pressure and storage overhead during the upgrade process of two-wheeled electric vehicle charging pile terminals, and improve the efficiency and reliability of firmware updates. Attached Figure Description

[0017] Figure 1 A flowchart of the first embodiment; Figure 2 A schematic diagram showing the proportion of the composite sorting key structure; Figure 3 This is a bar chart showing the expected prefix length for each round. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the first embodiment, the present invention proposes a firmware upgrade management method for a two-wheeled electric vehicle charging pile terminal, such as... Figure 1 It includes the following steps: S1, the old firmware and new firmware of the two-wheeled electric vehicle charging pile are concatenated into a continuous source data stream according to a preset order; wherein, the preset order is old firmware first and new firmware second, so that the offset range of the source data stream file corresponding to the old firmware is a continuous range from 0 to the total byte length of the old firmware; the source data stream is segmented based on fixed-length instruction codes and preset data boundary symbols, and the source data stream is converted into a symbol sequence composed of multiple variable-length semantic blocks.

[0020] The program calls a file open and read function to read the old and new firmware binary files of the two-wheeled electric vehicle charging pile stored on external storage media. A file append function is then used to concatenate the data from the old and new firmware binary files sequentially, outputting the result to the local disk to obtain a single, continuous source data stream file. The operating system's memory mapping function is used to map the source data stream file into blocks in main memory. A circular pointer is set to read the memory data stream byte-by-byte in sequence. Within the actual boundary range of the code segment, fixed-length instruction boundaries are determined using a 4-byte alignment step. The read 32-bit machine code is bitwise ANDed with a preset embedded microcontroller reduced instruction set feature mask to verify if the current data meets the preset instruction format. Simultaneously, consecutive hexadecimal zero-byte or one-byte padding sequences in the data stream are used as preset data boundary markers. When the program has accumulated a preset number of fixed-length instructions in the code segment and completed the preset instruction format verification, or when a data boundary marker is matched in the data segment, a memory copy function is used to truncate and extract the data block between the current cursor and the previous boundary into an independent variable-length semantic block. For each variable-length semantic block, a symbol record is created. This symbol record includes a 32-bit unsigned integer hash value, the starting offset of the semantic block in the source data stream file, the byte length of the semantic block, the type identifier of the semantic block, and the storage index of the original byte content of the semantic block in the semantic block data area. The MurmurHash3 non-cryptographic hash algorithm is called to perform a hash calculation on each extracted variable-length semantic block, generating a corresponding 32-bit unsigned integer hash value. This hash value serves as a mapping symbol representing the characteristics of the semantic block, used for fast indexing and preliminary comparison, but does not replace the original byte content of the semantic block. All symbol records are appended to a temporary binary index file in the order they were truncated, and the corresponding original byte content of the semantic block is appended to the semantic block data file. The temporary binary index file and the semantic block data file together constitute the external storage structure of the symbol sequence. Thus, subsequent steps can extract the header byte sequence, read the type identifier, read the offset of the source data stream file, and calculate the longest common prefix value based on the symbol records.

[0021] In an optional embodiment, segmenting the source data stream based on fixed-length instruction codes and preset data boundary symbols, and converting the source data stream into a symbol sequence composed of multiple variable-length semantic blocks, includes: The memory mapping boundary parameters of code segments and data segments are extracted from the configuration files of the old firmware and the new firmware, and the actual boundary parameters in the source data stream are calculated by combining them with the file offset increment caused by concatenation. Within the actual boundary range of the code segment, data is read sequentially according to the preset fixed-length instruction code step size, the continuous instructions are divided into code semantic blocks, and the first type identifier is written into the symbol record corresponding to the code semantic block; Within the actual boundary range of the data segment, the data is segmented by scanning a pre-defined data boundary symbol to obtain a data semantic block, and a second type of identifier is written into the symbol record corresponding to the data semantic block; The code semantic blocks and data semantic blocks are arranged in order of their original offsets in the source data stream file to generate the symbol sequence.

[0022] The firmware linker script file, ELF segment table file, or firmware packaging configuration file are parsed to obtain the load address range of the code segment and data segment, as well as their file offset range in the firmware binary file. For example, the microcontroller load address range for the code segment is 0x08000000 to 0x08020000, corresponding to an offset range of 0x00000000 to 0x00020000 in the old firmware file; the microcontroller load address range for the data segment is 0x20000000 to 0x20005000, corresponding to an offset range of 0x00020000 to 0x00025000 in the old firmware file. The load address is used to identify the segment attributes of the code segment and data segment, and the file offset range is used to locate the actual read position in the source data stream file. Since the old and new firmware are concatenated consecutively in the source data stream, the total length of the old firmware is extracted, assumed to be 148KB, and the file offset 0x25000 is used as the concatenation offset of the new firmware in the source data stream. The file offset range of each segment of the new firmware in its independent firmware file is added to the concatenation offset to obtain the actual boundary parameters of each segment of the new firmware in the source data stream file. Therefore, the actual boundary parameters are all represented by the file offset of the source data stream, instead of using the absolute memory address of the microcontroller at runtime for subsequent matching and filtering.

[0023] Within the actual boundary range of the code segment, the fixed-length instruction code step size is set to 4 bytes. Following the rule of treating every 64 consecutive instructions as a whole, it is divided into code semantic blocks of fixed length 256 bytes, and a first type identifier with the value 0x01 is written into the symbol record corresponding to each code semantic block. Within the actual boundary range of the data segment, a consecutive 0x00 or 0xFF padding sequence of no less than a preset length is set as a preset data boundary symbol. The source data stream is scanned byte by byte; when a boundary symbol is matched, truncation logic is triggered to obtain a data semantic block, the actual length of which is usually between 8 and 512 bytes, and a second type identifier with the value 0x02 is written into the symbol record corresponding to this data semantic block. The code semantic blocks and data semantic blocks with corresponding type identifier symbol records are arranged in ascending order of their starting offsets in the source data stream file to generate the symbol sequence.

[0024] S2, construct a suffix array from the symbol sequence in external memory. The construction process of the suffix array includes multiple merge sorts to establish a composite sorting key consisting of semantic block prefixes, type identifiers, and original offsets. In each merge sort, the composite sorting key is used for comparison, and the length of the semantic block prefixes participating in the comparison in the composite sorting key is adjusted according to the current number of passes. The length threshold is calculated based on the statistical distribution of the longest common prefix value of the generated ordered subsequences.

[0025] Given that the symbol sequence is too large to exceed the available main memory, a merge sort algorithm architecture based on external memory is adopted. The symbol records in the temporary binary index file are divided into multiple fixed-capacity data blocks according to the system page frame size and read alternately into memory. For each data block read into memory, a composite sort key structure is established by traversing the internal symbol records. This composite sort key structure includes a semantic block prefix obtained by sequentially reading and extracting subsequent semantic blocks in the symbol sequence, starting from the corresponding semantic block; a business type identifier to distinguish between code semantic blocks and data semantic blocks; and the original offset of the semantic block in the source data stream file. When the remaining byte length of the current semantic block is insufficient to meet the prefix length required for the current pass, the original byte content of the next semantic block is read and concatenated until the required prefix length is reached or the end of the source data stream is reached. If the algorithm needs to mark the suffix sort category internally, a suffix sort category flag can be set separately. This suffix sort category flag is only used for merge sort control flow and is not used as the type identifier described in this embodiment, nor does it replace the business type identifier used to distinguish between code semantic blocks and data semantic blocks.

[0026] The introspective sorting function is invoked to perform the first internal sort on the composite sort keys within each data block, and the resulting ordered subsequences are persistently written back to a temporary file in external storage. The multi-way merge sort phase then begins, instantiating a priority queue container in memory as a min-heap data structure to maintain the order of the multiple input streams. During the Nth merge sort loop, the prefix length of the semantic blocks being compared is set to the sum of the base value and the current pass number multiplied by a preset length growth factor. ,in The base value is d, where d is the preset length growth coefficient and N is the current pass count. When the prefix content is completely identical, the business type identifier is compared first. If the business type identifier is still identical, the semantic block prefix length is increased in the next merge pass to continue the differentiation. When the actual remaining byte length of either suffix has been reached and the compared parts are completely identical, the suffix with the shorter actual remaining byte length is placed first. If the actual remaining byte lengths of the two suffixes are the same, the stable order is determined based on the original offset. When the maximum number of merge sort passes has been reached and the suffix is ​​still not differentiated, the corresponding suffix is ​​kept in the long prefix equivalence interval, and the equivalent interval is subjected to on-demand prefix extension comparison or direct byte-precise comparison in the final merge or matching stage. The original offset is not used to replace the complete suffix lexicographical comparison. While the multi-way merge sort continuously writes the results of each pass to external storage, it performs direct byte comparisons on adjacent suffixes in the currently merged ordered subsequence, calculates the temporary longest common prefix value, and pushes the obtained integer value into a global statistics array. This temporary longest common prefix value is used to estimate the length threshold, without using the Kasai algorithm which relies on the complete suffix array and ranking array. The arithmetic mean and standard deviation of all common prefix values ​​in this global array are calculated using traversal and accumulation operations. The calculated arithmetic mean and standard deviation are combined according to a preset proportional coefficient and, after amplitude limiting, set as the length threshold for subsequent judgment. Each merge sort pass regenerates the composite sort key based on the semantic block prefix length of the current pass, and re-executes internal sorting and external storage merging on undifferentiated equivalent intervals to ensure that the subsequences participating in the merge in the current pass satisfy the current comparison rules.

[0027] In an optional embodiment, establishing the composite sorting key consisting of a semantic block prefix, a type identifier, and the original offset includes: Allocate a variable-length structure in memory as storage space for the composite sort key; Using each semantic block in the symbol sequence as a suffix starting point, the first byte sequence is continuously read and extracted sequentially along the subsequent semantic blocks in the symbol sequence, and used as the prefix of the semantic block; Extract the type identifier of the current semantic block; Read the starting offset of the current semantic block in the source data stream file as the original offset; The extracted semantic block prefix, the type identifier, and the original offset are concatenated sequentially to generate a composite sorting key for the corresponding semantic block and stored in the storage space. The original offset serves as a suffix source location field and a stable sorting auxiliary field, and does not replace the extended comparison of subsequent semantic block prefixes when the semantic block prefix and type identifier are not completely distinguished.

[0028] The memory allocation interface is invoked to allocate a variable-length structure in memory, containing a prefix length field, a prefix buffer, a type identifier field, and a raw offset field, as storage space for the composite sort key. The processed symbol sequence is traversed. For each symbol record, based on the index of the original byte content of the semantic block stored in that symbol record, the corresponding semantic block is used as the suffix starting point. Following the order of subsequent semantic blocks in the symbol sequence, the original byte content is read continuously across one or more semantic blocks, and a continuous byte sequence of the required length for the current merge sort pass is extracted as the semantic block prefix. If the current semantic block length is insufficient, the original byte content of the next semantic block is read and concatenated. The truncation length of this prefix is ​​determined based on the current merge sort pass; the initial truncation length is, for example, set to 8 bytes. A single-byte business type identifier (0x01 or 0x02) is extracted from the symbol record, and the starting offset of the current semantic block in the source data stream file (32-bit unsigned integer form) is read, for example, the file offset 0x00001040, as the raw offset. Following the logical order of "semantic block prefix, type identifier, and original offset," the obtained fields are concatenated sequentially to generate a composite sort key, which is then stored in the allocated storage space. The proportions of the composite sort key structure are illustrated below. Figure 2 As shown.

[0029] In an optional embodiment, the step of using the composite sort key for comparison in each merge sort pass, and adjusting the length of the semantic block prefixes involved in the comparison in the composite sort key according to the current pass number, includes: Set the maximum number of execution passes for merge sort, and initialize the base value of the semantic block prefix length participating in the comparison to a preset fixed length; After the current round of merging and sorting is completed, the current round number is extracted and multiplied by the preset length growth coefficient to obtain the length update step size; Add the base value to the length update step size to obtain the expected prefix length for the next sorting pass; The expected prefix length is compared with the actual remaining byte length of the corresponding suffix. If it exceeds the actual remaining byte length, the prefix length of the semantic block in the next pass is truncated to the actual remaining byte length. In the comparison phase of the next sorting pass, the semantic block prefix substrings of the specified length in the two composite sort keys are first extracted and compared lexicographically. If the semantic block prefix substrings are the same, the type identifier is compared. For suffix equivalence intervals that are still not distinguished, the semantic block prefix length is extended in subsequent passes. The original offset is only used as an auxiliary field when a precise comparison is completed or when it is necessary to maintain the determinism of the output order.

[0030] During the algorithm configuration phase, the maximum number of merge sort passes is set to 32, and the base value of the semantic block prefix length participating in the lexicographical comparison in the first pass is initialized to a fixed 4 bytes. After the Nth pass of merge sort is completed, N is between 1 and 32. The loop count value is read to extract the current pass number N, and this pass number is multiplied by a preset multiplication factor, such as 4, to calculate the length update step size. For example, after the 3rd pass, the step size increases to 12 bytes. The base value of 4 bytes is added to the length update step size of 12 bytes to obtain the expected prefix length of the next pass, which is 16 bytes. To avoid out-of-bounds access, the expected length of 16 bytes is compared with the actual remaining available byte length within the boundary of the suffix currently participating in the comparison operation. Assuming that the current suffix actually has 10 bytes remaining from the end, if it is determined to exceed the limit, the semantic block prefix comparison length of the next pass is truncated to this actual remaining byte length of 10 bytes. In the comparison phase of the next pass, only the 10-byte semantic block prefix substrings from the two composite sort keys are read for lexicographical comparison based on this truncated value. The expected prefix length for each round is illustrated in the bar chart below. Figure 3 As shown.

[0031] In an optional embodiment, calculating the length threshold based on the statistical distribution of the longest common prefix value of the generated ordered subsequences includes: Create a length record array to store the data from the merge sort process; When comparing adjacent suffixes in each merge sorted subsequence, calculate and obtain the longest common prefix value between them, and store the longest common prefix value into the length record array. Before proceeding to the final merging stage, calculate the arithmetic mean and standard deviation of all record values ​​in the length record array; The arithmetic mean is added to the product of a preset proportional coefficient and the standard deviation to obtain the benchmark value; The length threshold is derived by limiting the benchmark value to a pre-configured range of minimum and maximum allowable thresholds.

[0032] A contiguous space is allocated in memory, and a variable-length container consisting of 32-bit unsigned integers, such as `uint32_t lcp_records[]`, is created. This variable-length container serves as the length record array for storing merge sort data. During each pass of merge sort to merge ordered subsequences, when comparing any two adjacent suffix entries, the index is stored based on the original byte content of the semantic block preserved in the symbol record. The suffix data corresponding to both entries is compared byte by byte until a different character is found. The longest common prefix value is calculated, for example, 24 consecutive overlapping bytes, and this record value (24) is inserted into the length record array for storage. The above calculation process is a direct adjacent suffix comparison process, used to form statistical samples, and does not require the ranking array corresponding to the complete suffix array. Before entering the final merging stage, the length record array is traversed, and the arithmetic mean of all record values ​​is calculated, assumed to be 12.5 bytes, and the standard deviation is assumed to be 4.2 bytes. Using a preset scaling factor, for example, set to 1.5, the arithmetic mean is multiplied by 1.5 times the standard deviation, i.e., 12.5 + 1.5 × 4.2 = 18.8, and rounded to obtain a baseline value of 19 bytes. A limiting function is then used to restrict the calculated baseline value to a pre-configured minimum allowable threshold of 8 bytes and a maximum allowable threshold of 64 bytes. Since 19 bytes fall within this range, the output is used as the final length threshold of 19 bytes.

[0033] S3, during the merging stage of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated; based on the suffix array and the strongly correlated suffix clusters, the cluster prefix digest is used to accelerate the search, and a greedy algorithm is used to traverse the new firmware symbol sequence, matching the reusable sequence of the old firmware within the original offset interval corresponding to the old firmware, and generating a differential upgrade package containing copy instructions and add instructions.

[0034] During the final pass of the external memory multi-way merge in the priority queue container, on-demand prefix extension comparisons or direct byte-by-byte precise comparisons are performed on the still undifferentiated long prefix equivalence intervals. After lexicographical sorting within the equivalence intervals, a complete suffix array file is output. Subsequently, a corresponding ranking array is constructed based on the complete suffix array file, and the Kasai linear-time algorithm is called to calculate the precise longest common prefix value between adjacent suffixes in the complete suffix array. If the available memory is insufficient to construct the complete ranking array, the original byte content of the semantic block can be directly accessed between adjacent suffixes in the final merged output, and precise byte comparisons can be performed to obtain the precise longest common prefix value of adjacent suffixes. A comparator logic is set up to compare the precise longest common prefix value of the current adjacent suffixes with the length threshold calculated in the previous steps in real time. If it is determined that the current longest common prefix value is not less than the threshold, a red-black tree mapping table is instantiated in main memory, and multiple original offset data with the same prefix content and consecutive arrangement are clustered and bound into a strongly correlated suffix cluster node and inserted into the red-black tree mapping table. The MurmurHash3 non-cryptographic hash algorithm is called to calculate a 32-bit hash value for the shared long common prefix byte sequence. This hash value and the actual byte length of the current long common prefix are packaged into a composite structure and stored in the node attribute as the cluster prefix digest.

[0035] When entering the differential instruction matching stage, a greedy algorithm architecture is used to sequentially read elements from the new firmware symbol sequence to fill the sliding window. During the global search for the longest matching sequence, the MurmurHash3 algorithm, consistent with the cluster prefix digest, is called. According to the index prefix length used in the cluster prefix digest, where the index prefix length is the length threshold or a preset fixed length not less than the length threshold, a byte sequence of the corresponding length is extracted from the starting position of the current sliding window and a hash value is calculated. This hash value is used as a key to search for a matching candidate digest set in the red-black tree mapping table. If the red-black tree node matches, one or more cluster prefix digest candidates corresponding to the hash key are traversed, and the actual byte length recorded in the digest is further compared with the corresponding original byte content to eliminate hash collisions and jump to the corresponding strongly related suffix cluster for local comparison. If no match is found, the lower bound search binary search algorithm is called to locate the candidate suffix interval corresponding to the current sliding window query prefix in the generated external storage suffix array file, and a precise byte comparison is performed on the suffixes in the candidate interval that meet the old firmware offset condition. The suffix with the largest matching length is selected as the source of the reusable sequence.

[0036] During the retrieval and matching process, a file offset judgment conditional branch statement is added to filter out suffix results whose original offset values ​​are greater than or equal to the total byte length of the old firmware in real time, thus limiting the search range to the source data stream file offset space of the old firmware. When determining the copy byte length of the copy instruction, the source offset address and the copy byte length are both limited to the offset range of the source data stream file corresponding to the old firmware, that is, the copy byte length does not exceed the difference between the total byte length of the old firmware and the source offset address. Since the original offset is defined as the starting offset in the source data stream file, rather than the absolute memory address at microcontroller runtime, it can be compared with the total byte length of the old firmware on the same scale. Once the longest reusable sequence of the old firmware is successfully matched, a copy instruction containing the copy opcode, source offset address, and copy byte length is concatenated and pushed into the instruction buffer queue according to the format specification, and the sliding window pointer is moved forward across the matched length. If no matching segment is found, the original byte stream corresponding to the current symbol sequence is extracted to generate an add instruction containing the new data payload and its payload length, which is then pushed into the instruction buffer queue. After the new firmware symbol sequence traversal is completed, all copy and add instructions in the instruction cache queue are input into the LZMA compression encoding library, such as liblzma or the LZMA encoding interface provided by XZ Utils, to perform lossless compression encoding on the instruction byte stream. The binary data stream output by the compressed encoding is then written as a complete differential upgrade package to a non-volatile storage medium.

[0037] In an optional embodiment, during the merging phase of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated, including: Calculate the exact longest common prefix value of adjacent suffixes in the final merging stage; When the exact longest common prefix value of at least one pair of adjacent suffixes is not less than the length threshold, or when the exact longest common prefix value of adjacent suffixes in a continuous suffix interval is not less than the length threshold, the common prefix byte sequence shared by the corresponding suffixes is extracted. Determine the index prefix length, wherein the index prefix length is not greater than the actual byte length of the common prefix byte sequence; Extract an index prefix byte sequence with a length equal to the index prefix length from the common prefix byte sequence; The index prefix byte sequence is calculated using a preset hash function to obtain the index hash value; Calculate the actual byte length of the common prefix byte sequence; The index hash value, the index prefix length, and the actual byte length are associated and stored to generate a cluster prefix digest of the strongly related suffix cluster.

[0038] During the merging phase of multi-way merging, the longest common prefix value between each pair of adjacent suffixes is calculated. Combining this with the established 19-byte length threshold, when consecutive adjacent suffixes are detected to have longest common prefix values ​​of 22 and 20 respectively, both greater than or equal to the 19-byte length threshold, the minimum of the longest common prefix values ​​within that consecutive interval, 20 bytes, is taken as the actual length of the shared common prefix for that group of consecutive suffixes. A 20-byte common prefix byte sequence is extracted from the source data stream. A preset hash function, preferably the MurmurHash3 algorithm, is called to perform a hash operation on this 20-byte common prefix byte sequence, obtaining a 32-bit hash value for the prefix, such as 0x8A4B9F3C. The actual byte length corresponding to this common prefix byte sequence is calculated, which is 20 bytes, represented in hexadecimal as 0x0014. This byte length is the byte length corresponding to the common prefix byte sequence in the source data stream file. When constructing the summary data structure, the suffixes X, Y, and Z that share the code sequence are aggregated into a strongly related suffix cluster, and the generated 4-byte hash value is concatenated with the 2-byte actual byte length, for example, to form a 6-byte cluster prefix summary 0x8A4B9F3C0014, thus generating the cluster prefix summary of the strongly related suffix cluster.

[0039] In the second embodiment, the present invention also proposes a firmware upgrade management system for a two-wheeled electric vehicle charging pile terminal, comprising the following modules: The conversion module is used to concatenate the old firmware and new firmware of the two-wheeled electric vehicle charging pile into a continuous source data stream according to a preset order; wherein, the preset order is old firmware first and new firmware second, so that the offset range of the source data stream file corresponding to the old firmware is a continuous range from 0 to the total byte length of the old firmware; the source data stream is segmented based on fixed-length instruction codes and preset data boundary symbols, and the source data stream is converted into a symbol sequence composed of multiple variable-length semantic blocks; The construction module is used to construct a suffix array from the symbol sequence in external memory. The construction process of the suffix array includes multiple merge sorts to establish a composite sorting key consisting of semantic block prefixes, type identifiers, and original offsets. In each merge sort, the composite sorting key is used for comparison, and the length of the semantic block prefixes participating in the comparison in the composite sorting key is adjusted according to the current number of passes. The length threshold is calculated based on the statistical distribution of the longest common prefix value of the generated ordered subsequences. The generation module is used during the merging stage of suffix array construction. When the longest common prefix value of adjacent suffixes is not less than the length threshold, it summarizes consecutive suffixes sharing a long common prefix into a strongly correlated suffix cluster and generates a cluster prefix digest containing hash value and byte length. Based on the suffix array and the strongly correlated suffix clusters, it uses the cluster prefix digest to accelerate the search, traverses the new firmware symbol sequence using a greedy algorithm, matches the reusable sequence of the old firmware within the original offset interval corresponding to the old firmware, and generates a differential upgrade package containing copy instructions and add instructions.

[0040] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A firmware upgrade management method for a charging pile terminal for two-wheeled electric vehicles, characterized in that, include: The old firmware and new firmware of the two-wheeled electric vehicle charging pile are concatenated into a continuous source data stream according to a preset order; wherein, the preset order is old firmware first and new firmware second, so that the offset range of the source data stream file corresponding to the old firmware is a continuous range from 0 to the total byte length of the old firmware; the source data stream is segmented based on fixed-length instruction codes and preset data boundary symbols, and the source data stream is converted into a symbol sequence composed of multiple variable-length semantic blocks; A suffix array is constructed from the symbol sequence in external storage. The construction process of the suffix array includes multiple passes of merge sorting to establish a composite sorting key consisting of semantic block prefixes, type identifiers, and original offsets. In each pass of merge sorting, the composite sorting key is used for comparison, and the length of the semantic block prefixes participating in the comparison in the composite sorting key is adjusted according to the current pass number. A length threshold is calculated based on the statistical distribution of the longest common prefix value of the generated ordered subsequences. During the merging phase of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated. Based on the suffix array and the strongly correlated suffix clusters, the cluster prefix digest is used to accelerate the search. A greedy algorithm is used to traverse the new firmware symbol sequence, and the reusable sequence of the old firmware is matched within the original offset interval corresponding to the old firmware to generate a differential upgrade package containing copy instructions and add instructions.

2. The method according to claim 1, characterized in that, The segmentation of the source data stream based on fixed-length instruction codes and pre-defined data boundary symbols, converting the source data stream into a symbol sequence composed of multiple variable-length semantic blocks, includes: The memory mapping boundary parameters of code segments and data segments are extracted from the configuration files of the old firmware and the new firmware, and the actual boundary parameters in the source data stream are calculated by combining them with the file offset increment caused by concatenation. Within the actual boundary range of the code segment, data is read sequentially according to the preset fixed-length instruction code step size, the continuous instructions are divided into code semantic blocks, and the first type identifier is written into the symbol record corresponding to the code semantic block; Within the actual boundary range of the data segment, the data is segmented by scanning a pre-defined data boundary symbol to obtain a data semantic block, and a second type of identifier is written into the symbol record corresponding to the data semantic block. The code semantic blocks and data semantic blocks are arranged in order of their original offsets in the source data stream file to generate the symbol sequence.

3. The method according to claim 2, characterized in that, The process of establishing a composite sorting key consisting of a semantic block prefix, a type identifier, and the original offset includes: Allocate a variable-length structure in memory as storage space for the composite sort key; Using each semantic block in the symbol sequence as a suffix starting point, the first byte sequence is continuously read and extracted sequentially along the subsequent semantic blocks in the symbol sequence, and used as the prefix of the semantic block; Extract the type identifier of the current semantic block; Read the starting offset of the current semantic block in the source data stream file as the original offset; The extracted semantic block prefix, the type identifier, and the original offset are concatenated sequentially to generate a composite sorting key for the corresponding semantic block and stored in the storage space. The original offset serves as a suffix source location field and a stable sorting auxiliary field, and does not replace the extended comparison of subsequent semantic block prefixes when the semantic block prefix and type identifier are not completely distinguished.

4. The method according to claim 1, characterized in that, The step of using the composite sort key for comparison in each merge sort pass, and adjusting the length of the semantic block prefix participating in the comparison in the composite sort key according to the current pass number, includes: Set the maximum number of execution passes for merge sort, and initialize the base value of the semantic block prefix length participating in the comparison to a preset fixed length; After the current round of merging and sorting is completed, the current round number is extracted and multiplied by the preset length growth coefficient to obtain the length update step size; Add the base value to the length update step size to obtain the expected prefix length for the next sorting pass; The expected prefix length is compared with the actual remaining byte length of the corresponding suffix. If it exceeds the actual remaining byte length, the prefix length of the semantic block in the next pass is truncated to the actual remaining byte length. In the comparison phase of the next sorting pass, the semantic block prefix substrings of the specified length in the two composite sort keys are first extracted and compared lexicographically. If the semantic block prefix substrings are the same, the type identifier is compared. For suffix equivalence intervals that are still not distinguished, the semantic block prefix length is extended in subsequent passes. The original offset is only used as an auxiliary field when a precise comparison is completed or when it is necessary to maintain the determinism of the output order.

5. The method according to claim 1, characterized in that, The calculation of the length threshold based on the statistical distribution of the longest common prefix value of the generated ordered subsequences includes: Create a length record array to store the data from the merge sort process; When comparing adjacent suffixes in each merge sorted subsequence, calculate and obtain the longest common prefix value between them, and store the longest common prefix value into the length record array. Before proceeding to the final merging stage, calculate the arithmetic mean and standard deviation of all record values ​​in the length record array; The arithmetic mean is added to the product of a preset proportional coefficient and the standard deviation to obtain the benchmark value; The length threshold is derived by limiting the benchmark value to a pre-configured range of minimum and maximum allowable thresholds.

6. The method according to claim 1 or 5, characterized in that, In the merging stage of suffix array construction, when the longest common prefix value of adjacent suffixes is not less than the length threshold, consecutive suffixes sharing a long common prefix are grouped into strongly correlated suffix clusters, and a cluster prefix digest containing hash value and byte length is generated, including: Calculate the exact longest common prefix value of adjacent suffixes in the final merging stage; When the exact longest common prefix value of at least one pair of adjacent suffixes is not less than the length threshold, or when the exact longest common prefix value of adjacent suffixes in a continuous suffix interval is not less than the length threshold, the common prefix byte sequence shared by the corresponding suffixes is extracted. Determine the index prefix length, wherein the index prefix length is not greater than the actual byte length of the common prefix byte sequence; Extract an index prefix byte sequence with a length equal to the index prefix length from the common prefix byte sequence; The index prefix byte sequence is calculated using a preset hash function to obtain the index hash value; Calculate the actual byte length of the common prefix byte sequence; The index hash value, the index prefix length, and the actual byte length are associated and stored to generate a cluster prefix digest of the strongly related suffix cluster.

7. A firmware upgrade management system for charging pile terminals of two-wheeled electric vehicles, characterized in that, include: The conversion module is used to concatenate the old firmware and new firmware of the two-wheeled electric vehicle charging pile into a continuous source data stream according to a preset order; wherein, the preset order is old firmware first and new firmware second, so that the offset range of the source data stream file corresponding to the old firmware is a continuous range from 0 to the total byte length of the old firmware; the source data stream is segmented based on fixed-length instruction codes and preset data boundary symbols, and the source data stream is converted into a symbol sequence composed of multiple variable-length semantic blocks; The construction module is used to construct a suffix array from the symbol sequence in external memory. The construction process of the suffix array includes multiple merge sorts to establish a composite sorting key consisting of semantic block prefixes, type identifiers, and original offsets. In each merge sort, the composite sorting key is used for comparison, and the length of the semantic block prefixes participating in the comparison in the composite sorting key is adjusted according to the current number of passes. The length threshold is calculated based on the statistical distribution of the longest common prefix value of the generated ordered subsequences. The generation module is used during the merging stage of suffix array construction. When the longest common prefix value of adjacent suffixes is not less than the length threshold, it summarizes consecutive suffixes sharing a long common prefix into a strongly correlated suffix cluster and generates a cluster prefix digest containing hash value and byte length. Based on the suffix array and the strongly correlated suffix clusters, it uses the cluster prefix digest to accelerate the search, traverses the new firmware symbol sequence using a greedy algorithm, matches the reusable sequence of the old firmware within the original offset interval corresponding to the old firmware, and generates a differential upgrade package containing copy instructions and add instructions.

8. The system according to claim 7, characterized in that, The segmentation of the source data stream based on fixed-length instruction codes and pre-defined data boundary symbols, converting the source data stream into a symbol sequence composed of multiple variable-length semantic blocks, includes: The memory mapping boundary parameters of code segments and data segments are extracted from the configuration files of the old firmware and the new firmware, and the actual boundary parameters in the source data stream are calculated by combining them with the file offset increment caused by concatenation. Within the actual boundary range of the code segment, data is read sequentially according to the preset fixed-length instruction code step size, the continuous instructions are divided into code semantic blocks, and the first type identifier is written into the symbol record corresponding to the code semantic block; Within the actual boundary range of the data segment, the data is segmented by scanning a pre-defined data boundary symbol to obtain a data semantic block, and a second type of identifier is written into the symbol record corresponding to the data semantic block. The code semantic blocks and data semantic blocks are arranged in order of their original offsets in the source data stream file to generate the symbol sequence.

9. The system according to claim 7, characterized in that, The process of establishing a composite sorting key consisting of a semantic block prefix, a type identifier, and the original offset includes: Allocate a variable-length structure in memory as storage space for the composite sort key; Using each semantic block in the symbol sequence as a suffix starting point, the first byte sequence is continuously read and extracted sequentially along the subsequent semantic blocks in the symbol sequence, and used as the prefix of the semantic block; Extract the type identifier of the current semantic block; Read the starting offset of the current semantic block in the source data stream file as the original offset; The extracted semantic block prefix, the type identifier, and the original offset are concatenated sequentially to generate a composite sorting key for the corresponding semantic block and stored in the storage space. The original offset serves as a suffix source location field and a stable sorting auxiliary field, and does not replace the extended comparison of subsequent semantic block prefixes when the semantic block prefix and type identifier are not completely distinguished.

10. The system according to claim 7, characterized in that, The step of using the composite sort key for comparison in each merge sort pass, and adjusting the length of the semantic block prefix participating in the comparison in the composite sort key according to the current pass number, includes: Set the maximum number of execution passes for merge sort, and initialize the base value of the semantic block prefix length participating in the comparison to a preset fixed length; After the current round of merging and sorting is completed, the current round number is extracted and multiplied by the preset length growth coefficient to obtain the length update step size; Add the base value to the length update step size to obtain the expected prefix length for the next sorting pass; The expected prefix length is compared with the actual remaining byte length of the corresponding suffix. If it exceeds the actual remaining byte length, the prefix length of the semantic block in the next pass is truncated to the actual remaining byte length. In the comparison phase of the next sorting pass, the semantic block prefix substrings of the specified length in the two composite sort keys are first extracted and compared lexicographically. If the semantic block prefix substrings are the same, the type identifier is compared. For suffix equivalence intervals that are still not distinguished, the semantic block prefix length is extended in subsequent passes. The original offset is only used as an auxiliary field when a precise comparison is completed or when it is necessary to maintain the determinism of the output order.

Images