A data file splicing method

Abstract

The present invention discloses a data file splicing method, which includes the following steps: step 1, reading APP data and Bootloader data, and loading and processing them; step 2, processing the APP data and Bootloader data, and placing the APP data and Bootloader data on the specified addresses respectively according to the memory address of the chip, and the processing method includes: using a concurrent traversal algorithm to traverse all documents, address identification, merging of similar data, and address data deduplication; step 3, merging the processed data and outputting them into files; step 4, loading the generated files, calling the burning program to burn the program into the MCU at one time. The present invention can reduce the burning time, can avoid the error that the single-chip microcomputer cannot start normally due to the change of the burning order of the data file; solve the situation that the current merging program can only process two files, can process multiple files at the same time, and the support for large-scale code engineering is more perfect.

Description

Technical Field

[0001] The present invention relates to the technical field of file splicing, and in particular to a data file splicing method. Background Art

[0002] The embedded industry is booming, and single-chip microcomputers are used in more and more scenarios. The relevant chips will be flashed with preset programs when they leave the factory. For some embedded chips, their data files are divided into two parts: APP and Bootloader. BootLoader is the boot area, and APP is the main part of the program. The format is S19 format file. The composition format of S19 format file is as follows:

[0003] Record Type Record length Storage address Code / Data Checksum

[0004] (1) The current file splicing technology mainly uses a manual splicing method, which is to open each file in turn, manually copy the content of each file, add the relevant content to one file, and manually delete the file header and file footer.

[0005] When splicing files, as the number of files increases, this method will spend a lot of time on copying files. At the same time, the data files are processed manually. As the data increases, it becomes increasingly difficult to find the data at a specific location and process it. At the same time, copying and pasting files during splicing also becomes more difficult, which is very easy to cause errors and burn the wrong program, resulting in the problem that the microcontroller cannot start after the program is burned into the microcontroller.

[0006] (2) Current file splicing technology mainly focuses on the synthesis of PDF files, splicing two PDF files into one file through a specific algorithm.

[0007] PDF can be spliced, but it simply merges two files into one file. It cannot process files by itself, such as deleting the first and last lines of a file and then splicing it. It is only used for splicing document files. For files containing data, its algorithm cannot effectively process files containing large amounts of data.

[0008] (3) Patent CN112328253A describes a method for concatenating multiple HEX files into one file. This method mainly converts the XML file into a HEX file, then concatenates the two HEX files into one file, and performs format conversion to form the BIN required for burning.

[0009] 1. The technical solution used in this method is complex, involving multiple file conversions and then splicing, and the labor cost of program algorithm compilation is high.

[0010] 2. With the development of embedded technology, there are gradually multiple HEX files in the project that need to be merged. This method can only merge two HEX files, and there is a lack of effective methods for processing multiple files.

[0011] 3. This method only implements the filling of partial data, but does not implement the data deletion process, and lacks the processing of duplicate data. Summary of the invention

[0012] The technical problem to be solved by the present invention is to provide a data file splicing method in view of the defects in the prior art.

[0013] The technical solution adopted by the present invention to solve its technical problem is:

[0014] The present invention provides a data file splicing method, which comprises the following steps:

[0015] Step 1: Read APP data and Bootloader data, load and process them;

[0016] Step 2: Process the APP data and the Bootloader data, and put the APP data and the Bootloader data into the designated addresses respectively according to the memory address of the chip. The processing method includes: using a concurrent traversal algorithm to traverse all documents, address recognition, merging similar data, and deduplication of address data;

[0017] Step 3: Merge the processed data and output them into a file;

[0018] Step 4: Load the generated file and call the burning program to burn the program into the MCU at one time.

[0019] Furthermore, the method for loading data in step 1 of the present invention is:

[0020] The source files of APP data and Bootloader data are S19 format files. S19 format files are text files with ASCII encoding format. Python's internal function open is used for direct reading and calling, and the corresponding data stream is obtained after reading.

[0021] Furthermore, the method for performing data processing in step 1 of the present invention is:

[0022] All data streams are loaded into the set lists respectively. The number of lists is determined by reading the number of S19 files under the project file. The length of the list changes with the length of the file data. By setting the number of data files, multiple file data streams can be read.

[0023] Further, the method for traversing all documents in step 2 of the present invention is as follows:

[0024] The concurrent algorithm for accelerating the traversal algorithm is as follows: Monitor the CPU load through the OS module of Python. When the load is lower than 90%, for each file found, perform a 2^n operation and generate a new file reading thread; in this way, the time complexity of traversing the file is O(log(n)), compared with the time complexity of reading files one by one O(n), the speed is greatly improved.

[0025] Further, the method for address recognition in step 2 of the present invention is as follows:

[0026] The third segment of each line of data is the address end of the data, and the storage address is 2, 3, or 4 bytes, determined according to the record type; thus, in order to parse the storage address well, obtain the storage address of the file by reading the record type; the algorithm is written as follows: After completing the document traversal, read the first byte of the first line of each document, and identify the S1, S2, and S3 fields therein. S1 is 2 bytes, S2 is 3 bytes, and S3 is 4 bytes, and they are divided into three categories according to S1, S2, and S3.

[0027] Further, the method for merging similar data in step 2 of the present invention is as follows:

[0028] After classification, parse the starting address field of the first line, and form the order of S1 < S2 < S3 according to the total sorting of S1, S2, and S3. Each S segment is sorted separately in ascending order of the data in the storage address segment.

[0029] Further, the method for removing duplicates of address data in step 2 of the present invention is as follows:

[0030] When the data of each file is sorted and classified, when each file is independent, each file is in the form of the first address of the first line + the last address of the last line; and when combined into one file, it is necessary to ensure that there is only one first address + last address for the whole file; to achieve this operation, denote the data of each file as a1, a2, a3...a n ; a n ,a n +1...a m ; a m ,a m +1,a m +2...a finally ;

[0031] It can be seen that for the sorted and classified files, the address of the last line of each file is repeated with the first line of the adjacent file. Therefore, adopt the sequential traversal algorithm, read each file in turn, and delete the last line of the file to form a new file.

[0032] Further, the method in step 3 of the present invention is:

[0033] After processing all the data, call the library function write to write them into a new file in sequence, generate a new file and output it.

[0034] The beneficial effect of the present invention is that the data file splicing method of the present invention merges data files into a single file, which will greatly reduce the workload when burning a single-chip microcomputer. In the past, factories used a method of burning multiple files one by one when burning a single-chip microcomputer, which was time-consuming and labor-intensive, and had a high error rate.

[0035] (1) After software synthesis, only one burning is required. Using the one-time burning method will shorten the burning time by more than 50%.

[0036] (2) At the same time, since the burning order of a single file does not need to be considered, the error of the microcontroller not being able to start normally due to the change of the burning order of the data files will be completely avoided.

[0037] (3) The current merging program can only process two files, and can process multiple files at the same time, providing better support for large code projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0039] Figure 1 is a method flow chart of an embodiment of the present invention;

[0040] Figure 2 is a data placement method of an embodiment of the present invention;

[0041] Figure 3 Schematic diagram of thread generation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] Definitions of Abbreviations and Key Terms in the Embodiments of the Present Invention

[0044] S19 file: A file launched by Motorola that encodes program and data files in a printable format. The file contains all the content that is ultimately downloaded to the microcontroller.

[0045] APP: The main part of the microcontroller data.

[0046] Bootloader: The boot part of the microcontroller data.

[0047] like Figure 1 As shown, the method of the embodiment of the present invention is written in Python language, and the main steps include:

[0048] 1. Read APP data and Bootloader data;

[0049] 1.1. Data stream loading: The source files of APP and Bootloader are S19 format files, and S19 files are text files with ASCII encoding format. Therefore, Python's internal function open is used for direct reading and calling. After reading, the corresponding data streams A, B, C... are obtained.

[0050] 1.2 Data stream processing: All data streams are loaded into pre-set lists. The number of lists is determined by reading the number of S19 files under the project file. Since the length of the list can change with the length of the file data, there is no need to worry about the array crossing the boundary due to the large file data. By setting the number of data files, multiple file data streams can be read.

[0051] 2. Process the data of APP and Bootloader;

[0052] This step is the core step of the file synthesis tool. After obtaining the APP and Bootloader data in the first step, the data needs to be processed. The main content of the processing is to put the APP data and Bootloader data into the specified address according to the memory address of the chip. The specific data placement method is as follows: Figure 2 shown.

[0053] 2.1 Traverse all documents;

[0054] In order to avoid the slow traversal speed of the software when there are many files, a concurrent algorithm is specially developed to speed up the traversal algorithm. The core method is to monitor the CPU load through the OS module of Python. When the load is less than 90%, a 2^n operation is performed every time a file is searched to generate a new file reading thread. The thread generation diagram is shown in the figure below. Figure 3 shown.

[0055] In this way, the time complexity of traversing files is O(log(n)), which is much faster than the time complexity of reading files one by one (O(n)).

[0056] 2.2 Address identification;

[0057] The third segment of each line of data is the address end of the data, and the storage address is 2, 3, or 4 bytes, determined according to the record type. Therefore, to parse the storage address correctly, the record type needs to be read to obtain the storage address of the file. The algorithm is written as follows: After completing the document traversal, read the first byte of the first line of each document, identify the S1 (2 bytes), S2 (3 bytes), and S3 (4 bytes) fields, and classify them into three categories according to S1, S2, and S3.

[0058] 2.3 Merging of similar data;

[0059] After classification, it is necessary to parse the start address field of the first line, and form the order of S1 < S2 < S3 according to the total sorting of S1, S2, and S3. Each S segment is sorted separately in ascending order of the data in the storage address segment.

[0060] 2.4 Duplicate removal of address data;

[0061] When the data of each file is sorted and classified, it enters the last step of data processing. When each file is independent, each file is in the form of the start address of the first line + the end address of the last line. To combine into one file, the whole file needs to have only one start address + end address. To achieve this operation, we denote the data of each file as a1, a2, a3...a n ; a n , a n+1 ...a m ; a m , a m+1 , a m+2 ...a finally ;

[0062] It can be seen that for the sorted and classified files, the address of the last line of each file is the same as the address of the first line of the adjacent file. Therefore, using the sequential traversal algorithm, read each file in turn, delete the last line of the file, and form a new file.

[0063] 3. Merge the processed data and output it as a file;

[0064] After processing all the data, call the library function write to write it sequentially into a new file, generate a new file and output it.

[0065] 4. File writing;

[0066] Load the generated file, and call the programming program to program the program into the MCU at one time.

[0067] The present invention is mainly used to solve the problem of merging multiple HEX files. The S19 format file is formatted by a software algorithm. The specific processing contents are: 1. The Bootloader file is divided into two parts according to the specified address 2. The address start line (first line) and address end line (last line) of all APP files are deleted, and finally the processed Bootloader file is unified and merged with the APP file to finally form a complete data file that can be burned into the single-chip microcomputer 3. One-key burning of files is realized.

[0068] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A data file splicing method, characterized in that: The method includes the following steps: Step 1, read APP data and Bootloader data, and load and process them; Step 2, process the APP data and Bootloader data, and place the APP data and Bootloader data at the specified addresses according to the memory addresses of the chips. The processing methods include: traversing all documents using a concurrent traversal algorithm, address recognition, merging of similar data, and deduplication of address data; The method of address recognition is: the third segment of each line of data is the address end of the data, and the storage address is 2, 3, or 4 bytes, determined according to the record type; thus, in order to parse the storage address well, obtain the storage address of the file by reading the record type; the algorithm is written as follows: after completing the traversal of the document, read the first byte of the first line of each document, and identify the S1, S2, and S3 fields therein. S1 is 2 bytes, S2 is 3 bytes, and S3 is 4 bytes, and they are divided into three categories according to S1, S2, and S3; The method of merging similar data is: after classification, parse the starting address field of the first line, and form the order of S1 < S2 < S3 according to the total sorting of S1, S2, and S3. Each S segment is sorted separately in ascending order of the data in the storage address segment; The method of deduplicating address data is as follows: after the data of each file is classified and sorted, when each file is independent, each file is in the form of the first address of the first line + the last address of the last line; and to synthesize a file, it is necessary to keep only one first address + last address for the entire file; record the data of each file as a1, a2, a3...a n ; a n , a n+1 ...a m ; a m ,a m+1 ,a m+2 ...a finally ; For the sorted files after classification, the address of the last line of each file is repeated with the first line of the adjacent file. Adopt a sequential traversal algorithm, read each file in turn, and delete the last line of the file to form a new file; Step 3, merge the processed data and output it as a file; Step 4, load the generated file, and call the burning program to burn the program into the MCU at one time.

2. The data file splicing method according to claim 1, characterized in that: The method of data loading in the said Step 1 is: The source files of the APP data and Bootloader data are S19 format files. The S19 format file is a text file with the encoding format of ASCII. Use the internal function open of Python to directly read and call, and obtain the corresponding data stream after reading.

3. The data file splicing method according to claim 2, characterized in that: The method of data processing in the said Step 1 is: Load all data streams into the pre-set lists respectively. The number of lists is determined by reading the number of S19 files in the project file. The length of the list changes with the length of the file data. By setting the number of data files, read the data streams of multiple files.

4. The data file splicing method according to claim 1, characterized in that: The method of traversing all documents in the said Step 2 is: The concurrent algorithm to accelerate the traversal algorithm is: monitor the CPU load through the OS module of Python. When the load is lower than 90%, perform a 2^n operation every time a file is searched, where n represents the number of files, and generate a new file reading thread; the time complexity of traversing the file is O(log(n)), compared with the time complexity of reading files one by one O(n), the speed is greatly improved.

5. The data file splicing method according to claim 1, characterized in that: The method in the said Step 3 is: After processing all the data, call the library function write to write them into a new file in sequence, generate a new file and output it.

Images