Intelligent data burning method and system based on data stream

Abstract

The present application relates to the technical field of electric data processing, and especially relates to an intelligent data burning method and system based on data flow. The method comprises the following steps: performing format identification and semantic analysis on real-time collected input source data to be burned to generate metadata information corresponding to the data to be burned; obtaining hardware storage structure information and burning protocol specification corresponding to a target device and performing data flow description output to generate a corresponding data flow description file; obtaining storage bandwidth and cache capacity corresponding to the target device and performing burning dynamic task scheduling and low-density redundancy coding block to generate fault-tolerant coding data flow blocks; performing real-time monitoring on device state data corresponding to the target device and performing burning writing processing and block-by-block CRC check, and simultaneously performing online repair verification analysis to generate a data flow burning verification completion result. The present application can design a repair mechanism and dynamic resource scheduling to realize efficient and reliable data burning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric data processing, and in particular to an intelligent data burning method and system based on data flow. BACKGROUND

[0002] In the manufacturing process of modern electronic products, data burning refers to an important process of writing solidification programs or configuration information into hardware devices (such as chips, memories, etc.). With the continuous development of technology, especially in the fields of Internet of Things, intelligent hardware and embedded systems, the requirements for the precision, efficiency and intelligence of data burning are also increasing, for example, for different versions of hardware devices, the selection of burning programs and the processing process of data need to be optimized in real time to ensure the efficiency and accuracy of the burning process. However, the current burning technology mostly adopts a single burning process, which is usually completed by manual or semi-automatic equipment according to the preset program to write data step by step. This method has certain disadvantages, which makes it necessary to constantly adjust the burning parameters according to different hardware when facing different device types and hardware configurations, and cannot intelligently optimize according to the real-time state data of the device, resulting in a high error rate in the burning process and reducing the burning efficiency. SUMMARY

[0003] Therefore, it is necessary to provide an intelligent data burning method and system based on data flow to solve at least one of the above technical problems.

[0004] To achieve the above purpose, an intelligent data burning method based on data flow includes the following steps:

[0005] Step S1: Real-time collection of input source data to be burned through a bus interface, and format recognition and semantic analysis of the input source data to be burned to generate metadata information corresponding to the data to be burned; obtaining hardware storage structure information and burning protocol specification corresponding to the target device, and outputting data flow description of the metadata information corresponding to the data to be burned based on the hardware storage structure information and the burning protocol specification corresponding to the target device, to generate a data flow description file containing metadata, structure information and burning instruction corresponding to the data to be burned;

[0006] Step S2: Obtaining the storage bandwidth and cache capacity corresponding to the target device, and performing burning dynamic task scheduling on the data flow description file containing metadata, structure information and burning instruction corresponding to the data to be burned based on the storage bandwidth and cache capacity corresponding to the target device, to generate a data flow burning task sequence;

[0007] Step S3: Low-density redundancy coding block is performed on the corresponding to-be-burned data stream in the data stream burning task sequence to generate fault-tolerant coding data stream blocks; device state data including storage temperature, voltage fluctuation and erasing times of the target device are monitored in real time, and the fault-tolerant coding data stream blocks are burned and written based on the device state data to generate device burning data stream blocks;

[0008] Step S4: CRC check is performed on the device burning data stream blocks, and the error blocks corresponding to the device burning data stream blocks are located, and the error blocks are repaired and verified online based on the corresponding redundancy information in the fault-tolerant coding data stream blocks and the device state data to generate a data stream burning verification completion result.

[0009] Further, step S1 includes the following steps:

[0010] Step S11: The bus interface is configured to include SPI, I2C and UART interface protocols to collect corresponding to-be-burned input source data in real time;

[0011] Step S12: The to-be-burned input source data is format-identified and parsed to design a multi-mode data parser to support automatic identification and parsing of various data formats including binary, text, XML and JSON, to obtain to-be-burned input format parsed data;

[0012] Step S13: The to-be-burned input format parsed data is subjected to data field semantic parsing to generate metadata information corresponding to the to-be-burned data;

[0013] Step S14: The hardware storage structure information and the burning protocol specification corresponding to the target device are obtained, wherein the hardware storage structure information includes flash block distribution, page address mapping table and bad block marking information;

[0014] Step S15: Based on the hardware storage structure information and the burning protocol specification corresponding to the target device, data stream description output is performed on the metadata information corresponding to the to-be-burned data to generate a data stream description file including metadata, structure information and burning instruction.

[0015] Further, step S15 includes the following steps:

[0016] Step S151: The corresponding to-be-burned data semantic entities are obtained through the metadata information corresponding to the to-be-burned data, and entity relationship extraction is performed according to the to-be-burned data semantic entities to obtain data entity relationships between the to-be-burned data entities;

[0017] Step S152: entity relationship graph construction is performed on each to-be-burned data semantic entity based on the data entity relationship between each to-be-burned data entity, to generate a to-be-burned data entity relationship graph;

[0018] Step S153: executable burn instruction generation is performed on the to-be-burned data entity relationship graph based on the burn protocol specification corresponding to the target device, to generate target device executable burn instructions corresponding to the to-be-burned data;

[0019] Step S154: data flow description output is performed on the metadata information corresponding to the to-be-burned data based on the hardware storage structure information corresponding to the target device and in combination with the target device executable burn instructions corresponding to the to-be-burned data, to generate a data flow description file containing the metadata, structure information and burn instructions.

[0020] Further, step S2 includes the following steps:

[0021] Step S21: the storage bandwidth corresponding to the target device is acquired through continuous writing test;

[0022] Step S22: the cache capacity corresponding to the target device is calculated and acquired according to the page address mapping table corresponding to the target device;

[0023] Step S23: burn dependency relationship analysis is performed on the data flow description file containing the metadata, structure information and burn instructions, to mine and analyze the dependency relationship corresponding to each to-be-burned data flow in the data flow description file based on the burn instructions, and generate data burn dependency relationships between each to-be-burned data flow;

[0024] Step S24: directed acyclic graph construction is performed on each to-be-burned data flow in the data flow description file based on the data burn dependency relationship between each to-be-burned data flow, to generate a to-be-burned data flow directed acyclic graph; and topological burn sorting is performed on each to-be-burned data flow in the data flow description file according to the to-be-burned data flow directed acyclic graph, to generate a data flow initial burn sorting sequence;

[0025] Step S25: burn dynamic task scheduling is performed on the data flow initial burn sorting sequence based on the storage bandwidth and the cache capacity corresponding to the target device, to generate a data flow burn task sequence.

[0026] Further, step S25 includes the following steps:

[0027] Step S251: dividing the data stream initial burn-in ordering sequence into data segments, obtaining the corresponding data size of each data segment, and constructing a data stream burn-in transmission topology network based on the data transmission order and dependency relationship of each data segment in the data stream initial burn-in ordering sequence and in combination with the corresponding data size of each data segment, wherein the data segment is taken as a node, the data size is taken as a node attribute, and the data transmission order and dependency relationship are taken as an edge;

[0028] Step S252: calculating the betweenness centrality of each node in the data stream burn-in transmission topology network, and determining the burn-in transmission priority of each data segment based on the betweenness centrality of each node to generate a data stream burn-in transmission priority sequence;

[0029] Step S253: evaluating the bandwidth burn-in transmission pressure of the corresponding data segment in the data stream burn-in transmission priority sequence based on the storage bandwidth of the target device to obtain the corresponding storage bandwidth burn-in transmission pressure of each data segment in the target device;

[0030] Step S254: quantifying the cache transmission saturation of the corresponding data segment in the data stream burn-in transmission priority sequence based on the cache capacity of the target device to obtain the corresponding cache burn-in transmission saturation rate of each data segment in the target device;

[0031] Step S255: analyzing the current load of the target device according to the storage bandwidth burn-in transmission pressure and the cache burn-in transmission saturation rate of each data segment in the target device to obtain the current burn-in load of the target device; dynamically adjusting the burn-in transmission rate of each data segment according to the current burn-in load of the target device, and performing burn-in dynamic task scheduling on the data stream initial burn-in ordering sequence based on the burn-in transmission rate of each data segment to generate a data stream burn-in task sequence.

[0032] Further, step S3 includes the following steps:

[0033] Step S31: performing low-density redundancy coding block on the corresponding data stream to be burned in the data stream burn-in task sequence to generate a fault-tolerant coding data stream block;

[0034] Step S32: real-time monitoring of the storage temperature of the target device;

[0035] Step S33: real-time monitoring of the voltage fluctuation of the target device;

[0036] Step S34: real-time monitoring of the number of erasing and writing times of the target device;

[0037] Step S35: Burn write processing is performed on the fault-tolerant encoding data stream block based on the device state data, by taking the storage temperature, voltage fluctuation and erase count corresponding to the target device as the device state data, to generate a device burn data stream block.

[0038] Further, step S31 includes the following steps:

[0039] The corresponding to-be-burned data stream in the data stream burn task sequence is processed in blocks to obtain each to-be-burned data stream block;

[0040] The to-be-burned data content corresponding to each block is obtained through each to-be-burned data stream block, and each to-be-burned data stream block is grouped based on the to-be-burned data content corresponding to each block to obtain key to-be-burned data blocks and non-key to-be-burned data blocks;

[0041] The content correlation coefficient and the erase count of the corresponding data block are obtained through the key to-be-burned data block and the non-key to-be-burned data block, and the optimal number of check bits of the corresponding data block is calculated based on the content correlation coefficient and the erase count, and the key to-be-burned data block is encoded with high redundancy and the non-key to-be-burned data block is encoded with low redundancy according to the optimal number of check bits of the corresponding data block, to generate key to-be-burned encoding blocks and non-key to-be-burned encoding blocks;

[0042] An index tag containing position information and redundancy information corresponding to each key to-be-burned encoding block and non-key to-be-burned encoding block is added to generate a fault-tolerant encoding data stream block.

[0043] Further, the burn write processing of the fault-tolerant encoding data stream block based on the device state data in step S35 includes the following steps:

[0044] The device historical burn state data is obtained, and a Bayesian network model is trained based on the device historical burn state data to predict the write error probability of the target device corresponding to the data stream block;

[0045] The write prediction analysis of the fault-tolerant encoding data stream block is performed by inputting the current corresponding device state data into the Bayesian network model to obtain the write error probability of each fault-tolerant encoding data stream block corresponding to the target device;

[0046] The write error probability of each fault-tolerant encoding data stream block is determined, and if the write error probability is high, the burn-in voltage is increased by less than or equal to 5% of the standard burn-in voltage, and the clock frequency is reduced by less than or equal to 10% of the standard clock frequency; if the write error probability is low, the burn-in voltage is increased by less than 5% of the high error probability block, and the clock frequency is increased by less than 10% of the high error probability block, thereby performing the write operation of each fault-tolerant encoding data stream block to generate the device burn-in data stream block.

[0047] Further, step S4 includes the following steps:

[0048] Step S41: Perform block-by-block CRC verification on the device burn-in data stream block to obtain device burn-in verification success blocks and device burn-in verification failure blocks;

[0049] Step S42: Obtain the corresponding block address and error type of the device burn-in verification failure block, and locate the error block corresponding to the device burn-in verification failure block based on the block address and error type;

[0050] Step S43: Obtain the index label of the block address in the fault-tolerant encoding data stream block based on the block address, and obtain the corresponding redundant information in the fault-tolerant encoding data stream block based on the index label, and use the corresponding redundant information in the fault-tolerant encoding data stream block to repair the error block online, using an iterative decoding algorithm to repair the error block using redundant information, if the repair fails, triggering a re-burn-in process, otherwise, iterative CRC verification until verification is successful;

[0051] Step S44: Perform burn-in verification analysis on the device burn-in verification success blocks corresponding to the CRC verification success based on the device state data, to calculate the bit error rate and the remaining erase life of each burn-in block corresponding to the device state data, and verify the bit error rate and the remaining erase life of each burn-in block corresponding to the preset threshold, if the verification fails, re-repair and verification to generate a data stream burn-in verification completion result.

[0052] Further, the present application also provides an intelligent data burn-in system based on data stream, which is used to execute the intelligent data burn-in method based on data stream as described above, and the intelligent data burn-in system based on data stream comprises:

[0053] The to-be-burned data stream description output module is configured to collect to-be-burned input source data in real time through a bus interface, and perform format recognition and semantic analysis on the to-be-burned input source data to generate metadata information corresponding to the to-be-burned data; obtain hardware storage structure information and a burn protocol specification corresponding to a target device, and perform data stream description output on the metadata information corresponding to the to-be-burned data based on the hardware storage structure information and the burn protocol specification corresponding to the target device, thereby generating a data stream description file corresponding to the metadata, the structure information and the burn instruction;

[0054] The burn dynamic scheduling module is configured to obtain storage bandwidth and cache capacity corresponding to the target device, and perform burn dynamic task scheduling on the data stream description file corresponding to the metadata, the structure information and the burn instruction based on the storage bandwidth and the cache capacity corresponding to the target device, thereby generating a data stream burn task sequence.

[0055] The data stream block burn write module is configured to perform low-density redundancy coding block on the to-be-burned data stream corresponding to the data stream burn task sequence to generate fault-tolerant coding data stream blocks; monitor device state data corresponding to the target device in real time, the device state data including storage temperature, voltage fluctuation and erasing times, and perform burn write processing on the fault-tolerant coding data stream blocks based on the device state data to generate device burn data stream blocks.

[0056] The burn error online repair module is configured to perform block-by-block CRC check on the device burn data stream blocks, locate error blocks corresponding to the device burn data stream blocks, and perform online repair verification analysis on the error blocks by using corresponding redundancy information in the fault-tolerant coding data stream blocks and combining the device state data, to generate a data stream burn verification completion result.

[0057] The present application has the following advantages:

[0058] 1、The intelligent data burning method based on data stream provided in the application has the beneficial effects compared with the prior art that the bus interface is used to collect the input source data to be burned in real time, which is crucial for ensuring the accuracy and timeliness of the data, the original data can be converted into metadata with structured information through format identification and semantic analysis of the input source data, which not only improves the efficiency of data processing, but also provides a solid foundation for subsequent operations, the generation of metadata involves comprehensive analysis and accurate identification of data, so that it can correspond to the hardware structure and burning specification of the target device, and obtaining the hardware storage structure information and burning protocol specification of the target device is the core part of this step, which can effectively guide the output of the subsequent data stream description, and ensure that the burning process meets the actual requirements of the device, the generated data stream description file contains metadata, hardware structure information and burning instructions, which is crucial for subsequent data transmission and burning process, the biggest advantage of this step is its high automation and flexibility, which can provide personalized burning scheme for different devices, and can significantly improve the accuracy and stability of the burning process, secondly, the storage bandwidth and cache capacity of the target device are obtained, and the resource is reasonably scheduled to determine the burning task sequence, the storage bandwidth and cache capacity directly affect the speed and efficiency of data burning, therefore, when scheduling the data stream description file, these hardware parameters need to be optimized in detail, the core goal of dynamic task scheduling is to reasonably allocate the burning process of the data stream, ensure that each burning task can be completed within the specified time, and avoid performance degradation due to insufficient bandwidth or cache overflow, through accurate scheduling of the task sequence, the burning efficiency can be improved on the premise of ensuring the burning quality, this process makes the data stream burning not a single, linear task, but a dynamic adjustment and flexible process, which can adapt to the different performance characteristics of the target device, so that the whole burning process is more efficient and accurate, then, the data stream to be burned in the data stream burning task sequence needs to be processed by low-density redundancy coding block, the main purpose of this process is to improve the fault tolerance of data in the burning process, and the redundancy coding block can provide repair basis for subsequent possible device failures by increasing additional data redundancy information, in the burning process, the state of the target device, such as storage temperature, voltage fluctuation and erase times, will affect the storage and burning process of the data, real-time monitoring of these device state data helps to dynamically adjust the burning strategy, so that the data can be stored under the best conditions, the burning write processing based on the device state data can prevent the burning failure problem caused by high temperature, unstable voltage or too many erase times, and intelligent optimization is further carried out according to the real-time state data of the device, the advantage of this stage is that it fully considers various actual situations in the device operation, so as to reduce the error rate of the data stream in the burning process.Finally, by performing CRC check on each block of the data stream burned into the device, it is ensured that no damage or error occurs to each block of data during the burning process, and the error in the data transmission process can be quickly identified and located. When an error block is detected, the redundant information in the fault-tolerant coded data stream block can be used to repair the error online. Combined with the device state data, this process further improves the accuracy and efficiency of error repair. At the same time, the repaired data stream is verified and analyzed to ensure that the final burning result meets the expected standard. This method greatly improves the reliability of the burning process, thereby ensuring that the device can stably store and read data during long-term operation.

[0059] 2、The intelligent data burning system based on data stream proposed in the present application is composed of a data stream description output module, a burning dynamic scheduling module, a data stream block burning and writing module, and an online burning error repair module, and can realize the intelligent data burning method based on data stream described in the present application. The system is used to realize the intelligent data burning method based on data stream by combining the operations between the computer programs running on each module. The internal structure of the system cooperates with each other, which can greatly reduce the repetitive work and labor input, and can quickly and effectively provide more accurate and efficient intelligent data burning process based on data stream, thereby simplifying the operation process of the intelligent data burning system based on data stream. BRIEF DESCRIPTION OF DRAWINGS

[0060] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments made with reference to the accompanying drawings:

[0061] Figure 1 The intelligent data burning method based on data stream of the present application is shown in the following steps:

[0062] Figure 2 The detailed step flowchart of step S1 in the intelligent data burning method based on data stream of the present application is shown in the following steps: Figure 1

[0063] The detailed step flowchart of step S15 in the intelligent data burning method based on data stream of the present application is shown in the following steps. Figure 3 DETAILED DESCRIPTION Figure 2 The technical method of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0064] To achieve the above-mentioned purpose, please refer to

[0065] To achieve the above-mentioned purpose, please refer to Figures 1 to 3 ​The application provides an intelligent data burning method based on a data stream, and the method comprises the following steps:

[0066] Step S1: collecting input source data to be burned in real time through a bus interface, performing format recognition and semantic analysis on the input source data to be burned, to generate metadata information corresponding to the data to be burned, obtaining hardware storage structure information and burning protocol specification corresponding to a target device, and performing data stream description output on the metadata information corresponding to the data to be burned based on the hardware storage structure information and the burning protocol specification corresponding to the target device, to generate a data stream description file corresponding to metadata, structure information and burning instructions;

[0067] Step S2: obtaining storage bandwidth and cache capacity corresponding to the target device, and performing burning dynamic task scheduling on the data stream description file corresponding to the metadata, the structure information and the burning instructions based on the storage bandwidth and the cache capacity corresponding to the target device, to generate a data stream burning task sequence;

[0068] Step S3: performing low-density redundancy coding block on the data stream to be burned in the data stream burning task sequence, to generate fault-tolerant coding data stream blocks, monitoring device state data corresponding to the target device in real time, and the device state data comprises storage temperature, voltage fluctuation and erasing times, and performing burning writing processing on the fault-tolerant coding data stream blocks based on the device state data, to generate device burning data stream blocks;

[0069] Step S4: performing block-by-block CRC check on the device burning data stream blocks, locating error blocks corresponding to the device burning data stream blocks, and performing online repair verification analysis on the error blocks by using corresponding redundancy information in the fault-tolerant coding data stream blocks and combining the device state data, to generate a data stream burning verification completion result.

[0070] In the embodiment of the application, please refer to Figure 1 The intelligent data burning method based on a data stream comprises the following steps:

[0071] Step S1: collecting input source data to be burned in real time through a bus interface, performing format recognition and semantic analysis on the input source data to be burned, to generate metadata information corresponding to the data to be burned, obtaining hardware storage structure information and burning protocol specification corresponding to a target device, and performing data stream description output on the metadata information corresponding to the data to be burned based on the hardware storage structure information and the burning protocol specification corresponding to the target device, to generate a data stream description file corresponding to metadata, structure information and burning instructions;

[0072] In the embodiment of the application, the PCIe 4.0 bus interface is adopted to collect the input source data to be burned in real time, the collection frequency is 100 MHz, the transmission rate reaches 32 GB / s, the data collection module is internally provided with a format recognition engine, the engine is based on a regular expression matching algorithm, can recognize more than 10 file system formats such as FAT32, NTFS and EXT4, and performs sector-by-sector scanning on the input source data to be burned, and every time 512 bytes of data are scanned, the format recognition engine compares with a preset file system feature library, for example, when it is detected that a data block contains a "FAT" identifier and the subsequent bytes conform to the cluster allocation table structure of the FAT32 file system, it can be determined that the data format is FAT32, the semantic analysis module performs deep analysis on the data with the recognized format, extracts metadata such as file name, size, creation time and permission information. For a to-be-burned data containing 100 files, the semantic analysis module can complete the analysis within 5 seconds to generate a metadata list containing file name, file type, file size, timestamp and other information. At the same time, by querying the hardware specification database of the target device, the corresponding hardware storage structure information such as storage medium type (NAND Flash, NOR Flash, etc.), page size (usually 2KB, 4KB, etc.), block size (usually 128KB, 256KB, etc.) and burning protocol specification (such as JTAG, SPI, I2C, etc.) is obtained, based on the obtained hardware storage structure information and burning protocol specification, the data stream description generator converts the metadata information into a data stream description file in a specific format. The file contains three parts: the metadata part records the basic information of the file; the structure information part divides the data into units suitable for storage according to the storage structure of the target device, such as dividing a large file into multiple data blocks matching the page size of the device; the burning instruction part generates specific burning operation instructions such as address setting, data transmission, verification, etc. according to the burning protocol specification. The finally generated data stream description file is in XML format, the size is 5% of the original data, and provides a structured data basis for subsequent burning task scheduling.

[0073] Step S2: Obtain the storage bandwidth and cache capacity corresponding to the target device, and perform burning dynamic task scheduling on the data stream description file containing the metadata, structure information and burning instruction corresponding to the target device based on the storage bandwidth and cache capacity corresponding to the target device, to generate a data stream burning task sequence;

[0074] In the embodiment of the present application, by querying the hardware specification parameters of the target device, its storage bandwidth of 50 MB / s and cache capacity of 256 MB are obtained, and the task scheduler schedules the data stream description file based on these parameters using the dynamic time window algorithm. First, the data blocks in the data stream description file are sorted according to size and dependency relationship to form an initial task queue. Then, according to the storage bandwidth and cache capacity, the amount of data that can be processed in each time window (set to 100 ms) is calculated, that is, 50 MB / s x 0.1 s = 5 MB, and the task scheduler selects data blocks from the initial task queue in turn to form task groups with a size not exceeding 5 MB, each task group containing 10-15 data blocks. In the selection process, key data blocks and data blocks with dependencies are given priority. For example, for a data burning task containing system boot program and application program, the system boot program data block will be arranged in the front task group. At the same time, considering the limitation of cache capacity, it is ensured that the data of each task group can be completely loaded into the device cache to avoid buffer overflow, and when a task group is executed, the task scheduler will dynamically adjust the composition and execution order of the subsequent task group according to the real-time state of the device (such as cache usage, storage bandwidth utilization, etc.). For example, if it is found that the device cache usage is close to the upper limit, the size of the task group will be appropriately reduced; if the storage bandwidth utilization is low, the data amount of the task group will be increased. Through such dynamic adjustment, a data stream burning task sequence containing 100 task groups is finally generated, each task group has a clear execution order and time arrangement, ensuring that the data burning process is efficient and stable.

[0075] Step S3: Low-density redundancy coding block is performed on the corresponding to-be-burned data stream in the data stream burning task sequence to generate fault-tolerant coding data stream blocks; device state data including storage temperature, voltage fluctuation and erase count of the target device are monitored in real time, and based on the device state data, the fault-tolerant coding data stream blocks are burned and written to generate device burning data stream blocks;

[0076] In the embodiment of the application, the data stream to be burned in the data stream burning task sequence is block encoded by adopting a BCH (15, 5) low-density redundancy encoding algorithm, each data block size is set to 16 KB, the encoding module starts from the head of the task sequence, reads data in sequence, and generates an independent block every 16 KB. For each block, the encoding module divides it into a plurality of 15-bit data blocks, 5 bits of which are original data and 10 bits of which are check bits, the check bits are calculated by a specific generating polynomial, for example, for original data 10110, the check bits are calculated to be 1100101110, and the two are combined to form a 15-bit encoded data block. In the burning process, the storage temperature of the target device is monitored in real time by a DS18B20 temperature sensor, the sensor accuracy is ±0.5℃, and the sampling period is 200 ms; the voltage fluctuation is monitored in real time by an INA219 voltage monitoring chip, the monitoring accuracy is ±0.01V, and the sampling period is 100 ms; the erase-write times are obtained in real time by reading the page address mapping table of the storage medium, the reading period is 500 ms, and the three data are taken as device state data and input into the burning parameter adjustment model. The model dynamically adjusts the burning parameters according to the device state data, when the storage temperature exceeds 30℃, the burning voltage is increased by 5%, from 3.3V to 3.465V; when the voltage fluctuation exceeds 0.1V, the burning clock frequency is reduced by 10%, from 50MHz to 45MHz; when the erase-write times exceed 100 times, the number of check bits is increased from 10 bits to 12 bits, for example, when the storage temperature is detected to be 32℃, the voltage fluctuation is 0.12V, and the erase-write times are 120 times, the above three adjustment measures are applied at the same time, in this way, it is ensured that the data burning process can maintain a high success rate under different device states, and finally the device burning data stream blocks are generated.

[0077] Step S4: performing block-by-block CRC check on the device burning data stream blocks, locating the error blocks corresponding to the device burning data stream blocks, simultaneously using the corresponding redundancy information in the fault tolerance encoding data stream blocks and combining the device state data to perform online repair verification analysis on the error blocks, to generate a data stream burning verification completion result.

[0078] In the embodiment of the present application, the CRC-32 check algorithm is used to check the device burning data stream block by block, the polynomial is 0x04C11DB7, the check module reads each block from the device storage medium in order, reads 16KB data each time, calculates the CRC value and compares it with the original CRC value recorded during the burning process, if they are not consistent, it is determined that the block has errors, and the address resolution module is used to locate the physical address of the error block in the storage medium. For example, if it is found that the CRC value of BLOCK_005 block does not match, its physical address range is located at 0x00010000 to 0x00013FFF, the redundant information of the fault-tolerant encoding data stream block generated in step S3 is used to repair the error block online. For the BCH encoded block, the repair module first constructs the check equation according to the check bit and the original data, and then corrects the error bit step by step through the iterative decoding algorithm. After each iteration, the CRC value is recalculated for verification. If it is not repaired successfully after 5 iterations, the re-burning process is triggered. For example, for BLOCK_005 block, the repair module determines that there are 3 error bits by analyzing the redundant information, and successfully repairs it after 3 iterations. During the repair process, the repair effect is evaluated in combination with the device state data. If the storage temperature is too high (more than 40℃) or the voltage fluctuation is too large (more than 0.2V), even if the CRC check is passed, it is considered that the repair is unreliable, and the repair is performed again. After the repair is completed, the CRC check is performed again until the check is successful. For all blocks that pass the check, the error rate and the remaining erase life are calculated according to the device state data. If the error rate exceeds 0.001% or the remaining erase life is less than 80%, the block is repaired again or marked as a potential risk block. Finally, after all blocks are verified, the data stream burning verification completion result containing the check result, repair record, error rate and remaining erase life is generated.

[0079] Further, as an embodiment of the present application, referring to Figure 2 , it is Figure 1 the detailed step flowchart of step S1 in the embodiment. In step S1 of the embodiment, the following steps are included:

[0080] Step S11: configure the bus interface including SPI, I2C and UART interface protocols to collect the corresponding to-be-burned input source data in real time;

[0081] In the embodiment of the present application, the dedicated data acquisition module is adopted to collect the input source data to be burned in real time through the bus interface. The module integrates SPI, I2C and UART interfaces, supports simultaneous and parallel collection of multiple protocol data, the SPI interface works in mode 3 (CPOL=1, CPHA=1) with a maximum clock frequency of 10 MHz, and adopts 4-wire communication (SCK, MISO, MOSI, SS); the I2C interface works in fast mode with a transmission rate of 400 kHz, and supports 7-bit and 10-bit addressing; the UART interface has a baud rate of 115200 bps, 8-bit data and 1-bit stop bit without parity bit. In the collection process, the module realizes accurate timing control through a hardware timer to ensure the accuracy of data collection. For the SPI interface, when the SS signal is detected to be pulled down, data collection is started, and data is received synchronously according to the clock signal; for the I2C interface, the data transmission direction and target device are determined by detecting the start bit and address bit; for the UART interface, asynchronous data reception is realized through a baud rate generator and a data buffer, and the collected data is stored in the module SRAM in frames. Each frame of data contains time stamp, interface type, data length and other information, forming the input source data to be burned.

[0082] Step S12: performing format recognition and analysis on the input source data to be burned to design a multi-mode data parser to support automatic recognition and analysis of various data formats including binary, text, XML and JSON, and obtaining input format analysis data to be burned;

[0083] In the embodiment of the present application, the input source data to be burned is analyzed by format recognition and analysis, and a multi-mode data parser is used. The parser integrates multiple format recognition algorithms and can automatically recognize binary, text, XML, JSON and other data formats. First, the data format is identified by file header characteristics. For binary files, the specific byte sequence of the file header is checked, such as the 0xFFD8 start marker of a JPEG file. For text files, it is detected whether ASCII printable characters are included. For XML files, it is checked whether "<" and ">" tag symbols are included. For JSON files, it is checked whether "{}" or "[ ]" symbols are included. If the file header characteristics are not obvious, further identification is performed by statistical analysis method. For example, the frequency of different characters in the data is calculated. XML and JSON files usually contain more specific symbols, while the character distribution of binary files is more random. For the recognized data of different formats, the corresponding analysis algorithm is used for analysis. For XML files, a SAX parser is used for line-by-line analysis to construct a document object model. For JSON files, a recursive descent parser is used for analysis to convert it into a key-value pair structure. The analyzed data is uniformly converted into an internal standard format to form the input format analysis data to be burned.

[0084] Step S13: Perform data field semantic parsing on the to-be-burned input format parsing data to generate metadata information corresponding to the to-be-burned data;

[0085] In the embodiment of the present application, by performing data field semantic parsing on the to-be-burned input format parsing data, using a semantic parsing engine, which can identify the meaning and relationship of the data field based on a predefined semantic rule library and a machine learning model, first, the data is processed by word segmentation, which divides continuous data strings into meaningful words or phrases. For structured data such as XML and JSON, the tag name and attribute name are directly extracted as the word segmentation result. For unstructured data, the word segmentation algorithm in natural language processing is used for processing. Then, named entity recognition is performed to identify the entity types in the data, such as device ID, firmware version, timestamp, etc. A trained BiLSTM-CRF model is used for entity recognition, which has an F1 value of 0.95 on a specific domain dataset. Then, relationship extraction is performed to analyze the relationship between entities, such as "belongs to", "associated with", "time sequence", etc. Finally, according to the identified entities and relationships, the metadata information corresponding to the to-be-burned data is generated, including data source, data type, data structure, data meaning, etc. Forming a structured metadata description.

[0086] Step S14: Obtain the hardware storage structure information corresponding to the target device and the burn protocol specification, wherein the hardware storage structure information includes flash block distribution, page address mapping table and bad block marking information;

[0087] In the embodiment of the present application, by obtaining the hardware storage structure information corresponding to the target device and the burn protocol specification, the hardware storage structure information is stored in a dedicated hardware information database, which adopts a distributed architecture with high availability and fast query capability. The corresponding hardware storage structure information, including flash block distribution, page address mapping table and bad block marking information, is retrieved from the database by the model and serial number of the target device. The flash block distribution information describes the physical structure of the flash chip, such as the total number of blocks, the number of pages per block, the number of bytes per page, etc. The page address mapping table records the mapping relationship between logical address and physical address, which is used for address conversion. The bad block marking information records the damaged blocks in the flash memory, avoiding writing data to these blocks. The burn protocol specification is stored in the protocol library, and the corresponding burn protocol specification is retrieved by the device model, including burn command format, data transmission method, verification algorithm, etc. It provides the basis for generating burn instructions subsequently.

[0088] Step S15: Based on the hardware storage structure information corresponding to the target device and the burn protocol specification, the metadata information corresponding to the to-be-burned data is output for data stream description, and a data stream description file containing metadata, structure information and burn instruction is generated.

[0089] In the embodiment of the present application, the metadata information corresponding to the data to be burned is output in a data stream description manner based on the hardware storage structure information corresponding to the target device and the burning protocol specification. A data stream generator is used to determine the storage location and manner of the data to be burned in the target device according to the hardware storage structure information. For example, the physical address where the data should be stored is calculated according to the flash memory block distribution and the page address mapping table; the damaged block is avoided according to the bad block marking information. In combination with the burning protocol specification, the corresponding burning instruction is generated. For each data block, the burning instruction containing the address information, the data length, the check code and the like is generated. The metadata information, the hardware storage structure information and the burning instruction are integrated, and are organized in a standardized data stream description format to generate a data stream description file containing the metadata, the structure information and the burning instruction. The file adopts a hierarchical structure, clearly describes the source of the data, the storage manner and the burning process, and provides complete guidance for the actual data burning operation.

[0090] Further, as an embodiment of the present application, referring to FIG. 6, a detailed step flow diagram of step S15 is shown. Figure 3 Figure 2 In the embodiment, step S15 includes the following steps:

[0091] Step S151: Obtain the corresponding each data to be burned semantic entity through the metadata information corresponding to the data to be burned, and extract the entity relationship among the data to be burned semantic entities to obtain the data entity relationship among the data to be burned entities.

[0092] ​In an embodiment of the present invention, each semantic entity of the data to be burned is extracted from the metadata information of the data to be burned. The metadata information is stored in a dedicated metadata warehouse. The warehouse adopts a distributed file system architecture and has the ability to process 100,000 records per second. The semantic entities of the data to be burned are extracted with the help of a natural language processing tool. The tool has a built-in named entity recognition model and uses a BERT pre-trained language model. It is fine-tuned on a specific domain dataset and achieves an F1 value of 0.95. For example, for a data file containing "Device ID: D20250617, Firmware version: V2.3.1, Production batch: P202503, Verification code: 0xABCD1234", the tool can accurately identify semantic entities such as "device ID", "firmware version", "production batch", and "check code". Subsequently, it extracts entity relationships and analyzes the grammatical relationships between various semantic entities through dependency parsing technology. For the sentence "The firmware version of device D20250617 is V2.3.1", by analyzing the dependency structure of the sentence, it is determined that there is an "attribute-value" relationship between "device ID" and "firmware version". Then, using the remote supervision algorithm and predefined relationship templates, it automatically obtains training data from a large amount of unlabeled text to further identify the relationship between entities. For example, there is a "production association" relationship between "production batch" and "device ID". Finally, the data entity relationship between each data entity to be burned is obtained.

[0093] Step S152: constructing an entity relationship graph for each semantic entity of the data to be programmed based on the data entity relationship between each entity of the data to be programmed, so as to generate an entity relationship graph of the data to be programmed;

[0094] In the embodiment of the present application, the data entity relationship graph to be burned is constructed based on the previously obtained data entity relationship, a graph database Neo4j is used as a storage and processing platform, the database supports 100,000 graph operations per second, can efficiently process the storage and query of entity relationship, each data semantic entity to be burned is taken as a node in the graph, and the data entity relationship between entities is taken as an edge in the graph. For example, the semantic entities such as “device ID: D20250617”, “firmware version: V2.3.1”, “production batch: P202503”, “check code: 0xABCD1234” are converted into nodes in the graph, each node contains entity type, name, value and other attributes, the “property-value” relationship between “device ID” and “firmware version” is converted into a directed edge, the type of the edge is “property-value”, and similarly, the “production association” relationship between “production batch” and “device ID” is also converted into a directed edge. In this way, all data semantic entities to be burned and their relationships are converted into a graph structure to generate a data entity relationship graph to be burned. The graph comprehensively displays the association between each part of the data to be burned, and provides a structured data basis for subsequent generation of executable burning instructions.

[0095] Step S153: executable burning instructions are generated based on the burning protocol specification corresponding to the target device for the data entity relationship graph to be burned, to generate target device executable burning instructions corresponding to the data to be burned;

[0096] In the embodiment of the present application, executable burning instructions are generated for the data entity relationship graph to be burned based on the burning protocol specification corresponding to the target device, the burning protocol specification of the target device is stored in a dedicated protocol library, the protocol library adopts a classified and layered storage structure, contains burning protocol specifications of more than 1000 different devices, and a protocol parsing engine is used to parse the protocol specification, the engine can identify information such as command format, parameter definition, data transmission mode in the protocol. For example, for a certain type of target device, the burning protocol specification requires that the burning instruction format is “CMD+parameter 1+parameter 2+check sum”, wherein “CMD” is a command code, “parameter 1” is a device ID, and “parameter 2” is a data block address. Related information is extracted from the data entity relationship graph to be burned, “device ID: D20250617” is converted into the format required by the protocol specification, the data block address is determined, the check sum is calculated, and according to the command definition in the protocol specification, these information is combined into executable burning instructions, such as “CMD001+D20250617+0x0000+0xABCD”. All data entities in the entity relationship graph are parsed and converted according to the protocol specification to generate a series of target device executable burning instructions, ensuring that the instructions meet the burning requirements of the target device.

[0097] Step S154: Based on the hardware storage structure information corresponding to the target device and in combination with the target device executable burn instruction corresponding to the to-be-burned data, data stream description output is performed on the metadata information corresponding to the to-be-burned data, to generate a data stream description file containing metadata, structure information and burn instruction.

[0098] In the embodiment of the present application, by performing data stream description output on the metadata information corresponding to the to-be-burned data based on the hardware storage structure information corresponding to the target device and in combination with the target device executable burn instruction corresponding to the to-be-burned data, the hardware storage structure information of the target device is stored in a special hardware information library, and the information library contains detailed information such as storage partition, address range, read-write permission and the like of various devices. For example, the hardware storage structure of a certain target device is divided into a boot area (address range 0x0000-0x1000), a firmware area (address range 0x1000-0x8000) and a configuration area (address range 0x8000-0xFFFF). According to the nature and burn instruction of the to-be-burned data, the storage position of the data in the hardware storage is determined. For firmware data, it is determined according to the burn instruction that it should be stored at the position starting from the address 0x1000 of the firmware area. By combining the metadata information, the information such as source, purpose and format of the data is associated with the hardware storage structure information and the burn instruction. Using a standardized data stream description language, these information is organized into a structured document, to generate a data stream description file containing metadata, structure information and burn instruction. The file clearly describes the storage mode and burn process of the to-be-burned data in the target device, and provides detailed guidance for the actual data burn operation.

[0099] Further, step S2 includes the following steps:

[0100] Step S21: Obtain the storage bandwidth corresponding to the target device through continuous writing test;

[0101] In the embodiment of the present application, the storage bandwidth corresponding to the target device is obtained by using a dedicated storage performance test device to perform continuous writing test. The test device is equipped with a high-speed data transmission interface and is connected to the target device through an SPI bus. The working frequency is set to 20MHz. During the test, a continuous data block with a size of 128MB is generated. The data block is filled with a random byte sequence to ensure the randomness of the data. The data block is continuously written into the storage medium of the target device at a fixed transmission rate. After each writing operation is completed, the start time and the end time of the data transmission are recorded to the microsecond level. Ten continuous writing tests are performed, and the interval between each test is 5 seconds to avoid the influence of factors such as overheating of the storage device on the test results. For example, the first writing of 128MB data takes 6.5 seconds, the second takes 6.3 seconds, and so on. The bandwidth value of each test is calculated by the formula "storage bandwidth=data amount / transmission time". The final storage bandwidth is obtained by performing arithmetic average on the results of the 10 tests. Assuming that the bandwidth values calculated by the 10 tests are 19.69MB / s, 20.32MB / s, 19.87MB / s, etc., the average storage bandwidth is 20.05MB / s. This value serves as an important basis for subsequent burning task scheduling.

[0102] Step S22: calculating and obtaining the cache capacity corresponding to the target device according to the page address mapping table corresponding to the target device;

[0103] In the embodiment of the present application, the page address mapping table is called from the hardware information library of the target device. The table is stored in the form of a binary file and contains detailed information such as address mapping information of each physical page in the device storage medium, page size, and whether it is a bad block. Assuming that the target device uses a NAND Flash storage medium, the page size is 4KB, and the total number of pages is 2048. By traversing the page address mapping table, the number of valid pages is counted. The cache capacity corresponding to the target device can be calculated by multiplying the number of valid pages by the page size. In the traversal process, a special binary file analysis tool is used to read the page address mapping table content byte by byte. After reading the information of a page, the "bad block flag" field is judged. If the flag is "0", the page is considered as a valid page and counted. After traversing the entire page address mapping table, assuming that the number of valid pages is 2000, the cache capacity of the target device is 8MB, which is calculated as "2000 pages x 4KB / page = 8MB". This cache capacity information will be used to evaluate the use of cache during data burning.

[0104] Step S23: performing burning dependency relationship analysis on the data stream description file corresponding to the metadata, structure information and burning instruction, to mine and analyze the dependency relationship of each to-be-burned data stream in the data stream description file based on the burning instruction, and generating the data burning dependency relationship between each to-be-burned data stream;

[0105] In the embodiment of the present application, the data stream description file containing metadata, structure information and burning instructions is processed by using a special text parser, the parser reads line by line, first identifies the burning instruction field in the file, the burning instruction is in a specific format, for example, "CMD: WRITE, ADDR: 0x1000, DATA: FILE1", which explicitly indicates the write operation, the target address and the data source. By analyzing the burning instructions, the dependency relationship between each data stream to be burned is mined. If the data FILE2 in the instruction "CMD: WRITE, ADDR: 0x2000, DATA: FILE2" depends on the data read by the instruction "CMD: READ, ADDR: 0x1000, DATA: FILE1" for preprocessing, it is determined that there is a data dependency relationship between FILE2 and FILE1; if the instruction "CMD: WRITE, ADDR: 0x3000, DATA: FILE3" can be executed only after the execution of the instruction "CMD: WRITE, ADDR: 0x2000, DATA: FILE2" is completed, it is determined that there is a sequential dependency relationship between FILE3 and FILE2. In the above manner, all the burning instructions in the data stream description file are analyzed, the identified dependency relationship is recorded in the form of a table, the table contains the fields of "source data stream", "target data stream" and "dependency type", and finally the complete data burning dependency relationship between each data stream to be burned is generated.

[0106] Step S24: based on the data burning dependency relationship between each data stream to be burned, the directed acyclic graph of the data stream to be burned is constructed between each data stream to be burned in the data stream description file, to generate a data stream directed acyclic graph to be burned; and based on the data stream directed acyclic graph to be burned, the topological burning sequence of each data stream to be burned in the data stream description file is sorted, to generate a data stream initial burning sequence;

[0107] In the embodiment of the present application, by using the data burning dependency relationship generated according to step S23, a directed acyclic graph of the to-be-burned data stream is constructed using a graph algorithm, each to-be-burned data stream is taken as a node in the graph, the node attribute contains information such as data stream name, data size, data type, and the like, according to the data burning dependency relationship, a directed edge is established between nodes with dependency relationship, the direction of the edge is from the dependency source node to the dependency target node, and the dependency type is marked in the attribute of the edge. For example, if the data stream "FILE2" depends on "FILE1", a directed edge is drawn from the node "FILE1" to the node "FILE2", and the edge attribute is marked as "data dependency". In the construction process, a topological sorting algorithm is used to detect and eliminate possible circular dependencies, so as to ensure that the generated graph is a directed acyclic graph. Assuming that there is a circular dependency "FILE4→FILE5→FILE4" in the graph constructed initially, the circular dependency is broken by adjusting the dependency relationship, introducing a virtual node or re-planning the data stream processing order, after the directed acyclic graph is constructed, the topological sorting algorithm is used again to sort the nodes in the graph, and the initial burning order sequence of the data stream is generated according to the topological order of the nodes. In the sorting process, the nodes without predecessor nodes (i.e. without dependency on other data streams) are preferentially arranged at the front end of the sequence, and are sequentially arranged to the rear, so as to ensure that the burning order of the data streams in the sequence conforms to the dependency relationship.

[0108] Step S25: performing burning dynamic task scheduling on the initial burning order sequence of the data stream based on the storage bandwidth and the cache capacity corresponding to the target device, and generating a data stream burning task sequence.

[0109] In the embodiment of the present application, by combining the previously acquired storage bandwidth 20.05 MB / s and the calculated cache capacity 8 MB, the initial data flow burning sequence is scheduled for burning dynamic task scheduling. The data in the sequence is divided into a task unit of 32 KB, and the theoretical transmission time of each task unit under the current storage bandwidth is calculated, which is "transmission time = data volume / storage bandwidth". At the same time, considering the cache capacity of the target device, the use of the cache during the data burning process is simulated. When the data volume of a certain task unit exceeds the remaining capacity of the cache, it is divided into multiple sub-tasks to ensure that the data can be written into the cache smoothly. For example, for a 64 KB data task unit, the remaining capacity of the cache is 48 KB, which is divided into 48 KB and 16 KB two sub-tasks. According to the transmission time of the task unit and the cache usage, the priority of each task unit is calculated, which takes into account the data volume, the urgency of the dependency relationship, and the impact on the overall burning process. For example, for the task unit related to the key system startup file, a higher priority is given; for the task unit of the non-key configuration file, a lower priority is given. The task units are rearranged according to the priority order to generate a data flow burning task sequence. In the burning process, the data is transmitted to the target device by the DMA controller according to the task sequence order, and the storage bandwidth and cache usage are monitored in real time to dynamically adjust the task execution order and transmission rate, ensuring that the data burning process is completed efficiently and stably.

[0110] Further, step S25 comprises the following steps:

[0111] Step S251: by dividing the initial data flow burning sequence into data segments, obtaining the corresponding data volume of each data segment, and constructing a data flow burning transmission topology network based on the data transmission order and dependency relationship of each data segment in the initial data flow burning sequence, and combining the data volume of each data segment, wherein the data segment is taken as a node, the data volume is taken as a node attribute, and the data transmission order and dependency relationship are taken as an edge;

[0112] In the embodiment of the present application, by dividing the data stream initial burning order sequence into 2048 data segments according to the fixed size rule, the size of each data segment is set to 2KB, which ensures the integrity and independence of the data unit, and the CRC32 check algorithm is used to calculate the check value of each data segment, and the check value is stored as part of the data segment for accuracy verification of subsequent data transmission. When constructing the data stream burning transmission topology network, a high-performance graph database is used as the storage and calculation carrier. Each data segment is taken as a node of the topology network, and the node attributes include data size, check value, creation timestamp and other key information. For example, the data size attribute of node D100 is 2048 bytes, the check value is 0x12345678, and the creation timestamp is 1678901234567. The edges between nodes are determined by the data transmission order and the dependency relationship. If the burning of data segment D150 needs to be performed after D149 is completed, a directed edge is established from D149 to D150, and the edge attribute records the transmission order number as 149→150, and the dependency type is sequential dependency. If the burning of D160 depends on the completion of data preparation of D155 and D158, directed edges are established from D155 and D158 to D160, and the edge attribute indicates that the dependency type is data dependency. For data segment sets with complex cyclic dependencies, virtual nodes are introduced for decoupling processing to ensure that the topology network is a directed acyclic graph structure. The finally constructed topology network contains 2048 nodes and 3000 directed edges, and is stored in a 1GB memory database, supporting 150,000 graph query and update operations per second.

[0113] Step S252: Calculate the betweenness centrality of each node through the data stream burning transmission topology network, and determine the burning transmission priority of each data segment based on the betweenness centrality of each node to generate a data stream burning transmission priority sequence.

[0114] In the embodiment of the present application, the Brandes algorithm is used to calculate the betweenness centrality of each node in the data stream burning transmission topology network. In order to improve the calculation efficiency, 12 CPU cores are allocated for parallel processing of the calculation task. Each node maintains a dependency count table and a path contribution table during the calculation process. The dependency count table records the number of shortest paths passing through the node, and the path contribution table records the contribution degree of the node to each shortest path. After 40 rounds of iteration calculation, the betweenness centrality values of all nodes are obtained, which are in the range of 0.0005 to 0.12. Based on the betweenness centrality values, the burning transmission priority of the data segments is determined. The radix sort algorithm is used to sort the nodes. The time complexity of this algorithm is O(n), which can efficiently complete the sorting task of large-scale data. The sorting results are stored in a double-ended queue. The data segment corresponding to the node with high betweenness centrality value is located at the head of the queue and is preferentially burned and transmitted. For example, the data segment D200 with a betweenness centrality value of 0.12 is arranged at the head of the queue because it is in a critical position in the data stream transmission path and has a greater impact on the overall burning process. The data segment D1800 with a betweenness centrality value of 0.0005 is arranged at the rear position of the queue. The finally generated data stream burning transmission priority sequence clearly arranges the data segments on the critical path (such as system core startup files and basic driver data) in the front row and the data segments on the non-critical path (such as user configuration files and log recording data) in the rear row, providing clear priority guidance for subsequent burning transmission.

[0115] Step S253: Based on the storage bandwidth corresponding to the target device, the bandwidth burning transmission pressure of the corresponding data segment in the data stream burning transmission priority sequence is evaluated to obtain the corresponding storage bandwidth burning transmission pressure of each data segment in the target device.

[0116] In the embodiment of the present application, the data segments in the data stream burn-in transmission priority sequence are evaluated for bandwidth burn-in transmission pressure by targeting the storage bandwidth characteristics of the target device. The target device uses a SPI Flash storage interface, and its theoretical maximum bandwidth is 32 MB / s. Considering the protocol overhead, bus competition and other factors in actual transmission, the actual available bandwidth is 26 MB / s. The data segments in the priority sequence are evaluated in groups of 32, and the total data amount of each group of data segments is 64 KB. The theoretical transmission time of each group of data segments is calculated using a sliding window algorithm, and the formula is: transmission time = data amount / storage bandwidth. If the total data amount of a certain group of data segments is 64 KB, the theoretical transmission time is 64 KB / 26 MB / s≈2.46 ms. However, in actual transmission, due to the existence of instruction transmission overhead, bus arbitration delay and other conditions, a loss factor of 1.3 times is introduced, and the actual transmission time is about 3.2 ms. Through the evaluation of all groups, the groups whose transmission time exceeds 5 ms are marked, such as the 40th to 45th group of data segments, whose actual transmission time reaches 5.5 ms, which is determined as a high pressure area, indicating that these data segments will bring great pressure to the storage interface when transmitted under the current bandwidth condition, which may affect the burn-in efficiency and stability.

[0117] Step S254: quantifying the cache transmission saturation of the corresponding data segments in the data stream burn-in transmission priority sequence based on the cache capacity corresponding to the target device, to obtain the cache burn-in transmission saturation rate of each data segment in the target device;

[0118] In the embodiment of the present application, the cache transmission saturation of the data segments in the data stream burn-in transmission priority sequence is quantified according to the cache capacity of the target device. The target device is equipped with a direct mapping cache with a capacity of 8 KB, and the cache block size is set to 32 bytes. The data segments are loaded into the cache in order according to the priority sequence, and the LRU (Least Recently Used) replacement strategy is used to manage the cache content to ensure that frequently used data is retained in the cache. The cache hit rate and miss rate of each data segment are accurately calculated by simulating the cache access process. For example, the data segment D300 is 2 KB in size and contains 64 cache blocks. It is accessed 120 times during the burn-in process, of which 90 times are hits and 30 times are misses. The cache hit rate is 75%. For all data segments, the cache occupancy rate is calculated using the formula: cache occupancy rate = data segment size / cache capacity x cache hit rate. It is found that the cache occupancy rates of data segments D500 to D550 exceed 85%, which are marked as a high saturation area, meaning that these data segments occupy too much space in the cache, which may cause frequent cache replacement operations, affecting data reading efficiency and thus adversely affecting the burn-in process.

[0119] Step S255: Perform target device current load analysis according to the corresponding storage bandwidth burn-in transmission pressure and cache burn-in transmission saturation rate of each data segment in the target device to obtain the current burn-in load of the target device; dynamically adjust the burn-in transmission rate of each data segment according to the current burn-in load of the target device, and perform burn-in dynamic task scheduling on the initial burn-in sorting sequence of the data stream based on the burn-in transmission rate of each data segment to generate a data stream burn-in task sequence.

[0120] In the embodiment of the application, by comprehensively analyzing the corresponding storage bandwidth burn-in transmission pressure and cache burn-in transmission saturation rate of each data segment in the target device, the current load of the target device is analyzed, and a load calculation formula is defined: load amount = storage bandwidth pressure coefficient × 0.6 + cache saturation rate coefficient × 0.4, wherein the storage bandwidth pressure coefficient and the cache saturation rate coefficient are normalized to between 0 and 1, for example, the storage bandwidth pressure of a certain data segment is 80%, and the normalized pressure coefficient is 0.8; the cache saturation rate is 90%, and the normalized saturation rate coefficient is 0.9, so the load amount = 0.8 × 0.6 + 0.9 × 0.4 = 0.84. Based on the calculated load amount, the burn-in transmission rate of each data segment is dynamically adjusted using a PID controller, and a load amount threshold is set. When the load amount exceeds 0.8, the transmission rate is reduced by 15%; when the load amount is less than 0.4, the transmission rate is increased by 8%, and the initial transmission rate is set to 22 MB / s. After multiple iterations of adjustment, for the high-load data segment D600, the transmission rate is reduced from 22 MB / s to 18.7 MB / s according to the load amount 0.88; for the low-load data segment D700, the transmission rate is increased to 23.8 MB / s according to the load amount 0.35, and finally stabilizes at a suitable transmission rate of 19 MB / s. According to the adjusted transmission rate, the initial burn-in sorting sequence of the data stream is rearranged, the data segments in the high-load area are dispersed and inserted into the low-load period for transmission, and the final data stream burn-in task sequence is generated. The sequence is transmitted through the DMA (Direct Memory Access) controller, effectively reducing the CPU load, realizing efficient and stable operation of the data burn-in process, and ensuring that the data burn-in task can be quickly and accurately completed under the resource limitation of the target device.

[0121] Further, step S3 includes the following steps:

[0122] Step S31: Low-density redundancy coding block is performed on the corresponding data stream to be burned in the data stream burn-in task sequence to generate fault-tolerant coding data stream blocks.

[0123] In the embodiment of the present application, the data stream to be burned in the task sequence is burned, and the fixed size and low redundancy coding are combined for block processing, each data block size is set to 32 KB, a special data block device is used, data is read in sequence from the starting position of the task sequence, and each 32 KB generates an independent block and is assigned a unique number. For example, for a 512 KB data stream to be burned, 16 blocks can be obtained, numbered from BLOCK_001 to BLOCK_016. After the block is completed, low density redundancy coding is implemented for each block, and a parity check coding algorithm is used. The algorithm generates a check bit by calculating the parity of the binary bits in the data block. Taking the BLOCK_001 block as an example, it contains 32 KB (i.e. 32*1024=32768 bytes) of data, each 8-bit data is taken as a group, and the number of "1"s in each group is calculated. If there are an odd number of "1"s, the check bit is set to 0 for odd parity, and the check bit is set to 1 for even parity; otherwise, the generated check bit is appended to the end of the block to form a fault-tolerant coded data stream block. After this processing, each block not only has basic error detection capability, but also reduces the additional overhead of data storage and transmission due to low redundancy coding, and prepares for subsequent burning operation.

[0124] Step S32: Real-time monitoring of the storage temperature corresponding to the target device;

[0125] In the embodiment of the present application, the storage temperature of the target device is monitored in real time by the built-in temperature sensor. The temperature sensor is selected as DS18B20, the measurement range is -55℃ to +125℃, the accuracy can reach ±0.5℃, the sensor collects temperature data at intervals of every 100 milliseconds, and the data is transmitted to the device monitoring module through the one-wire protocol. The monitoring module processes the received temperature data. First, the data is filtered to remove abnormal values caused by electromagnetic interference and other factors. A median filtering algorithm is used, and 5 consecutive temperature data are collected each time, sorted from small to large, and the middle value is taken as the effective temperature data. For example, the 5 consecutive data collected are 25.3℃, 25.1℃, 24.9℃, 25.5℃ and 25.2℃, and the middle value 25.2℃ after sorting is the effective temperature data. The processed data is stored in the cache of the monitoring module and is updated in real time, providing an accurate basis for subsequent burning strategy adjustment based on temperature data.

[0126] Step S33: Real-time monitoring of the voltage fluctuation corresponding to the target device;

[0127] In the embodiment of the present application, the voltage fluctuation of the target device is monitored in real time by using a high-precision voltage monitoring chip. The AD7865 chip is selected, which has a resolution of 16 bits and a sampling rate of up to 100 kSPS, and can accurately capture the slight changes in voltage. The chip is connected to the power line of the target device through an analog front-end circuit, and the voltage signal is collected in real time. The collected analog voltage signal is converted to a digital signal in the chip and then transmitted to the data processing unit. The data processing unit analyzes the digital signal and calculates the average value, maximum value, minimum value, and fluctuation range of the voltage. For example, 100 voltage data are collected in 1 second, and the average value is calculated as 3.28 V, the maximum value is 3.32 V, the minimum value is 3.25 V, and the fluctuation range is 0.07 V. At the same time, the normal range threshold of the voltage is set as 3.3 V±0.1 V. When the monitored voltage exceeds this range, the early warning mechanism is triggered, and the abnormal information is marked and stored, so as to adjust the burning parameters in combination with the voltage fluctuation during the burning process.

[0128] Step S34: monitoring the corresponding erase-write times of the target device in real time;

[0129] In the embodiment of the present application, the erase-write times are monitored in real time by reading the page address mapping table of the storage medium of the target device. The storage medium uses NAND Flash, and its page address mapping table records the usage state information of each storage page, including the erase-write times, whether it is a bad block, etc. A special storage medium management module is used to read the page address mapping table every 500 milliseconds. The module summarizes and counts the read erase-write times data, calculates the average erase-write times of the entire storage medium, and the distribution of pages with high erase-write times. For example, it is found that there are 2048 storage pages in the storage medium, and the total erase-write times are 102400 times, so the average erase-write times are 102400÷2048=50 times. At the same time, it is found that the erase-write times of the 10 pages numbered PAGE_001 to PAGE_010 exceed 80 times. These information is sorted and recorded. By continuously monitoring the erase-write times, the wear degree of the storage medium can be understood, and key reference data for the burning operation is provided.

[0130] Step S35: taking the storage temperature, voltage fluctuation, and erase-write times of the target device as device state data, and performing burning write processing on the fault-tolerant encoding data stream block based on the device state data to generate device burning data stream blocks.

[0131] In the embodiment of the present application, by integrating the previously acquired storage temperature, voltage fluctuation data and the number of erasing times into device state data, a Bayesian network learning algorithm is used to train the model, the burn-in data stream block feature parameters and device parameters in the data are taken as network nodes, and the burn-in result is taken as the output node. The network structure is set to three layers, the input layer contains 10 nodes corresponding to data size, encoding type and other features, the hidden layer contains 8 nodes, and the output layer is 1 node representing the write error probability. The maximum likelihood estimation method is used to calculate the conditional probability table between nodes. For example, when the data block size is 16KB, BCH encoding is used, and the target device storage page erasing times exceed 50 times, the maximum likelihood estimation method is used to calculate the conditional probability between nodes according to the historical burn-in state data of the device. For example, assuming that in the historical data, when the data block size is in the interval of 16KB-32KB, BCH encoding is used, and the target device storage page erasing times are in the range of 50-100 times, a total of 100 burn-in operations are performed, of which 12 times fail to write, then under this condition combination, the conditional probability of write failure is 12÷100=0.12. Store this probability relationship in the conditional probability table to obtain a Bayesian network model that can predict the write error probability of the target device for the data stream block. Secondly, the state data of the current target device is acquired, including the real-time temperature of the device (measured by the built-in temperature sensor, accuracy ±0.5℃), the current erasing times of the storage medium (obtained by reading the counter in the page address mapping table), the remaining available storage space (calculated according to the total capacity of the storage medium and the used space), and the detailed parameters of the fault-tolerant encoding data stream block (such as block number, data content digest, encoding type, number of check bits), etc. The above data is arranged into a vector that conforms to the input format of the Bayesian network model, and input into the trained model. The model performs inference calculation according to the input data and the established conditional probability relationship. For example, when the data size is 16KB and BCH encoding is used, a probability value is obtained. Combined with the storage page erasing times of 60 times, another adjustment probability is obtained. Considering the influence of the real-time temperature of 28℃, etc., a series of probability operations are performed, for example, the final write error probability of the block is 0.15. The write error probability of each block is calculated to form a list containing the block number and the corresponding error probability. Then, the high error probability threshold is set to 0.1 and the low error probability threshold is set to 0.03. The write error probability list obtained previously is traversed, and each fault-tolerant encoding data stream block is processed. For BLOCK_001 block with a write error probability of 0.15, it is determined as a high error probability block, and the corresponding standard burn-in voltage is increased from 3.3V to 3.3V×(1+5%)=3.465V, and the standard clock frequency is reduced from 50MHz to 50MHz×(1-10%)=45MHz.BLOCK_010 block of 02 is determined as a low error probability block, the burn voltage is increased to 3.465V*(1+3%)=3.569V (the increase is less than 5% of the high error probability block), the clock frequency is increased to 45MHz*(1+8%)=48.6MHz (the increase is less than 10% of the high error probability block), and the adjusted burn voltage and clock frequency are used to sequentially perform a write operation on all fault-tolerant encoding data blocks by a dedicated burn controller to write data into the target device storage medium, and finally generate a device burn data block.

[0132] Further, step S31 includes the following steps:

[0133] The corresponding to-be-burned data stream in the data stream burn task sequence is processed by blocking to obtain each to-be-burned data stream block;

[0134] In the embodiment of the present application, the to-be-burned data stream in the data stream burn task sequence is processed by blocking by adopting a fixed size blocking strategy, each data block size is set to 16KB, and the operation is performed by a dedicated data block processing device, the device starts from the head of the data stream burn task sequence, reads data in sequence, and generates an independent data block after reading 16KB data, and at the same time, a unique identification number is allocated to each block. For example, for a to-be-burned data stream with a size of 256KB, after blocking, 16 data blocks are obtained, numbered BLOCK_001 to BLOCK_016. In the blocking process, the device checks the boundary of each block to ensure the integrity of the data. If the amount of data at the end of the data stream is less than 16KB, the remaining data is separately used as a block, and specific padding bytes (such as 0xFF) are filled after it to make the block size reach the standard. After the blocking is completed, all blocks are stored in the cache area in the order of the original data stream to form each to-be-burned data stream block, providing a structured data unit for subsequent processing.

[0135] Preferably, the to-be-burned data content corresponding to each block is obtained through each to-be-burned data stream block, and each to-be-burned data stream block is grouped based on the to-be-burned data content corresponding to each block to obtain a key to-be-burned data block and a non-key to-be-burned data block.

[0136] In the embodiment of the present application, the content of each to-be-burned data block is obtained by using a content analysis tool to analyze the content of each to-be-burned data block, which can identify the key fields, data types and the association between data in the data. Taking a data stream containing system configuration information and log records as an example, the data in a certain block contains system startup parameters, network configuration and other key information, and another block is the recent operation log record. Through the pre-set content classification rules, the block data is analyzed, the data blocks containing system core parameters, driver code and other data important to the system operation are classified as key to-be-burned data blocks, and the data blocks containing user operation records, temporary cache data and other data less affecting the core function of the system are classified as non-key to-be-burned data blocks. For example, after analysis, it is determined that 6 blocks of BLOCK_001, BLOCK_003, BLOCK_005, etc. are key to-be-burned data blocks, and the remaining 10 blocks are non-key to-be-burned data blocks, and the classification results are stored in the form of a table, which contains block number, data content summary, category and other fields, facilitating subsequent processing.

[0137] Preferably, the content correlation coefficient and the number of erasing times of the corresponding data block are obtained through the key to-be-burned data block and the non-key to-be-burned data block, and the optimal number of check bits of the corresponding data block is calculated based on the content correlation coefficient and the number of erasing times, and the key to-be-burned data block is encoded with high redundancy and the non-key to-be-burned data block is encoded with low redundancy according to the optimal number of check bits of the corresponding data block, to generate key to-be-burned encoded blocks and non-key to-be-burned encoded blocks;

[0138] In the embodiment of the present application, the content correlation coefficient of each data block is calculated by using a special data analysis algorithm, and the content correlation coefficient is obtained by calculating the similarity between the block data and the standard key data template. The cosine similarity algorithm is used to convert the data into a vector form for calculation, and the value range is 0 to 1. The value closer to 1 indicates a higher correlation. The erase count is obtained according to the page address mapping table and the historical operation record of the target device. Each storage page has a corresponding erase count counter. For example, the content correlation coefficient of the key to-be-burned data block BLOCK_001 and the standard key data template is 0.95, and the erase count of the storage page where it is located is 50 times. The content correlation coefficient of the non-key to-be-burned data block BLOCK_010 is 0.3, and the erase count is 10 times. According to the pre-established mathematical model, the optimal check bit number is calculated in combination with the content correlation coefficient and the erase count. The model considers factors such as data importance and storage reliability. For the key to-be-burned data block, the calculated optimal check bit number is relatively large because of the high content correlation coefficient and the erase count that may affect the data stability. The optimal check bit number of the non-key to-be-burned data block is relatively small. According to the optimal check bit number, a high-redundancy error correction encoding algorithm such as BCH encoding is used for the key to-be-burned data block to increase the check bit to improve the data error correction capability. A low-redundancy encoding algorithm such as parity check code is used for the non-key to-be-burned data block to reduce the encoding overhead while ensuring basic data checking. After the encoding processing, the key to-be-burned encoding block and the non-key to-be-burned encoding block are finally generated, effectively improving the reliability and efficiency of data storage.

[0139] Preferably, by adding an index tag containing position information and redundant information corresponding to each key to-be-burned encoding block and non-key to-be-burned encoding block, a fault-tolerant encoding data stream block is generated.

[0140] In the embodiment of the present application, the index tags are added to the key to-be-burned encoding blocks and non-key to-be-burned encoding blocks by the index tag generation tool, the index tags contain two parts of position information and redundancy information, the position information records the offset address and storage order of the blocks in the original data stream, which is accurate to the byte level, so as to quickly locate in the process of burning and reading, the redundancy information contains the encoding type of the blocks, the number of check bits and the association relationship with other blocks, etc., which is used for data checking and recovery. For example, in the index tag generated for the key to-be-burned encoding block BLOCK_001, the position information shows that its offset address in the original data stream is 0 and the storage order is the first; the redundancy information shows that the BCH encoding is used, the number of check bits is 16 bits, and there is data association with BLOCK_003 and BLOCK_005. In the index tag of the non-key to-be-burned encoding block BLOCK_010, the position information is the offset address of 96 KB and the storage order is the seventh, and the redundancy information shows that the parity check code is used, the number of check bits is 1 bit, and there is no special association relationship. The index tags are added to the head of the block data in a specific format, each tag occupies a fixed number of bytes (such as 64 bytes), after the addition is completed, the fault-tolerant encoding data stream blocks containing complete index tags are generated, these blocks will be stored and transmitted according to the new order, in the data burning process, the fault-tolerant capability and recovery efficiency of the data can be effectively improved, and the integrity and accuracy of the data are ensured.

[0141] Further, the step S35 of performing the burn-write processing on the fault-tolerant encoding data stream blocks based on the device state data comprises the following steps:

[0142] Obtaining device historical burn state data, and training a Bayesian network model based on the device historical burn state data to predict the write error probability of the target device corresponding to the data stream blocks;

[0143] In the embodiment of the present application, the device historical burning state data is retrieved from the device historical data storage library which adopts a distributed file system architecture and stores detailed data of 10,000 sets of burning tasks in the past 3 years. Each set of data contains burning time, feature parameters of burning data stream blocks (such as data size, encoding type, number of check bits), target device model, storage medium parameters (such as number of erasing and writing times, page address mapping table state), and burning result (writing success or failure and error type). The Bayesian network learning algorithm is used to train the model, the burning data stream block feature parameters and device parameters in the data are taken as network nodes, and the burning result is taken as the output node. The network structure is set to three layers, the input layer contains 10 nodes corresponding to data size, encoding type and other features, the hidden layer contains 8 nodes, and the output layer is one node representing the writing error probability. The maximum likelihood estimation method is used to calculate the conditional probability table between nodes. For example, when the data block size is 16 KB, BCH encoding is used, and the number of erasing and writing times of the target device storage page exceeds 50 times, the maximum likelihood estimation method is used to calculate the conditional probability between nodes according to the device historical burning state data. For example, it is assumed that in the historical data, when the data block size is in the interval of 16 KB-32 KB, BCH encoding is used, and the number of erasing and writing times of the target device storage page is in the range of 50-100 times, a total of 100 burning operations are performed, of which 12 times fail to write. Therefore, the conditional probability of writing failure under this condition combination is 12÷100=0.12. The probability relationship is stored in the conditional probability table. For each different state combination of the input node and the state of the output node, the corresponding conditional probability is calculated and stored in this way. The probability relationship is stored in the conditional probability table. After 500 iterations of training, the prediction accuracy of the model on the test set reaches 85%, and a Bayesian network model capable of predicting the writing error probability of the target device for the data stream block is obtained.

[0144] Preferably, the target device writing prediction analysis of the fault-tolerant encoding data stream block is performed by inputting the current corresponding device state data into the Bayesian network model to obtain the writing error probability of each fault-tolerant encoding data stream block.

[0145] In the embodiment of the present application, by acquiring the state data of the current target device, including the real-time temperature of the device (measured by the built-in temperature sensor, accuracy ±0.5℃), the current erase-write times of the storage medium (obtained by reading the counter in the page address mapping table), the remaining available storage space (calculated according to the total capacity of the storage medium and the used space), and the detailed parameters of the fault-tolerant encoding data stream block (such as block number, data content digest, encoding type, number of check bits), the above data is arranged into a vector conforming to the input format of the Bayesian network model, and input into the trained model. According to the input data, the model performs inference calculation according to the established conditional probability relationship. Taking the fault-tolerant encoding data stream block numbered BLOCK_001 as an example, the block data size is 16KB, BCH encoding is used, the number of check bits is 16, the current storage page erase-write times of the target device is 60, and the real-time temperature is 28℃. The model will consider the probability relationship between multiple node states such as data size, encoding type, erase-write times, real-time temperature, etc. The conditional probability table is traversed, and each related probability is combined and operated. For example, when the data size is 16KB and BCH encoding is used, a probability value is obtained, combined with the storage page erase-write times of 60, another adjustment probability is obtained, and the influence of the real-time temperature of 28℃ is considered. After a series of probability operations, for example, the final write error probability of the block is 0.15; all 100 fault-tolerant encoding data stream blocks are calculated in turn to obtain the write error probability corresponding to each block, and a list containing the block number and the corresponding error probability is formed, which provides a basis for subsequent programming strategy adjustment.

[0146] Preferably, based on the write error probability corresponding to each fault-tolerant encoding data stream block, the corresponding fault-tolerant encoding data stream block is subjected to programming write processing. If it is determined as a high error probability block according to the write error probability, the programming voltage is increased by less than or equal to 5% of the standard programming voltage, and the clock frequency is reduced by less than or equal to 10% of the standard clock frequency; if it is determined as a low error probability block according to the write error probability, the programming voltage is increased by less than 5% of the high error probability block, and the clock frequency is increased by less than 10% of the high error probability block, thereby performing the write operation of each fault-tolerant encoding data stream block to generate the device programming data stream block.

[0147] In the embodiment of the present application, by setting the high error probability threshold as 0.1, the low error probability threshold as 0.03, traversing the previously obtained write error probability list, processing each fault-tolerant encoding data stream block, for the BLOCK_001 block with a write error probability of 0.15, it is determined as a high error probability block, the corresponding standard burning voltage is increased from 3.3V to 3.3V x (1+5%) = 3.465V, the voltage is adjusted accurately by the voltage adjustment module, the adjustment accuracy is ±0.01V; the standard clock frequency is reduced from 50MHz to 50MHz x (1-10%) = 45MHz, the clock signal frequency is adjusted by using the clock generator, the adjustment accuracy is ±1MHz, for the BLOCK_010 block with a write error probability of 0.02, it is determined as a low error probability block, the burning voltage is increased to 3.465V x (1+3%) = 3.569V (the increment is less than 5% of the high error probability block), the clock frequency is increased to 45MHz x (1+8%) = 48.6MHz (the increment is less than 10% of the high error probability block), according to the adjusted burning voltage and clock frequency, the dedicated burning controller is used to sequentially execute the write operation on all fault-tolerant encoding data stream blocks, the data is written into the target device storage medium, and finally the device burning data stream block is generated, and the data burning process is completed, which ensures the data write success rate and improves the burning efficiency.

[0148] Further, step S4 comprises the following steps:

[0149] Step S41: performing block-by-block CRC check on the device burning data stream block to obtain device burning check success blocks and device burning check failure blocks;

[0150] In the embodiment of the present application, the block-by-block check on the device burning data stream block is performed by using a dedicated CRC check module, the module has a built-in CRC-32 algorithm, the polynomial is 0x04C11DB7, the check module reads each device burning data stream block from the device storage medium in sequence, 64KB data (consistent with the block size) is read each time, taking the block numbered as BLOCK_001 as an example, the check module first calculates the CRC value of the block, the module inputs the block data bit by bit into the CRC calculation circuit, the circuit performs shift and XOR operation according to the CRC-32 algorithm, and finally outputs a 32-bit CRC value, after the calculation is completed, the obtained CRC value is compared with the original CRC value recorded in the burning process, if the two are consistent, it is determined that the block check is successful, and is marked as a device burning check success block; if they are not consistent, it is determined that the check fails, and is marked as a device burning check failure block, the operation is sequentially performed on all blocks, and finally the check results are arranged into a list to clearly distinguish the check success and failure blocks, which provides a basis for subsequent processing.

[0151] Step S42: Obtain the corresponding block address and error type of the device burn check failed block through the device burn check failed block, and locate the error block corresponding to the device burn check failed block based on the block address and error type;

[0152] In the embodiment of the application, for the device burn check failed block, the corresponding block address is obtained through the address analysis module, the module directly accesses the page address mapping table of the storage medium, and the starting address and ending address of the failed block in the physical storage are extracted therefrom, for example, for the BLOCK_001 block that fails the check, the address analysis module determines that the physical storage address range thereof is 0x00000000 to 0x0000FFFF, and at the same time, the error type identification module is used to analyze the error characteristics of the failed block, the module judges the error type by checking the integrity, continuity and state of the redundant check bits of the block data and the like information. If it is found that there are continuous multiple bit errors in the data, and the check bits do not match the data, it is determined to be a burst error; if only individual bit errors exist, it is determined to be a random error. Combined with the block address and the error type, the specific error block is accurately located, for example, it is determined that the 32KB to 32.5KB region of the BLOCK_001 block has a burst error, which provides a clear target for subsequent repair work.

[0153] Step S43: Obtain the index label of the block in the fault-tolerant encoding data stream block based on the block address, and obtain the corresponding redundant information of the block in the fault-tolerant encoding data stream block based on the index label, and at the same time, the error block is repaired online using the corresponding redundant information of the block in the fault-tolerant encoding data stream block, so as to repair the error block using the redundant information by using the iterative decoding algorithm, if the repair fails, the re-burning process is triggered, otherwise, the iterative CRC check is performed until the check is successful;

[0154] In the embodiment of the present application, by means of the previously acquired block address, the corresponding index tag is searched in the fault-tolerant encoding data stream block by the index tag retrieval module, each fault-tolerant encoding data stream block is added with an index tag containing position information and redundant information when generated, and is stored in a special index table. For example, for the error block BLOCK_001, the corresponding index tag is found in the index table through its physical address, the redundant information of the block is acquired therefrom, including the encoding type (such as BCH encoding) adopted, the number of check bits, and the association relationship with other blocks, etc. By means of the acquired redundant information, the error block is repaired online by using an iterative decoding algorithm. Taking the BCH encoding as an example, the decoding algorithm first constructs a check equation according to the check bits and the original data, and then gradually corrects the error bits through iterative calculation. After each iteration, the CRC value is recalculated and compared with the original value. If they are consistent, the repair is successful. If they are not consistent after 10 iterations, it is determined that the repair fails, and the re-burning process is triggered. If the repair is successful, the repaired block is subjected to CRC check again, and the above process is repeated until the check is successful, so as to ensure the accuracy of the data.

[0155] Step S44: Based on the device state data, the device burning check success block corresponding to the CRC check success is subjected to burning verification analysis, so as to calculate the error code rate and the remaining erase life of each burning block corresponding to the device state data, and verify the error code rate and the remaining erase life of each burning block corresponding to the preset threshold value. If the verification fails, the repair and check are performed again, so as to generate a data stream burning verification completion result.

[0156] In the embodiment of the present application, the storage temperature, voltage fluctuation and erase times in the device state data are input into the error code rate calculation module. Based on the preset mathematical model, the error code rate of each burning block is calculated in combination with the device state data. For example, for the device burning check success block BLOCK_001, if the current storage temperature is 28℃, the voltage fluctuation range is 0.12V, and the erase times are 60 times, the model calculates that the error code rate is 0.001%. At the same time, the erase life assessment module calculates the remaining erase life of the storage medium according to the erase times data. Assuming that the total erase life of the storage medium is 1000 times, and the current average erase times are 60 times, the remaining erase life is (1000-60)÷1000×100%=94%. The calculated error code rate and the remaining erase life are compared with the preset threshold value (the error code rate threshold value is 0.005%, and the remaining erase life threshold value is 80%). If the error code rate and the remaining erase life of BLOCK_001 both meet the threshold value requirements, the verification is passed. If any one of them does not meet the requirements, the repair and check are performed again until all blocks pass the verification, and finally a data stream burning verification completion result is generated.

[0157] Further, the application also provides an intelligent data burning system based on data stream, which is used for executing the intelligent data burning method based on data stream as described above, and the intelligent data burning system based on data stream comprises:

[0158] A to-be-burned data stream description output module is configured to collect to-be-burned input source data in real time through a bus interface, and perform format recognition and semantic analysis on the to-be-burned input source data to generate metadata information corresponding to the to-be-burned data; obtain hardware storage structure information and burning protocol specification corresponding to a target device, and perform data stream description output on the metadata information corresponding to the to-be-burned data based on the hardware storage structure information and the burning protocol specification corresponding to the target device, so as to generate a data stream description file corresponding to the metadata, the structure information and the burning instruction;

[0159] A burning dynamic scheduling module is configured to obtain storage bandwidth and cache capacity corresponding to the target device, and perform burning dynamic task scheduling on the data stream description file corresponding to the metadata, the structure information and the burning instruction based on the storage bandwidth and the cache capacity corresponding to the target device, so as to generate a data stream burning task sequence;

[0160] A data stream block burning write module is configured to perform low-density redundancy coding block on the to-be-burned data stream corresponding to the data stream burning task sequence, so as to generate fault-tolerant coding data stream blocks; monitor device state data corresponding to the target device in real time, the device state data including storage temperature, voltage fluctuation and erasing times, and perform burning write processing on the fault-tolerant coding data stream blocks based on the device state data, so as to generate device burning data stream blocks;

[0161] A burning error online repair module is configured to perform block-by-block CRC check on the device burning data stream blocks, locate error blocks corresponding to the device burning data stream blocks, and perform online repair verification analysis on the error blocks by using corresponding redundancy information in the fault-tolerant coding data stream blocks and combining the device state data, so as to generate a data stream burning verification completion result.

[0162] The above is only a specific embodiment of the application, which enables those skilled in the art to understand or implement the application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent data burning method based on data stream, characterized in that: The following steps are involved: Step S1: collecting input source data to be burned in real time through the bus interface, and performing format recognition and semantic analysis on the input source data to be burned to generate metadata information corresponding to the data to be burned; Obtaining the hardware storage structure information and burning protocol specifications corresponding to the target device, and outputting a data flow description of the metadata information corresponding to the burning data based on the hardware storage structure information and burning protocol specifications corresponding to the target device, and generating a data flow description file containing the metadata, structure information and burning instructions; Step S2: Obtain the storage bandwidth and cache capacity corresponding to the target device, and perform dynamic burning task scheduling for the data stream description file containing metadata, structural information, and burning instructions based on the storage bandwidth and cache capacity corresponding to the target device to generate a data stream burning task sequence; wherein step S2 includes the following steps: Step S21: obtaining the storage bandwidth corresponding to the target device through a continuous write test; Step S22: Calculate and obtain the cache capacity corresponding to the target device according to the page address mapping table corresponding to the target device; Step S23: performing a burning dependency analysis on the data stream description file containing the metadata, the structural information, and the burning instructions, so as to mine and analyze the dependencies corresponding to the data streams to be burned in the data stream description file based on the burning instructions, and generate data burning dependencies between the data streams to be burned; Step S24: Based on the data burning dependency relationship between the data streams to be burned, a directed acyclic construction is performed between the data streams to be burned in the data stream description file to generate a directed acyclic graph of the data streams to be burned; and according to the directed acyclic graph of the data streams to be burned, a topological burning sorting is performed between the data streams to be burned in the data stream description file to generate an initial burning sorting sequence of the data streams; Step S25: performing dynamic burning task scheduling on the data stream initial burning sorting sequence based on the storage bandwidth and cache capacity corresponding to the target device to generate a data stream burning task sequence; Step S3: performing low-density redundant coding and segmenting on the corresponding data stream to be programmed in the data stream programming task sequence to generate fault-tolerant coded data stream segments; monitoring the device status data corresponding to the target device, including storage temperature, voltage fluctuation, and erase / write counts, in real time, and programming and writing the fault-tolerant coded data stream segments based on the device status data to generate device programming data stream segments; Step S4: Perform CRC check on each block of the device burning data stream block, and locate the error block corresponding to the device burning data stream block. At the same time, use the corresponding redundant information in the fault-tolerant coded data stream block and combine it with the device status data to perform online repair verification analysis on the error block to generate a data stream burning verification completion result.

2. The intelligent data burning method based on data stream according to claim 1, wherein Step S1 includes the following steps: Step S11: The bus interface configuration includes SPI, I2C and UART interface protocols to collect the corresponding input source data to be burned in real time; Step S12: Format identification and parsing of the input source data to be burned is performed, and a multi-mode data parser is designed to support automatic identification and parsing of various data formats including binary, text, XML, and JSON to obtain the input format parsed data to be burned; Step S13: performing data field semantic analysis on the input format parsed data to be burned, so as to generate metadata information corresponding to the data to be burned; Step S14: Obtaining hardware storage structure information and burning protocol specifications corresponding to the target device, wherein the hardware storage structure information includes flash memory block distribution, page address mapping table and bad block marking information; Step S15: outputting a data flow description of the metadata information corresponding to the data to be burned based on the hardware storage structure information corresponding to the target device and the burning protocol specification, and generating a data flow description file including the metadata, structure information and burning instructions.

3. The intelligent data burning method based on data stream according to claim 2, wherein Step S15 includes the following steps: Step S151: obtaining corresponding semantic entities of the data to be programmed through metadata information corresponding to the data to be programmed, and extracting entity relationships between the semantic entities of the data to be programmed to obtain data entity relationships between the entities of the data to be programmed; Step S152: constructing an entity relationship graph for each semantic entity of the data to be programmed based on the data entity relationship between each entity of the data to be programmed, so as to generate an entity relationship graph of the data to be programmed; Step S153: generating executable burning instructions for the entity relationship diagram of the data to be burned based on the burning protocol specification corresponding to the target device, so as to generate executable burning instructions for the target device corresponding to the data to be burned; Step S154: Based on the hardware storage structure information corresponding to the target device and the target device executable burning instruction corresponding to the data to be burned, a data flow description file containing metadata, structure information and burning instructions is generated.

4. The intelligent data burning method based on data stream according to claim 1, characterized in that: Step S25 includes the following steps: Step S251: Dividing the data stream initial burn-in sorting sequence into various data segments, obtaining the corresponding data size from each data segment, and constructing a data stream burn-in transmission topology network based on the corresponding data transmission order and dependency relationship of each data segment within the data stream initial burn-in sorting sequence and the corresponding data size of each data segment, wherein the data segments are used as nodes, the data size is used as a node attribute, and the data transmission order and dependency relationship are used as edges; Step S252: calculating the betweenness centrality corresponding to each node through the data stream burning transmission topology network, and determining the burning transmission priority of each data segment based on the betweenness centrality corresponding to each node to generate a data stream burning transmission priority sequence; Step S253: performing bandwidth burning and transmission pressure evaluation on the data segments corresponding to the data stream burning and transmission priority sequence based on the storage bandwidth corresponding to the target device, so as to obtain the storage bandwidth burning and transmission pressure corresponding to each data segment in the target device; Step S254: performing cache transmission saturation quantification on the data segments corresponding to the data stream burning transmission priority sequence based on the cache capacity corresponding to the target device, so as to obtain the cache burning transmission saturation rate corresponding to each data segment in the target device; Step S255: Analyze the current load of the target device based on the storage bandwidth burning transmission pressure and cache burning transmission saturation rate corresponding to each data segment in the target device to obtain the current burning load corresponding to the target device; dynamically adjust the burning transmission rate corresponding to each data segment according to the current burning load corresponding to the target device, and perform dynamic burning task scheduling on the data stream initial burning sorting sequence based on the burning transmission rate corresponding to each data segment to generate a data stream burning task sequence.

5. The intelligent data burning method based on data stream according to claim 1 is characterized in that, Step S3 includes the following steps: Step S31: performing low-density redundant coding and blocking on the corresponding data stream to be burned in the data stream burning task sequence to generate fault-tolerant coded data stream blocks; Step S32: monitoring the storage temperature corresponding to the target device in real time; Step S33: monitoring the voltage fluctuation corresponding to the target device in real time; Step S34: monitoring the number of erase and write times corresponding to the target device in real time; Step S35: The storage temperature, voltage fluctuation and erase / write times corresponding to the target device are used as device status data, and the fault-tolerant coded data stream blocks are burned and written based on the device status data to generate device burning data stream blocks.

6. The intelligent data burning method based on data stream according to claim 5, characterized in that: Step S31 includes the following steps: Perform block processing on the corresponding data stream to be burned in the data stream burning task sequence to obtain each data stream block to be burned; Obtaining the to-be-burned data content corresponding to each block through each to-be-burned data stream block, and grouping each to-be-burned data stream block based on the to-be-burned data content corresponding to each block to obtain key to-be-burned data blocks and non-key to-be-burned data blocks; Obtaining content correlation coefficients and erasure times of corresponding data blocks from the key data blocks to be programmed and the non-key data blocks to be programmed, and calculating an optimal number of check bits for the corresponding data blocks based on the content correlation coefficients and the erasure times, and simultaneously applying high redundancy encoding to the key data blocks to be programmed and low redundancy encoding to the non-key data blocks to be programmed according to the optimal number of check bits for the corresponding data blocks, so as to generate key coding blocks to be programmed and non-key coding blocks to be programmed; An index tag containing position information and redundant information is added to each key coding block to be burned and a non-key coding block to be burned, so as to generate a fault-tolerant coding data stream block.

7. The intelligent data burning method based on data stream according to claim 5, characterized in that: The process of burning and writing the fault-tolerant coded data stream blocks based on the device status data in step S35 includes the following steps: Obtain the historical device programming status data and train a Bayesian network model based on the historical device programming status data to predict the write error probability of the target device corresponding to the data stream block; By inputting the current corresponding device status data into the Bayesian network model, the write prediction analysis of the target device is performed on the fault-tolerant coded data stream block to obtain the write error probability corresponding to each fault-tolerant coded data stream block; Based on the write error probability corresponding to each fault-tolerant coded data stream block, the corresponding fault-tolerant coded data stream block is burned and written. If it is determined to be a high error probability block according to the write error probability, the burning voltage is increased, the increase is less than or equal to 5% of the standard burning voltage, and the clock frequency is reduced, and the reduction is less than or equal to 10% of the standard clock frequency; if it is determined to be a low error probability block according to the write error probability, the burning voltage is increased, the increase is less than 5% of the high error probability block, and the clock frequency is increased, the increase is less than 10% of the high error probability block, thereby executing the write operation of each fault-tolerant coded data stream block to generate a device burning data stream block.

8. The intelligent data burning method based on data stream according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing CRC check on each block of the device burning data stream to obtain blocks where the device burning check succeeds and blocks where the device burning check fails; Step S42: Obtain the corresponding block address and error type through the block that failed device burning verification, and locate the error block corresponding to the block that failed device burning verification based on the block address and error type; Step S43: obtaining an index tag in the fault-tolerant coded data stream block based on the block address, and obtaining corresponding redundant information in the fault-tolerant coded data stream block based on the index tag. At the same time, online repair of the erroneous block is performed using the corresponding redundant information in the fault-tolerant coded data stream block, so as to use the redundant information to repair the erroneous block using an iterative decoding algorithm. If the repair fails, a re-burning process is triggered. Otherwise, an iterative CRC check is performed until the check succeeds. Step S44: Based on the device status data, the device burning verification and successful blocks corresponding to the CRC verification are analyzed to calculate the bit error rate and the remaining erase life of each burning block according to the device status data, and verify the bit error rate and the remaining erase life of each burning block according to the preset threshold. If the verification fails, the verification is repaired again to generate a data stream burning verification completion result.

9. An intelligent data burning system based on data stream, characterized in that: For executing the data stream-based intelligent data burning method according to claim 1, the data stream-based intelligent data burning system comprises: The module for outputting a data stream description to be programmed is used to collect the input source data to be programmed in real time through the bus interface, perform format recognition and semantic analysis on the input source data to be programmed, and generate metadata information corresponding to the data to be programmed; obtain the hardware storage structure information corresponding to the target device and the programming protocol specification, and output a data stream description of the metadata information corresponding to the data to be programmed based on the hardware storage structure information corresponding to the target device and the programming protocol specification, thereby generating a data stream description file containing the metadata, structure information, and programming instructions; The dynamic programming scheduling module is used to obtain the storage bandwidth and cache capacity corresponding to the target device, and perform dynamic programming task scheduling on the data stream description file containing metadata, structural information and programming instructions based on the storage bandwidth and cache capacity corresponding to the target device, thereby generating a data stream programming task sequence; The data stream block burning and writing module is used to perform low-density redundant coding and blocking on the corresponding data stream to be burned in the data stream burning task sequence to generate fault-tolerant coded data stream blocks; by real-time monitoring of the target device's corresponding device status data including storage temperature, voltage fluctuation and erase and write count, and burning and writing the fault-tolerant coded data stream blocks based on the device status data, to generate device burning data stream blocks; The online repair module for programming errors is used to perform CRC check on each block of the device programming data stream, locate the error block corresponding to the device programming data stream block, and use the corresponding redundant information in the fault-tolerant coded data stream block and combine it with the device status data to perform online repair verification analysis on the error block to generate a data stream programming verification completion result.

Images