Adaptive parsing and fault-tolerant processing system and method for STDF files

Abstract

This invention provides an adaptive parsing and fault-tolerant processing system and method for STDF files, belonging to the field of semiconductor integrated circuit test data processing technology. The system includes: a streaming splitting engine module for reading the raw binary data stream and decoupling it into control records and data records; a global context state machine module for establishing a mapping relationship between test item numbers and test names when parsing TSRs; a physical offset mapping index module for associating the chip's unique identifier with the start and end offsets of all records belonging to that chip in real time during the parsing process; a heterogeneous parallel scheduling module for cloning a current context snapshot of the corresponding task package when distributing each task package, and appending the context snapshot as header information to the data block associated with the task package; and a nonlinear fault-tolerant resynchronization module for repositioning the file pointer to the starting position of the next chip when an anomaly is detected.

Description

Technical Field

[0001] This invention relates to the field of semiconductor integrated circuit test data processing technology, and in particular to an adaptive parsing and fault-tolerant processing system and method for STDF files. Background Technology

[0002] In the testing phase of semiconductor manufacturing, test results generated by automated test equipment (ATE) are typically stored as binary Standard Test Data Format (STDF) files.

[0003] Existing parsing techniques primarily employ linear scanning logic, meaning the parser starts from the file header and reads and decodes each record sequentially according to the protocol-defined record sequence. The principle is as follows: a single thread reads the 4-byte record header, identifies the record type, performs binary deserialization based on the protocol-defined field structure, and writes the result to memory or a database.

[0004] The applicant discovered that existing parsing techniques have the following problems when processing large-scale STDF files: 1. Low parsing efficiency: With the surge in test data volume, single STDF files often reach several gigabytes or more. Due to the contextual coupling between STDF records, such as the PTR record depending on the test item name of the front-end TSR record, current technology generally considers that it must be parsed serially, which cannot utilize the parallel computing capabilities of multi-core CPUs; 2. Lack of physical retrieval capability: STDF is a streaming structure and does not support random access. If only data from a specific chip needs to be retrieved, the parser still needs to traverse the file from the beginning, generating a large amount of redundant disk I / O; 3. Poor fault tolerance and robustness: Binary files are prone to corruption during transmission, and existing parsers rely on strict sequential alignment. Once an unaligned record header is encountered, the entire parsing process often crashes due to the anomaly, resulting in the inability to extract all valid data after the damaged point. Summary of the Invention

[0005] This invention provides an adaptive parsing and fault-tolerant processing system and method for STDF files, addressing the problems of low parsing efficiency, inability to perform non-linear searches, and global parsing failure due to local data corruption when processing large-scale STDF files in existing parsing technologies. The technical solution is as follows: In a first aspect, embodiments of the present invention provide an adaptive parsing and fault-tolerant processing system for STDF files, comprising: The streaming splitting engine module is used to read the raw binary data stream, decouple the raw binary data stream into control records and data records, and calculate the absolute physical offset of the current record on the disk in real time; wherein the control record includes at least the Master Information Record (MIR) and the Test Summary Record (TSR), and the data record includes at least the Chip Start Record (PIR) and the Parameter Test Result (PTR). The global context state machine module is used to establish at least a mapping relationship between the test item number TEST_NUM and the test name TEST_NAM when the TSR is parsed. The physical offset mapping index module is used to associate the chip's unique identifier Part ID with the start offset and end offset of all records to which the chip belongs in real time during the parsing process. The heterogeneous parallel scheduling module is used to determine the number of concurrent decoding units based on the current computing environment resources, and to divide the original binary data stream into multiple task packages, wherein the size of the data block chunk associated with each task package can be dynamically adjusted, and to clone a current context snapshot corresponding to the task package when distributing each task package, and to attach the context snapshot as header information to the chunk associated with the task package. The nonlinear fault-tolerant resynchronization module is used to retrieve the Start Offset of the next chip from the physical offset mapping index module when an abnormal situation is detected, and reposition the file pointer to the Start Offset position of the next chip.

[0006] Optionally, the streaming splitting engine module reads the raw binary data stream, including: The streaming splitting engine module performs high-speed pre-scanning of the original binary data stream through a predefined double buffering mechanism.

[0007] Optionally, the global context state machine module establishes at least a mapping relationship between the test item number TEST_NUM and the test name TEST_NAM, including: The global context state machine module dynamically maintains a global mapping matrix, which contains the correspondence between test item number TEST_NUM and test name TEST_NAM.

[0008] Optionally, the global context state machine module is also used to provide timestamps.

[0009] Optionally, the physical offset mapping index module uses the high-performance relational storage engine SQLite to store the association between the Part ID and the Start Offset and End Offset.

[0010] Optionally, the physical offset mapping index module adopts an index B+ tree structure, which optimizes sequential file search into O(log n) retrieval, providing a non-linear random access interface for upper-layer applications.

[0011] Optionally, the heterogeneous parallel scheduling module includes a resource environment awareness submodule, a dynamic sharding submodule, and a snapshot mapping injection submodule; The resource environment awareness submodule is pre-configured with a resource occupancy assessment model. The resource environment awareness submodule is used to determine the number of concurrent decoding units that can be activated based on the current computing environment resources and the resource occupancy assessment model. The current computing environment resources include at least one of the following: CPU core load, L2 / L3 cache hit rate, and available physical memory striped bandwidth. The dynamic sharding submodule is used to divide the original binary data stream into multiple task packets according to the current CPU throughput. The multiple task packets are of different, equal, or partially equal sizes, and the size of a single chunk is dynamically adjusted according to the memory availability. The snapshot mapping injection submodule is used to clone a current context snapshot corresponding to each task package when distributing each task package, and to attach the context snapshot as header information to the Chunk associated with the task package.

[0012] Secondly, embodiments of the present invention provide an adaptive parsing and fault-tolerant processing method for STDF files, applied to the adaptive parsing and fault-tolerant processing system for STDF files described in the first aspect, wherein the adaptive parsing and fault-tolerant processing method for STDF files includes: The streaming splitting engine module reads the raw binary data stream, decouples the raw binary data stream into control records and data records, and calculates the absolute physical offset of the current record on the disk in real time; wherein the control record includes at least the Master Information Record (MIR) and the Test Summary Record (TSR), and the data record includes at least the Chip Start Record (PIR) and the Parameter Test Result (PTR). The global context state machine module parses all control records. When the TSR is parsed, at least a mapping relationship is established between the test item number TEST_NUM and the test name TEST_NAM. When the first PIR is detected, the physical offset mapping index module captures the starting offset of the current pointer and associates the chip's unique identifier, Part ID, with the starting offset; For the large number of PTRs generated subsequently, the heterogeneous parallel scheduling module divides them into data blocks of a preset size, clones the context snapshot containing the association between TEST_NUM and TEST_NAM, and encapsulates the context snapshot into a task package and sends it to the concurrent decoding unit. The concurrent decoding unit restores the binary result to the physical quantity value based on the context snapshot.

[0013] Optionally, the method further includes: When an anomaly is detected during the parsing process, the nonlinear fault-tolerant resynchronization module retrieves the start offset of the next chip from the physical offset mapping index module and forcibly repositions the file pointer to the start offset position of the next chip.

[0014] Optionally, the method further includes: After the adaptive parsing and fault-tolerant processing system for STDF files is started, the number of concurrent decoding units is determined based on the current system memory size.

[0015] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the adaptive parsing and fault-tolerant processing method for STDF files described in the second aspect.

[0016] Fourthly, embodiments of the present invention provide a computer program product including computer instructions, which, when executed by a processor, implement the steps of the adaptive parsing and fault-tolerant processing method for STDF files as described in the second aspect.

[0017] The beneficial effects of the above-mentioned technical solution of the present invention are: In the adaptive parsing and fault-tolerant processing system and method for STDF files provided in this embodiment of the invention, the streaming splitting engine module is used to read the original binary data stream, decouple the original binary data stream into control records and data records, and calculate the offset (absolute physical offset) of the current record on the disk in real time; wherein the control record includes at least MIR (Master Information Record) and TSR (Test Synopsis Record), and the data record includes at least PIR (Periodic Interval Recording) and PTR (Parametric Test Result); the global context state machine module is used to establish at least a mapping relationship between TEST_NUM (test item number) and TEST_NAM (test name) when parsing the TSR; the physical offset mapping index module is used to real time match the Part ID (chip unique identifier) ​​with the Start Offset and End Offset of all records to which the chip belongs during the parsing process. The offset is associated with the task; the heterogeneous parallel scheduling module is used to determine the number of concurrent decoding units based on the current computing environment resources, and to divide the original binary data stream into multiple task packages, the size of the chunk associated with each task package can be dynamically adjusted, and to clone a current context snapshot corresponding to the task package when distributing each task package, and to attach the context snapshot as header information to the chunk associated with the task package; the nonlinear fault-tolerant resynchronization module is used to retrieve the start offset of the next chip from the physical offset mapping index module when an abnormal situation is detected, and to reposition the file pointer to the start offset position of the next chip.

[0018] In this embodiment of the invention, the heterogeneous parallel scheduling module uses snapshot injection technology to enable each parallel worker process to hold a complete decoding dictionary. It can independently complete the physical quantity conversion without communicating with the main thread, solving the industry problem of STDF parallelization. This allows the parsing speed to scale linearly with the number of CPU cores, and significantly improves the efficiency of processing GB-level files.

[0019] In this embodiment of the invention, the physical offset mapping index module will associate the Part ID with the Start Offset and End Offset of all records (from PIR to PRR) belonging to the chip in real time during the parsing process, so that the system has the ability to transform from streaming storage to random access, and the secondary query latency is greatly reduced.

[0020] In this embodiment of the invention, when an abnormal situation is detected, the nonlinear fault-tolerant resynchronization module retrieves the start offset of the next chip from the physical offset mapping index module and forcibly repositions the file pointer to the start offset of the next chip. Through this "overcoming the damage point" strategy, the system can automatically restore the parsing process, ensure the integrity of the extraction of subsequent undamaged data, and recover valid test data to the maximum extent. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of an adaptive parsing and fault-tolerant processing system for STDF files disclosed in an embodiment of the present invention; Figure 2 This is a flowchart of an adaptive parsing and fault-tolerant processing method for STDF files disclosed in an embodiment of the present invention. Detailed Implementation

[0022] To make the technical problems, technical solutions, and advantages of this invention clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as particular configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of this invention. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this invention. Furthermore, for clarity and brevity, descriptions of known functions and structures have been omitted.

[0023] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0024] In various embodiments of the present invention, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0025] This invention provides an adaptive parsing and fault-tolerant processing system for STDF files (hereinafter referred to as the system). This system operates based on computer hardware resources and includes a streaming splitting engine module, a global context state machine module, a physical offset mapping index module, a heterogeneous parallel scheduling module, and a nonlinear fault-tolerant resynchronization module. The modules are logically interconnected through an internal bus and data interface.

[0026] like Figure 1As shown, the adaptive parsing and fault-tolerant processing system for STDF files provided in this embodiment of the invention includes: The streaming splitting engine module 10 is used to read the raw binary data stream, decouple the raw binary data stream into control records and data records, and calculate the offset of the current record on the disk in real time; wherein the control record includes at least MIR and TSR, and the data record includes at least PIR and PTR.

[0027] The raw binary data stream is the raw STDF file. Control records may also include SDR (Site Description Record), and data records may include MPR (Multi-Parameter Test Result) and FTR (Functional Test Result). In this embodiment of the invention, the streaming splitting engine module 10 performs high-speed pre-scanning of the raw binary data stream through a predefined double buffering mechanism.

[0028] The core of this scanning process is to identify the 4-byte record header in the STDF protocol, extract the record type code (REC_TYP) and record subtype code (REC_SUB), and calculate the absolute physical offset of the current record on the disk in real time. The streaming splitting engine module 10 decouples the file stream (i.e., the raw binary data stream) into control records and data records. The control records are responsible for transmitting metadata information, while the data records are responsible for sorting computationally intensive records.

[0029] The global context state machine module 20 is used to establish at least a mapping relationship between the test item number TEST_NUM and the test name TEST_NAM when the TSR is parsed.

[0030] In this embodiment of the invention, the global context state machine module 20 is specifically responsible for synchronously processing control records, such as MIR, SDR, and TSR. Whenever the global context state machine module 20 parses a TSR, it triggers the establishment of a mapping relationship between TEST_NUM and TEST_NAM. In practical applications, the global context state machine module 20 can dynamically maintain a global mapping matrix, using this global mapping matrix to establish and store the mapping relationship between TEST_NUM and TEST_NAM. The mapping relationship can also include business attributes such as units and limits.

[0031] In addition, the global context state machine module 20 is also used to provide timestamps, that is, it has a "timestamp marking" function, which can lock the logical relationship at a certain moment according to the current parsing progress, and provide a static and consistent logical view for asynchronous computing.

[0032] The physical offset mapping index module 30 is used to associate the Part ID with the Start Offset and End Offset of all records belonging to the chip in real time during the parsing process.

[0033] In this embodiment of the invention, the physical offset mapping index module 30 can utilize the high-performance relational storage engine SQLite to store the association between the Part ID and the Start Offset and End Offset. The physical offset mapping index module 30 constructs an index synchronously in real time during the parsing process, associating the chip's unique identifier (Part ID) with the start offset and end offset of all records (from PIR to PRR) belonging to that chip on the disk. Furthermore, the physical offset mapping index module 30 adopts a B+ tree index structure, optimizing sequential file search into O(log n) retrieval, providing a non-linear random access interface for upper-layer applications, enabling the system to transform from streaming storage to random access, and significantly reducing secondary query latency.

[0034] The heterogeneous parallel scheduling module 40 is used to determine the number of concurrent decoding units based on the current computing environment resources, and to divide the original binary data stream into multiple task packages, wherein the size of the data block chunk associated with each task package can be dynamically adjusted, and to clone a current context snapshot corresponding to the task package when distributing each task package, and to attach the context snapshot as header information to the chunk associated with the task package.

[0035] It should be noted that the adjustment of the Chunk size in this embodiment of the invention is based on the premise of not destroying the physical integrity of the STDF record entity.

[0036] The heterogeneous parallel scheduling module 40 is the control center of this system. It integrates a "resource awareness engine" and a "snapshot injection controller". Specifically, it can include a resource environment awareness submodule, a dynamic sharding submodule, and a snapshot mapping injection submodule.

[0037] The resource environment awareness submodule pre-configures a resource occupancy assessment model. This submodule determines the number of concurrent decoding units that can be activated based on the current computing environment resources using this model. The current computing environment resources may include at least one of the following: CPU core load, L2 / L3 cache hit rate, and available physical memory striped bandwidth. In this embodiment, the pre-configured resource occupancy assessment model can adjust the dynamic load factor of concurrent decoding units in real time through multi-dimensional trade-off calculations of the current computing environment resources, thereby achieving a dynamic balance between computational throughput and I / O stability. In practical applications, the resource environment awareness submodule monitors CPU core load, L2 / L3 cache hit rate, and available physical memory striped bandwidth through underlying interfaces. By calling the resource occupancy assessment model, it dynamically determines the number of parallel workers to be activated, ensuring that I / O blocking is not caused by multi-process contention.

[0038] The dynamic sharding submodule is used to divide the raw binary data stream into multiple task packets based on the current CPU throughput. These task packets can be of unequal, equal, or partially equal sizes. It also dynamically adjusts the size of individual chunks based on available memory. It is important to note that chunk size adjustments are made without compromising the physical integrity of the STDF record entities. In practical applications, the dynamic sharding submodule dynamically adjusts the size of individual chunks based on available memory. Furthermore, when processing massive amounts of PTR data, it divides the raw binary data stream into multiple task packets of unequal, equal, or partially equal sizes based on the current CPU throughput, achieving load balancing of computational pressure.

[0039] The snapshot mapping injection submodule is used to clone a current context snapshot corresponding to each task package when distributing it, and append the context snapshot as header information to the chunk associated with the task package. In this way, each concurrent decoding unit can independently parse based on the context snapshot within the package and push the results directly to the backend storage.

[0040] This is a key technology for resolving protocol coupling in this embodiment of the invention. In practical applications, when distributing each task package, the snapshot mapping injection submodule clones a copy of the current "state machine image" (i.e., a context snapshot) and appends it as header information to the binary data block. The snapshot mapping injection submodule extracts the current context snapshot from the state machine (e.g., Test 105 corresponds to the current test) and packages it with the binary slice. The task package is delivered to the heterogeneous parallel decoding pool, and each concurrent decoding unit independently parses it based on the context snapshot within the package and pushes the results directly to the backend storage. In this embodiment of the invention, each parallel worker process holds a complete decoding dictionary and can independently complete the physical quantity conversion without communicating with the main thread, completely solving the performance degradation caused by Python GIL locks or inter-thread synchronization.

[0041] The nonlinear fault-tolerant resynchronization module 50 is used to retrieve the start offset of the next chip from the physical offset mapping index module when an abnormal situation is detected, and reposition the file pointer to the start offset position of the next chip.

[0042] In this embodiment of the invention, the nonlinear fault-tolerant resynchronization module 50 is responsible for intervening in runtime anomalies. When a child process reports an anomaly, such as a data block verification error or a lost record header alignment, the nonlinear fault-tolerant resynchronization module 50 will capture the anomaly in real time and intervene immediately. The nonlinear fault-tolerant resynchronization module 50 will no longer attempt to find the record header in the damaged area, but will instead retrieve the nearest valid anchor point position from the physical offset mapping index module 30 and drive the scheduling engine to execute a hardware-level seek instruction.

[0043] It should be noted that this system uses a "master-slave" architecture. The master process is responsible for traffic distribution, global state, and resource awareness, while the child processes are responsible for executing heterogeneous parallel decoding of data logic.

[0044] For example, during the parsing process, if a segment of binary data cannot be recognized by the parser due to disk bad blocks (such as an abnormal length identifier), the non-linear fault-tolerant resynchronization module 50 will capture this anomaly in real time. At this point, the system stops the current corrupted task, and the non-linear fault-tolerant resynchronization module 50 queries the physical offset mapping index module 30 for the starting position of the next chip, forcibly repositioning the file pointer to the next PIR record using the physical offset address. Through this "overcoming the corruption point" strategy, the system can automatically resume the parsing process, ensuring the integrity of the extraction of subsequent undamaged data.

[0045] The adaptive parsing and fault-tolerant processing system for STDF files provided in this embodiment of the invention utilizes a heterogeneous parallel scheduling module 40. Through snapshot injection technology, each parallel worker process holds a complete decoding dictionary, enabling independent physical quantity conversion without communication with the main thread. This solves the industry-wide challenge of STDF parallelization, allowing parsing speed to scale linearly with the number of CPU cores, significantly improving efficiency in processing GB-level files. The physical offset mapping index module 30, during parsing, associates the Part ID with the StartOffset and EndOffset of all records (from PIR to PRR) belonging to that chip in real time on the disk. This enables the system to transition from streaming storage to random access, significantly reducing secondary query latency. When an anomaly is detected, the nonlinear fault-tolerant resynchronization module 50 retrieves the StartOffset of the next chip from the physical offset mapping index module 30 and forcibly repositions the file pointer to the StartOffset of that next chip. Through this "overcoming the damage point" strategy, the system can automatically recover the parsing process, ensuring the integrity of subsequent undamaged data extraction and maximizing the recovery of valid test data.

[0046] Based on the adaptive parsing and fault-tolerant processing system for STDF files provided in the foregoing embodiments of the present invention, this embodiment also provides an adaptive parsing and fault-tolerant processing method for STDF files. This method is applied to the aforementioned adaptive parsing and fault-tolerant processing system for STDF files. For example... Figure 2 As shown, the method includes: Step 101: The streaming splitting engine module reads the raw binary data stream, decouples the raw binary data stream into control records and data records, and calculates the absolute physical offset of the current record on the disk in real time; wherein the control record includes at least the Master Information Record (MIR) and the Test Summary Record (TSR), and the data record includes at least the Chip Start Record (PIR) and the Parameter Test Result (PTR).

[0047] In practical applications, the streaming splitting engine module starts its work from the file header. For each record read, the physical offset mapping index module updates and records the current physical address.

[0048] Step 102: The global context state machine module parses all control records. When the TSR is parsed, at least the mapping relationship between the test item number TEST_NUM and the test name TEST_NAM is established.

[0049] For example, when the global context state machine module parses a TSR record, it will establish and record the mapping relationship between Test_Num:105 and Test_Name: Vcc_Current.

[0050] Step 103: When the first PIR is identified, the physical offset mapping index module captures the starting offset of the current pointer and associates the chip's unique identifier Part ID with the starting offset.

[0051] When the engine scans the first PIR, the physical offset mapping index module captures the current pointer position StartOffset, for example, 0x005A00, and associates it with Part_ID: 1. This association is then stored in SQLite.

[0052] Step 104: For the large number of PTRs generated subsequently, the heterogeneous parallel scheduling module divides them into data blocks of a preset size, clones the context snapshot containing the association between TEST_NUM and TEST_NAM, and encapsulates the context snapshot into a task package and sends it to the concurrent decoding unit.

[0053] In practical applications, the number of PTRs generated subsequently may reach millions. The heterogeneous parallel scheduling module divides these into data blocks, for example, 50MB blocks. Simultaneously, the heterogeneous parallel scheduling module clones a context snapshot containing the definition of 105:Vcc_Current and encapsulates it into a task packet, sending it to the concurrent decoding unit. The concurrent decoding unit directly restores the binary results to physical quantity values ​​based on the context snapshot.

[0054] Step 105: The concurrent decoding unit restores the binary result to the physical quantity value based on the context snapshot.

[0055] More preferably, the adaptive parsing and fault-tolerant processing method for STDF files provided in this embodiment of the invention may further include: When an anomaly is detected during the parsing process, the nonlinear fault-tolerant resynchronization module retrieves the start offset of the next chip from the physical offset mapping index module and forcibly repositions the file pointer to the start offset position of the next chip.

[0056] Further assuming that the file loses 4 bytes of data due to interference at the 5.5GB mark, causing an error in the concurrent decoding unit, the non-linear fault-tolerant resynchronization module receives the interrupt signal and immediately retrieves the starting coordinates 0x560000 of the next chip's Part_ID: 1001 from the physical offset mapping index module. Then, the scheduling engine drives the file stream to jump directly to position 0x560000, skipping the corrupted binary segment and realigning the record header. This ensures that the parsing of the subsequent 4.5GB of test data is not affected by the front-end corruption, and all remaining yield metrics for this batch are fully extracted.

[0057] More preferably, the adaptive parsing and fault-tolerant processing method for STDF files provided in this embodiment of the invention may further include: After the adaptive parsing and fault-tolerant processing system for STDF files is started, the number of concurrent decoding units is determined based on the current system memory size.

[0058] When the system starts up in this embodiment of the invention, the heterogeneous parallel scheduling module can first call the kernel interface to obtain the current computing environment resources. If the system memory is sufficient (e.g., greater than 16GB), a larger task buffer is configured and the maximum number of computing units are enabled; if memory is limited, it automatically switches to the "small steps, fast runs" mode, reducing the memory footprint of a single task by shortening the sharding cycle.

[0059] For example, an 8-core CPU can be used as an example in this embodiment of the invention. The heterogeneous parallel scheduling module presets 8 concurrent decoding units and sets the task buffer queue depth according to the remaining 16GB of memory.

[0060] The adaptive parsing and fault-tolerant processing system and method for STDF files provided in this invention have at least the following advantages: 1. A significant leap in parsing performance: The snapshot injection technology solves the industry challenge of STDF parallelization, enabling parsing speed to scale linearly with the number of CPU cores, resulting in a significant improvement in the efficiency of processing GB-level files; 2. Enables second-level random access: The physical indexing mechanism enables the system to transform from streaming storage to random access, significantly reducing the latency of secondary queries; 3. Enhanced industrial-grade robustness: The introduced fault-tolerant resynchronization logic solves the pain point of parsing interruption caused by partial damage to binary files, and maximizes the recovery of effective test data; 4. Optimization of hardware resource utilization: The adaptive scheduling algorithm effectively balances throughput and system stability, and effectively prevents memory overflow or system hangs under large-scale concurrency.

[0061] This invention also provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the methods described in the preceding embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0062] This invention also provides a computer program product, including computer instructions. When these computer instructions are executed by a processor, they implement the various processes of the method embodiments described above and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0063] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This is something that those skilled in the art can achieve using their basic programming skills after reading the description of the present invention.

[0064] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0065] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An adaptive parsing and fault-tolerant processing system for STDF files, characterized in that, include: The streaming splitting engine module is used to read the raw binary data stream, decouple the raw binary data stream into control records and data records, and calculate the absolute physical offset of the current record on the disk in real time; wherein the control record includes at least the Master Information Record (MIR) and the Test Summary Record (TSR), and the data record includes at least the Chip Start Record (PIR) and the Parameter Test Result (PTR). The global context state machine module is used to establish at least a mapping relationship between the test item number TEST_NUM and the test name TEST_NAM when the TSR is parsed. The physical offset mapping index module is used to associate the chip's unique identifier Part ID with the start offset and end offset of all records to which the chip belongs in real time during the parsing process. The heterogeneous parallel scheduling module is used to determine the number of concurrent decoding units based on the current computing environment resources, and to divide the original binary data stream into multiple task packages, wherein the size of the data block chunk associated with each task package can be dynamically adjusted, and to clone a current context snapshot corresponding to the task package when distributing each task package, and to attach the context snapshot as header information to the chunk associated with the task package. The nonlinear fault-tolerant resynchronization module is used to retrieve the Start Offset of the next chip from the physical offset mapping index module when an abnormal situation is detected, and reposition the file pointer to the Start Offset position of the next chip.

2. The adaptive parsing and fault-tolerant processing system for STDF files according to claim 1, characterized in that, The streaming splitting engine module reads the raw binary data stream, including: The streaming splitting engine module performs high-speed pre-scanning of the original binary data stream through a predefined double buffering mechanism.

3. The adaptive parsing and fault-tolerant processing system for STDF files according to claim 1, characterized in that, The global context state machine module establishes at least the mapping relationship between the test item number TEST_NUM and the test name TEST_NAM, including: The global context state machine module dynamically maintains a global mapping matrix, which contains the correspondence between test item number TEST_NUM and test name TEST_NAM.

4. The adaptive parsing and fault-tolerant processing system for STDF files according to claim 3, characterized in that, The global context state machine module is also used to provide timestamps.

5. The adaptive parsing and fault-tolerant processing system for STDF files according to claim 1, characterized in that, The physical offset mapping index module uses the high-performance relational storage engine SQLite to store the association between the Part ID and the StartOffset and EndOffset.

6. The adaptive parsing and fault-tolerant processing system for STDF files according to claim 5, characterized in that, The physical offset mapping index module adopts an index B+ tree structure, which optimizes sequential file search into O(log n) retrieval, providing a non-linear random access interface for upper-layer applications.

7. The adaptive parsing and fault-tolerant processing system for STDF files according to claim 1, characterized in that, The heterogeneous parallel scheduling module includes a resource environment awareness submodule, a dynamic sharding submodule, and a snapshot mapping injection submodule; The resource environment awareness submodule is pre-configured with a resource occupancy assessment model. The resource environment awareness submodule is used to determine the number of concurrent decoding units that can be activated based on the current computing environment resources and the resource occupancy assessment model. The current computing environment resources include at least one of the following: CPU core load, L2 / L3 cache hit rate, and available physical memory striped bandwidth. The dynamic sharding submodule is used to divide the original binary data stream into multiple task packets according to the current CPU throughput. The multiple task packets are of different, equal, or partially equal sizes, and the size of a single chunk is dynamically adjusted according to the memory availability. The snapshot mapping injection submodule is used to clone a current context snapshot corresponding to each task package when distributing each task package, and to attach the context snapshot as header information to the Chunk associated with the task package.

8. An adaptive parsing and fault-tolerant processing method for STDF files, characterized in that, The adaptive parsing and fault-tolerant processing method for STDF files, applied to any one of claims 1-7, comprises: The streaming splitting engine module reads the raw binary data stream, decouples the raw binary data stream into control records and data records, and calculates the absolute physical offset of the current record on the disk in real time; wherein the control record includes at least the Master Information Record (MIR) and the Test Summary Record (TSR), and the data record includes at least the Chip Start Record (PIR) and the Parameter Test Result (PTR). The global context state machine module parses all control records. When the TSR is parsed, at least a mapping relationship is established between the test item number TEST_NUM and the test name TEST_NAM. When the first PIR is detected, the physical offset mapping index module captures the starting offset of the current pointer, StartOffset, and associates the chip's unique identifier, Part ID, with the StartOffset; For the large number of PTRs generated subsequently, the heterogeneous parallel scheduling module divides them into data blocks of a preset size, clones the context snapshot containing the association between TEST_NUM and TEST_NAM, and encapsulates the context snapshot into a task package and sends it to the concurrent decoding unit. The concurrent decoding unit restores the binary result to the physical quantity value based on the context snapshot.

9. The adaptive parsing and fault-tolerant processing method for STDF files according to claim 8, characterized in that, The method further includes: When an anomaly is detected during the parsing process, the nonlinear fault-tolerant resynchronization module retrieves the start offset of the next chip from the physical offset mapping index module and forcibly repositions the file pointer to the start offset position of the next chip.

10. The adaptive parsing and fault-tolerant processing method for STDF files according to claim 8 or 9, characterized in that, The method further includes: After the adaptive parsing and fault-tolerant processing system for STDF files is started, the number of concurrent decoding units is determined based on the current system memory size.

Images