Method and apparatus for generating semiconductor product yield indicator

Abstract

To provide a computer program, a machine-readable storage medium, a method and an apparatus for generating a semiconductor product yield indicator.SOLUTION: In a test environment 100, a human-readable test data generator 110 of automatic test equipment 104 includes a test instance trace encoder that encodes a data string representing a sequence of test results corresponding to multiple test instances performed by the automatic test equipment 104 on a test target device 102, and an enhanced test report generator 216 outputs the data string to a workstation 106.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates generally to semiconductor testing, and more particularly to methods and apparatus for generating semiconductor product yield indicators. [Background technology]

[0002] In semiconductor manufacturing, yield analysis is performed to verify the percentage of functional units produced during the semiconductor manufacturing or fabrication process. Analysis of yield data helps identify and address problems that may result in defects or failures in semiconductor devices. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of an exemplary environment in which an exemplary human-readable test data generator and a human-readable test data analyzer operate to generate semiconductor product yield indicators.

[0004] [Figure 2] FIG. 2 is a block diagram of an example implementation of the human-readable test data generator of FIG. 1.

[0005] [Figure 3] 1 illustrates an example test instance arranged in a directed test flow graph.

[0006] [Figure 4] Shown is an example of two devices under test being subjected to two different test paths.

[0007] [Figure 5] 1 illustrates an exemplary test path with an exemplary test path descriptor.

[0008] [Figure 6]1 illustrates an exemplary hierarchical memory built-in self-test (MBIST) implementation that performs parallel testing of multiple memory instances within a single test instance.

[0009] [Figure 7] An exemplary HRY_RAWSTR data string for the MBIST of FIG. 6 is shown.

[0010] [Figure 8] FIG. 2 is a block diagram of an example implementation of the human-readable test data analysis device of FIG. 1.

[0011] [Figure 9] 1 illustrates an exemplary hierarchical tree.

[0012] [Figure 10] 1 illustrates an exemplary human readable yield (HRY) hierarchical tree indicator structure.

[0013] [Figure 11] 1 illustrates exemplary user-defined levels in a hierarchical tree.

[0014] [Figure 12] 1 shows an example HRY hierarchical tree that combines multiple design for test (DFT) features.

[0015] [Figure 13] 1 illustrates an exemplary application of the HRY framework.

[0016] [Figure 14] 10 shows an exemplary graphical comparison of combined indicators with overall die yield results.

[0017] [Figure 15] 1 shows exemplary bin categories.

[0018] [Figure 16] 1 illustrates an exemplary semiconductor manufacturing process.

[0019] [Figure 17] 10 is a flowchart representing example machine-readable instructions and / or example operations that may be executed, instantiated, and / or performed by example programmable circuitry to implement the human-readable test data generation apparatus of FIG. 2 and / or the human-readable test data analysis apparatus of FIG. 8.

[0020] [Figure 18] 3 is a flowchart representing example machine-readable instructions and / or example operations that may be executed, instantiated, and / or performed by example programmable circuitry to implement the human-readable test data generator of FIG. 2.

[0021] [Figure 19] 9 is a flowchart representing example machine-readable instructions and / or example operations that may be executed, instantiated, and / or performed by example programmable circuitry to implement the human-readable test data analysis apparatus of FIG. 8.

[0022] [Figure 20] 17 and 18. FIG. 18 is a block diagram of an exemplary processing platform including programmable circuitry configured to execute, instantiate, and / or perform the exemplary machine-readable instructions and / or perform the exemplary operations of FIGS. 17 and 18 to implement the human-readable test data generation apparatus of FIG. 2 and to perform the exemplary operations of FIGS. 17 and 19 to implement the human-readable test data analysis apparatus of FIG. 8.

[0023] [Figure 21] FIG. 21 is a block diagram of an example implementation of the programmable circuitry of FIG. 20.

[0024] [Figure 22]FIG. 21 is a block diagram of another exemplary implementation of the programmable circuitry of FIG. 20.

[0025] [Figure 23] FIG. 10 is a block diagram of an exemplary software / firmware / instruction distribution platform (e.g., one or more servers) for distributing software, instructions, and / or firmware (e.g., corresponding to the exemplary machine-readable instructions of FIGS. 17-19) to client devices associated with end users and / or consumers (e.g., for license, sale, and / or use), retailers (e.g., for sale, resale, license, and / or sublicense), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products distributed to other end users, such as retailers and / or direct buy customers).

[0026] Generally, the same reference numbers are used throughout the drawings and accompanying specification to refer to the same or similar parts. The figures are not necessarily to scale. Instead, thicknesses of layers or regions may be exaggerated in the drawings. While layers and regions having clear lines and boundaries are shown in the figures, some or all of these lines and / or boundaries may be idealized. In reality, boundaries and / or lines may be observable, blended, and / or irregular. DETAILED DESCRIPTION OF THE INVENTION

[0027] Examples disclosed herein are directed to a human-readable yield analysis (YA) framework that provides an efficient, reliable, and standardized method for automatically generating YA indicators. As disclosed in further detail below, the exemplary yield analysis framework includes a human-readable test data generator and a human-readable test data analyzer. The YA framework aims to both optimize database storage and provide an efficient, standardized, offline software solution for high-quality YA indicator generation that is portable across products and scales well regardless of product complexity.

[0028] The standardized, automated generation of indicators allows the HRY framework to be used as a standard across different types of products, regardless of the varying levels of product design complexity, providing key capabilities for cross-product comparison. This YA tool enhances yield analysis for products running in a semiconductor fabrication (fab) (e.g., manufacturing plant) as well as enabling cross-product yield comparison.

[0029] The yield health of a logic product can be quantified by the number of test passes on a device under test (DUT) (e.g., a die or unit). Traditionally, the test program itself aggregates test data results acquired across various DUT components. The aggregated test results separate the DUT population into pass and fail bins, such that a pass bin indicates that all screening tests passed on a given DUT. The count of DUTs in the pass bin represents good die yield. The fail bin can be further sorted into separate bins, with different bins indicating different primary failure modes of the DUT. When two or more components fail on the same DUT, a priority binning table is used to determine which component gets priority, with the assigned fail bin reflecting the higher priority failing component. For simple products with a relatively small number of components, the binning approach is sufficient to create a yield health trend in which die counts in fail bins are plotted over time. The time series changes indicate which DUT components are causing overall yield problems, which can be debugged to either correct the yield or detect and overcome any fab yield deviations. A deviation is when a process or equipment deviates from pre-established specifications, which can cause yield loss.

[0030] As the semiconductor device industry moves toward integrated system-on-chip (SOC) products and increasingly complex multi-chiplet co-packaged products using various chiplets and custom logic blocks / components, tracking yield health using die-level binning techniques may not be sufficient. For example, if two or more logic blocks fail on the same product, using a single die-level bin for failure classification may identify only one of the failed logic blocks, the one with the highest bin priority. Such a die-level binning technique may not have sufficient granularity for tracking yield across several die components because it is limited to identifying a single failure, thereby masking the full impact of other logic blocks on the overall yield health of the product or device under test (DUT).

[0031] For example, assume two process parameters (P1, P2) shift together. P1 affects one chiplet design style (C1), while P2 affects another chiplet design style (C2). For this example, further assume that identifying failures in design style C1 has a higher priority than C2 in the bin priority table. The test program may capture both C1 and C2 failures, but due to the single bin assignment, the bin trend associated with the C1 failure will show a change. The true impact of the C2 yield deviation is hidden by the C1 bin assignment. In a worst-case scenario, the failure bin associated with C2 may not show any change at all, even if the health of C2 is degraded.

[0032] In the above example, the test program captures the C1 failure and identifies C1 as causing a yield deviation. This allows the C1 problem to be correlated to the P1 process parameter so that the problem can be corrected. However, because the test program is limited to indicating failures in higher priority components (e.g., C1), the yield deviation for C2 is not indicated by the test program. Therefore, the C2 failure cannot be corrected until the C1 failure is resolved. Therefore, in such an example, the yield deviation is corrected in a sequential manner, starting with the highest priority component and progressing to the lowest priority component.

[0033] This example shows that for modern complex multi-chiplet / logic circuit block products, to enable efficient yield health monitoring, it is important to have granular or separate yield data where components can be tracked individually and monitored in parallel, instead of the traditional single-bin approach that highlights the highest priority defects and hides the true impact of other simultaneous defects on other product components.

[0034] In the examples disclosed herein, test program data can be decoded and processed to generate numeric indicators with human-readable names to generate highly granular yield indicator data for each component. The numeric indicators can be used to associate test data results with the various components tested. A Human Readable Yield (HRY) indicator framework can generate data for each component across a large number of DUTs, as further described below.

[0035] In newer, multi-function circuit block products, yield health is tracked across various functional circuit block components of a final product to understand the overall yield health of a complex product. In the examples disclosed herein, yield health can be tracked and managed using yield health indicators in the HRY framework. Yield indicators can provide a granular or individualized view of a product's yield. The HRY framework disclosed herein provides a method for generating indicators for components tested in a DUT to provide insight into product health and yield issues at high levels (e.g., the device level) and down to the lowest levels (e.g., the circuit component level). For example, the indicators disclosed herein can indicate the highest priority circuit components on the DUT as well as any failing circuit components. The HRY framework can also display yield indicators at higher levels. An exemplary higher-level yield indicator can be a yield indicator at the functional circuit block level, which includes multiple circuit components grouped together based on the design hierarchy.

[0036] The HRY framework disclosed herein employs techniques that provide a simplified, aggregated summary that provides a high-level overview of a problem while simultaneously providing the ability to drill down to find the cause of the problem at a lower level. This allows end users to quickly track a single circuit instance failure indicator at an aggregate level that indicates a failure across any circuit instance, and if this aggregate level indicator shows a change in the yield of the circuit instance over time, the end user can drill down to find the exact circuit instance that is contributing to the overall yield health. The ability to easily switch between different levels of detail reduces the complexity of the YA and can aid in understanding deviations in the fabrication process.

[0037] An example HRY framework implementation disclosed herein may use data encoding and compression techniques when storing raw test data, which may increase the speed of generating indicators. This is because the time used to decode and generate indicators is primarily dominated by data transfer from the database to the YA compute system, rather than the computation time itself. Thus, compressing raw test data may result in smaller test data that requires less time to transfer to the YA compute system.

[0038] The exemplary HRY framework implementation disclosed herein can provide a standard data format for storing test program data regardless of the internal circuit block data format. The HRY framework also standardizes post-processing decoding techniques by establishing clear guidelines for the values ​​generated. This standardization ensures the quality, repeatability, and reliability of the indicator generation process.

[0039] The exemplary HRY framework implementations disclosed herein can extend automated yield indicator generation for complex, multi-functional circuit block products. For example, an indicator for a functional circuit block on an existing product can be extracted and combined with another indicator from a different functional circuit block to generate a new YY indicator for a new product.

[0040] The exemplary HRY framework implementation disclosed herein has offline re-binning capabilities. When an additional screening criterion is added to an existing list of screening criteria, a new bin category is created (e.g., re-binned). This new bin category can be created based on the collected and combined test data to generate a new YA indicator for the new bin category without retesting the device (e.g., die) and without collecting additional test data.

[0041] 1 is a block diagram of an example environment 100 including an example device under test (DUT) 102, example automatic test equipment (ATE) 104, an example workstation 106, and an example output section 108. As disclosed in further detail below, the ATE 104 includes an example human-readable test data generator 110, and the example workstation 106 includes an example human-readable test data analyzer 112, which operate to generate product yield health indicators.

[0042] The exemplary device under test (DUT) 102 is a manufactured product undergoing test. In examples involving semiconductor testing, the DUT may be a die on a wafer or resulting packaged part. In some examples, the DUT 102 may be an electronic component such as an integrated circuit (IC) or a printed circuit board (PCB). In the illustrated example, a connection system 114 is used to connect the DUT 102 to the exemplary ATE 104. For example, the connection system 114 may be an interface test adapter (ITA).

[0043] The exemplary ATE 104 corresponds to a computer-controlled test system used to automatically perform tests on the DUT 102. In the illustrated example, the ATE 104 uses automation to perform measurements and evaluate the test results to determine whether the DUT 102 meets specified performance and / or quality standards. For example, the ATE 104 applies power to the DUT 102, provides stimulus signals to the DUT 102, and then measures and evaluates test outputs resulting from the exemplary DUT 102. The exemplary ATE 104 includes an exemplary human-readable test data generator 110. The human-readable test data generator 110 generates test data representing a sequence of test results from multiple test instances performed by the exemplary ATE 104 on the DUT 102. The exemplary human-readable test data generator 110 is described in more detail below in connection with FIG. 2.

[0044] An exemplary workstation 106 is used as part of the test environment 100. The workstation 106 (e.g., implemented by a computer or computing system) is used to control and manage the test process. The workstation is connected to the ATE 104 via a communication interface 116 (e.g., Ethernet, universal serial bus (USB), general purpose interface bus (GPIB), or any other interface). The workstation 106 serves as an interface between a user and the ATE system 104. A user creates test algorithms, defines test sequences, and executes test programs on the workstation 106. The workstation 106 can also be used to simulate and verify test scenarios before they are implemented on the actual ATE 104 system. The exemplary workstation 106 includes an exemplary human-readable test data analyzer 112. The exemplary human-readable test data analyzer 112 analyzes test data generated by the exemplary ATE 104 system. The exemplary human-readable test data analyzer 112 is described in more detail below in connection with FIG. 8 .

[0045] The exemplary output 108 includes results and data obtained from testing the DUT 102. For example, the output 108 of the test environment 100 may include a pass or fail status of the DUT 102. In some such examples, the DUT 102 may be given a pass status if it meets the specified standards of the test instance applied to the DUT 102, and a fail status may be assigned if the DUT 102 does not meet those standards. In some examples, the output 108 also includes detailed test result data, such as electrical measurements, waveforms, and other relevant information related to the testing of the DUT 102. This test result data provides insight into the behavior of the DUT 102 under different test conditions. The output 108 may also include a test report summarizing the results of the testing process. The test report may include statistics, histograms, and / or graphical representations of the test data. The output 108 may also include traceability features linking the test results to a particular lot of devices tested, a batch of devices tested, and / or individual components in the DUT 102. Such traceability can be useful in tracking and controlling the production process and identifying any problems that may arise.

[0046] 2 is a block diagram of an example implementation of the human-readable test data generator 110 of FIG. 1 for generating test data. The human-readable test data generator 110 of FIG. 2 may be instantiated (e.g., created an instance of, caused to occur for any length of time, embodied, implemented, etc.) by programmable circuitry, such as a central processing unit (CPU) executing a first instruction. Additionally or alternatively, the human-readable test data generator 110 of FIG. 2 may be instantiated (e.g., created an instance of, caused to occur for any length of time, embodied, implemented, etc.) by (i) an application-specific integrated circuit (ASIC) and / or (ii) a field-programmable gate array (FPGA) constructed and / or configured in response to execution of a second instruction to perform operations corresponding to the first instruction. Thus, it should be understood that some or all of the circuitry of FIG. 2 may be instantiated at the same or different times. 2 may be instantiated in one or more threads executing concurrently and / or serially on hardware, for example, and in some examples, some or all of the circuitry of FIG. 2 may be implemented by microprocessor circuitry that executes instructions and / or FPGA circuitry that performs operations to implement one or more virtual machines and / or containers.

[0047] The exemplary human-readable test data generator 110 is part of the ATE 104 system. The ATE 104 system includes an exemplary DUT interface 202, an exemplary test program 204, and an exemplary workstation interface 206. The DUT interface 202 connects to the DUT 102 via the connection system 114. In some examples, the DUT interface 202 is a probe card or test pads that provide a physical connection between the test program 204 running on the ATE 104 and the DUT 102.

[0048] Test program 204 tests the exemplary DUT 102 to determine whether the DUT 102 meets specified performance and quality standards. Test program 204 is a collection of various tests, called test instances, that are executed on the DUT 102. Test program 204 collects test data and sends it to the HRY test data generator 110 via test results interface 210. These test instances are arranged in a directed test flow graph, such as the exemplary directed test flow graph 300 shown in FIG. 3. In some examples, the test results of a given test instance determine the next test instance to be executed. The subsequent test instance can be a new, unexecuted test instance or one that has already been executed (e.g., looping back testing to an executed test instance). In the exemplary directed test flow graph 300 shown in FIG. 3, if the DUT 102 passes test instance A, the test program executes the next test instance, test instance B1. If the DUT 102 passes test instance B1, the test program 204 proceeds to execute test instance B2, etc., according to the direction of test flow in the directed graph 300. A composite test (e.g., a collection of test instances) may refer to different design components (e.g., chiplets, different logic blocks, etc.) or may represent individual tests on a given chiplet or logic block. In some examples, the test program 204 proceeds to test instance C1 if the DUT 102 passes all test instances in composite test B (e.g., test instance B1, test instance B2, and test instance B3). In some examples, the test program 204 proceeds to test instance C1 after a component in the DUT 102 passes test instance B1. The test program 204 may take different test paths depending on the different design components being tested and the different tests to run on the DUT 102. The directed test flow graph 300 is a graph in which the edges have directions. The directions of the edges are indicated by arrows at the edges.Edges indicate pass or fail results. Because test program 204 is a complete, directed graph, there is a finite number of paths connecting nodes (e.g., test instances) and edges. Once test program 204 is assembled and the testflow logic is defined, possible test paths can be computed to create a superset of all possible test paths.

[0049] The exemplary workstation interface 206 connects the test program 204 and the enhanced test report generator 216 of the automatic test equipment (ATE) 104 to the workstation 106 via the communication interface 116. The workstation interface 206 transmits data strings from the ATE 104 to the workstation 106.

[0050] Returning to FIG. 2, the example human-readable test data generator 110 includes an example test results interface 210, an example test instance trace encoder 212, an example parallel test execution encoder 214, and an example enhanced test report generator 216.

[0051] The example test results interface 210 receives test results for test instances executed on the DUT 102 by the test program 204. The test results are sent to the example test instance trace encoder 212.

[0052] The exemplary test instance trace encoder 212 encodes test instances with their respective test results. As the DUT 102 is tested and various tests are executed on the DUT 102, the DUT 102 can be described as following a path along a directed test flow graph (e.g., directed test flow graph 300 of FIG. 3). The example of FIG. 4 shows two DUTs following two different test paths along the directed test flow graph along which test instance information can be recorded: exemplary path 405 and exemplary path 410. In path 405, the sequence of test instances executed on an exemplary first DUT is test instances A, B1, B2, B3, C1, C2, and D. In path 410, the sequence of test instances executed on an exemplary second DUT is test instances A, B3, C21, C22, and D. In the illustrated example, each test instance indicates a pass / fail test result, so that test results for different test instances can be added to the directed graph paths.

[0053] 5 illustrates an example path 505 along with test results included in an example path descriptor 510 by the test instance trace encoder 212. Using the directed graph path 505, the human-readable test data generator 110 can trace the test instances executed on the DUT 102 as well as the test results of the test instances. The test instance trace encoder 212 encodes the path descriptor 510, which is an example data string 510 representing a sequence of test instances and their corresponding executed test results. This data string 510 includes the test path or trace 505 of the DUT 102 and indicates the test history of the DUT 102. The test instances and results in the test path 505 are substrings in the data string separated by delimiters (e.g., commas, periods, semicolons, colons, pipe symbols, slashes, backslashes, etc.). The data string 510 illustrates multiple substrings, each corresponding to a test instance and its test result. For example, the first substring in the data string 510 indicates a first test instance A with a passing test result. The first test instance is represented by the letter "A" and the test result is represented by the letter "pass." The first substring is separated from the second substring (e.g., the second test instance B1 in test path 505) by a slash delimiter. Data string 510 includes multiple substrings that represent the sequence of test instances in test path 505. This test path or trace with test results is also referred to as exemplary trace indicator 510.

[0054] An exemplary test program 204 is shown as a directed graph, such as directed graph 300, where nodes or points on the graph represent test instances and the nodes are connected by edges or lines that represent test results. In some examples, test program 204 has a finite number of test instances, and therefore the number of paths connecting the test instances (or nodes) and edges is finite. When test program 204 is assembled and testflow logic is defined, all possible test paths for a given test program 204 can be calculated before executing test program 204 to create a set of possible trace indicators that represent the set of possible test paths for the given test program 204. An exemplary path descriptor 510 representing possible paths through test program 204 is shown in FIG. 5. Any existing or future algorithm capable of tracing paths in a graph can be utilized to generate possible test paths or traces, such as exemplary trace 505 in FIG. 5. Each trace or test path is assigned a unique alphanumeric string key, referred to as a TraceID. A TraceID (e.g., a test path) includes a unique key (e.g., an alphanumeric character) and a value pair (e.g., a test instance and a test result). The key-value pairs form a dictionary or lookup table. For example, a first path descriptor "A[PASS] / B[FAIL]" can be compressed and encoded as "XY," and a second path descriptor "A[PASS] / B[FAIL]" can be compressed and encoded as "PQ." The path descriptor is composed of the substrings "A[PASS]" and "B[FAIL]." The data strings "XY" and "PQ" are TraceIDs pre-computed from a directed graph of test programs, assuming there are a finite number of test programs in the directed graph. In some examples, the data strings are compressed and encoded as a single alphanumeric character or a sequence of alphanumeric characters with fewer characters than the substrings. In some examples, a path descriptor, such as path descriptor 510, can be an arbitrarily long uncompressed data string with delimited string values.A dictionary or lookup table can be used to map the compressed abbreviated data strings to test instances and test results. Utilizing TraceID string keys to refer to test paths or traces is an efficient means of data compression. For example, 10,000 test instances with their respective pass / fail information can be stored as a TraceID string with only 1,000 characters.

[0055] In some examples, the test instance trace encoder 212 tracks the test results of test instances executed on the DUT 102. At the end of execution of the test program 204, the test instance trace encoder 212 assigns a matching TraceID to the test result. The test instance trace encoder 212 identifies the matching TraceID by using a previously calculated set of possible TraceIDs as a lookup and matching it with the test path taken.

[0056] The disclosed exemplary HRY framework utilizing TraceID increases the speed of yield indicator generation. The time used to decode and generate the indicator is primarily driven by the amount of data transfer from the database to the YA compute system. TraceID provides data encoding and compression by storing raw test execution data and pass / fail results, thereby reducing the amount of test data transferred over the network and resulting in faster execution times when creating yield indicators.

[0057] In some examples, the human-readable test data generator 101 includes means for encoding the data string. For example, the means for encoding may be implemented by test instance trace encoder circuitry, such as the test instance trace encoder 212. In some examples, the test instance trace encoder 212 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the test instance trace encoder 212 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least block 1804 of FIG. 18. In some examples, the test instance trace encoder 212 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or the FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the test instance trace encoder 212 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the test instance trace encoder 212 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier (op-amp), logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other structures are equally suitable.

[0058] The exemplary parallel test execution encoder 214 aggregates information generated by design for testability (DFT) into a single string by encoding test results of test instances as characters in this string. In DFT, multiple circuit instances are tested in parallel for a single test instance executed in a test program flow. For example, memory built-in self-test (MBIST) may test multiple memory instances of a DUT simultaneously or in parallel. A memory circuit block may be connected to a single MBIST engine in the DUT. Multiple MBIST engines may be connected to a global built-in self-test (BIST) interface in the DUT via a test access port network or a dedicated bus. FIG. 6 shows an exemplary hierarchical MBIST implementation 600 that performs parallel testing of multiple memory instances of a DUT within a single test instance in a test program flow. When an MBIST engine 602, 604, 606 completes testing one of its local memory instances (e.g., Mem1 612, Mem2 614, Mem3 616, Mem4 618, Mem5 620), the MBIST engine sends the test data over local BIST interface buses 622, 624. The local BIST interface buses 622, 624 take the test output data and send it over a global BIST interface 626 to a central joint test action group (JTAG) test access port (TAP) 628.

[0059] In a parallel DFT implementation, the test instance-level pass / fail results encoded in the TraceID (as described above) can indicate a single status (e.g., pass / fail) for a test instance executed in parallel on multiple circuit instances. The test instance-level results indicate a pass status if all circuit instances pass the test instance, or a fail status if any of the circuit instances fail the test instance. In this example, the TraceID does not track which memory instances passed or failed at the memory instance level.

[0060] In the case of DFT, which executes parallel tests on multiple circuit instances, the parallel test execution encoder 214 encodes test outcome data for multiple circuit instances tested (e.g., in parallel) by a single test instance into a string associated with the test instance, referred to as an exemplary multi-result string or exemplary HRY raw string (HRY_RAWSTR). FIG. 7 illustrates an exemplary multi-result string 705 or exemplary HRY_RAWSTR 705 for the above-described exemplary MBIST hierarchical implementation 600. In some implementations, a python script is used to aggregate test results for multiple circuit instances running the same test instance. In the example of FIG. 7, the HRY_RAWSTR 705 encodes test results 703 for five memories (e.g., memory (Mem)1 612, Mem2 614, Mem3 616, Mem4 618, and Mem5 620) as the five-character string “11R1U.” The HRY_RAWSTR character data string 705 provides efficient database storage and standardizes the generation of test result indicators.

[0061] Exemplary techniques for encoding a multi-result string or HRY_RAWSTR, such as multi-result string 705 or HRY_RAWSTR 705, are provided in Tables 1 and 2 below. Table 1 illustrates exemplary meanings of characters or bits in HRY_RAWSTR 705. In the example of Table 1, a character value of "0" indicates a fail, a character value of "1" indicates a pass, a character value of "R" indicates a repairable fail, and a character value of "U" indicates an irreparable fail. The HRY_RAWSTR character value is not limited to simply encoding test result values ​​(0, 1, R, U); HRY_RAWSTR can also log the results of tests that did not complete successfully on a specific circuit block among multiple parallel circuit blocks tested in a test program. In this case, HRY_RAWSTR may be assigned an exemplary value of "9," indicating that a specific circuit block did not complete the test successfully. The HRY_RAWSTR string concept not only encodes test result data, but also encodes the absence of a test result. [Table 1] Table 1 Table 2 shows an example mapping of character positions in HRY_RAWSTR. In some examples, the character positions in HRY_RAWSTR represent corresponding circuit blocks in a hierarchical MBIST implementation. Table 2 shows an example character position mapping corresponding to the hierarchical MBIST implementation 600. In Table 2, bit position 0 corresponds to memory circuit 1 612, bit position 1 corresponds to memory circuit 2 614, and so on. Decoding the multi-result string HRY_RAWSTR having a value of "11R1U" using Tables 1 and 2 indicates that memory circuit 1 612 passed, memory circuit 2 614 passed, memory circuit 3 616 had a repairable failure, memory circuit 4 618 passed, and memory circuit 5 620 had an unrepairable failure. [Table 2] Table 2

[0062] In some examples, the human-readable test data generator 101 includes means for generating a second data string to associate with a first one of the substrings of the first data string. For example, the means for generating the second data string may be implemented by parallel test execution encoder circuitry, such as the parallel test execution encoder 214. In some examples, the parallel test execution encoder 214 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the parallel test execution encoder 214 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least block 1810 of FIG. 18. In some examples, the parallel test execution encoder 214 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or the FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the parallel test execution encoder 214 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the parallel test execution encoder 214 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other structures are equally suitable.

[0063] The exemplary extended test report generator 216 outputs a test report including the TraceID data string and the HRY_RAWSTR data string for the DUT to a second device. The second device may include the exemplary workstation 106. The extended test report generator 216 standardizes the reported test result data across multiple DFTs by encoding the test results using the same table (e.g., Table 1) for a particular DFT logic block type. This helps ensure that different test programs using a particular DFT logic block test generate a standard HRY_RAWSTR data string regardless of the DUT being tested.

[0064] In some examples, the human-readable test data generator 101 includes means for outputting the data string to a second device. For example, the means for outputting the data string may be implemented by extended test report generator circuitry, such as extended test report generator 216. In some examples, the extended test report generator 216 may be instantiated by programmable circuitry, such as the exemplary programmable circuitry 2012 of FIG. 20 . For example, the extended test report generator 216 may be instantiated by the exemplary microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least block 1814 of FIG. 18 . In some examples, the extended test report generator 216 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or an FPGA circuitry 2200 of FIG. 22 , configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the extended test report generator 216 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the enhanced test report generator 216 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier (op-amp), logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other configurations are equally suitable.

[0065] FIG. 8 is a block diagram of an example implementation of the human-readable test data analysis device 112 of FIG. 1 for analyzing test data. The human-readable test data analysis device 112 of FIG. 8 may be instantiated (e.g., create an instance of, cause to occur for any length of time, embody, implement, etc.) by programmable circuitry, such as a central processing unit (CPU) executing a first instruction. Additionally or alternatively, the human-readable test data analysis device 112 of FIG. 8 may be instantiated (e.g., create an instance of, cause to occur for any length of time, embody, implement, etc.) by (i) an application-specific integrated circuit (ASIC) and / or (ii) a field-programmable gate array (FPGA) constructed and / or configured in response to execution of a second instruction to perform operations corresponding to the first instruction. Thus, it should be understood that some or all of the circuitry of FIG. 8 may be instantiated at the same or different times. Some or all of the circuitry of FIG. 8 may be instantiated, for example, in one or more threads executing concurrently on hardware and / or serially on hardware. Also, in some examples, some or all of the circuitry of FIG. 8 may be implemented by microprocessor circuitry that executes instructions and / or FPGA circuitry that performs operations to implement one or more virtual machines and / or containers.

[0066] The exemplary human-readable test data analyzer 112 is part of the workstation 106. The workstation 106 includes a test report receiver 802 for receiving a test report from the ATE 104 via the communication interface 116. The test report includes a TraceID data string and an HRY_RAWSTR data string.

[0067] The example human-readable test data analyzer 112 includes an example data string decoder 804, an example hierarchical test generator 806, an example user-specified test configuration generator 808, an example indicator generator 810, an example bin category generator 812, an example test result generator 814, and an example hierarchical tree database 816.

[0068] The exemplary data string decoder 804 decodes data strings received from the ATE 104 into test outcomes, which may be in the form of a list-based data structure. The data string decoder 804 decodes TraceID data strings and HRY_RAWSTR data strings. In some examples, the data string decoder 804 decodes TraceID data strings using a lookup table containing a set of possible TraceID data strings for a given directed test flow graph, such as directed test flow graph 300. This is because the set of TraceID lookup tables has a fixed set of TraceID data strings for a particular test program. The generated indicator is one of the TraceIDs in the lookup table, thereby creating a standardized indicator for the set of TraceIDs in the test program. Decoding the TraceID generates a result status (e.g., pass / fail data) for each test instance. The data string decoder 804 decodes multi-result HRY_RAWSTR data strings to provide detailed test outcome information for multiple circuit blocks tested (e.g., in parallel) by corresponding test instances. For example, decoding HRY_RAWSTR provides test outcomes 703 of the different memory instances 612, 614, 616, 618, 620 tested by the single test instance described above in connection with FIG.

[0069] In some examples, the human-readable test data analysis device 112 includes means for decoding the data strings. For example, the means for decoding may be implemented by data string decoder circuitry, such as data string decoder 804. In some examples, the data string decoder 804 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the data string decoder 804 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least blocks 1904, 1906 of FIG. 19. In some examples, the data string decoder 804 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or an FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the data string decoder 804 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the data string decoder 804 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other configurations are equally suitable.

[0070] The exemplary hierarchical test generator 806 generates a test indicator by combining the test outcomes of at least two circuit blocks based on hierarchical rules. The hierarchical rules are based on the physical grouping of the circuit blocks. The physical grouping of the circuit blocks is based on the physical design of the DUT. Thus, the test indicator indicates the test outcomes of two or more groups of circuit blocks. The data string decoder 804 decodes the multi-result data string to form a list-based data structure representing the bottom levels of the hierarchical tree, i.e., Level 2C 902, Level 2D 904, and Level 2E 906, such as the exemplary hierarchical tree 900 shown in FIG. 9 . The hierarchical test generator 806 forms an HRY hierarchical tree using elements from the design hierarchy. The hierarchical tree 900 allows a user to view the overall health of the DUT. In the illustrated example of FIG. 9 , the hierarchical tree 900 indicates the overall health of the DUT. In this example, the DUT is a die having multiple circuits. In some examples, the multiple circuits are memory circuits 902, 904, and 906. The hierarchical test generator 806 groups multiple memory circuits 902, 904, 906 into functional circuit blocks 908, 910. Using the hierarchical tree 900, a user can view the overall health of the circuits on the DUT (e.g., die) at DUT Level 0 912—in this case, the overall health of the memory circuits on the DUT. The user can also compare the health of the memory circuits in functional circuit block 908 at Level 1A with the health of the memory circuits in functional circuit block 910 at Level 1B.

[0071] The example indicator generator 810 generates different level yield health indicators by summarizing lower level test performance indicators 1002, 1004, 1006 to provide higher level combined yield health indicators 1008, 1010, 1012 based on hierarchical rules. FIG. 10 shows an example HRY hierarchy tree along with the structure of yield health indicators. The example indicator generator 810 summarizes the lower level test performance indicators 1002, 1004, 1006 to provide combined higher level yield health indicators 1008, 1010 (e.g., function block level 1A 908, function block level 1B 910) based on hierarchical rules using logical groupings. The higher level yield health indicators 1008, 1010, 1012 may be abstract views of DUT health based on different logical combinations of lower level test performance. The exemplary hierarchical test generator 806 provides the ability to combine individual circuit blocks 902, 904, 906 into functional circuit blocks 908, 910 to automatically generate functional circuit block health indicators 1008, 1010 across various DUTs that use the same functional circuit block. For example, the hierarchical test generator 806 combines test results 902 from a memory instance at Level 2C with test results 904 from a memory instance at Level 2D to form functional circuit block Level 1A 908. Using the HRY hierarchical tree summarization technique simplifies comparison of functional circuit block health across DUTs or products. Users can compare yield health indicators at Level 0 1012 for quick comparison. If there is a mismatch at Level 0 912 between two different DUTs that use the same functional circuit block, the user may drill down to lower levels (e.g., Level 1 908, 910 and / or Level 2 902, 904, 906) to identify the circuit instance causing the yield mismatch.10, if any of the lower levels 902, 904, 906 have a health indicator of "0" meaning fail, then the higher level yield health or summary indicators 1008, 1010, 1012 will be "0" meaning fail. If all of the lower levels 902, 904, 906 are "1" meaning pass, then the higher level summary indicators 1008, 1010, 1012 will be "1" meaning pass.

[0072] In some examples, the human-readable test data analysis device 112 includes means for combining at least two of the test outcomes based on a hierarchical rule. For example, the means for combining at least two of the test outcomes may be implemented by hierarchical test generator circuitry, such as hierarchical test generator 806. In some examples, the hierarchical test generator 806 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the hierarchical test generator 806 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least blocks 1908, 1910, 1912, 1914, 1916, and 1918 of FIG. 19. In some examples, the hierarchical test generator 806 may be instantiated by hardware logic circuitry, such as may be implemented by ASIC, XPU, or FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, hierarchical test generator 806 may be instantiated by any other combination of hardware, software, and / or firmware. For example, hierarchical test generator 806 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other configurations are equally suitable.

[0073] The exemplary user-specified test configuration generator 808 generates user-defined hierarchy rules to enable a user to provide a user-defined hierarchy tree. For example, a user may create user-defined hierarchy rules to compare the health of various bitcell types used on a DUT or across different DUTs. To this end, a user may invoke the user-specified test configuration generator 808 to generate user-defined rules such as those shown in the example of FIG. 11 . Although bitcell types are the lowest level in the design hierarchy, the user-specified test configuration generator 808 may reorder the user-defined hierarchy tree to find an answer to the problem of comparing yield per bitcell type. According to the DUT design hierarchy, the exemplary user-specified test configuration generator 808 uses the DUT physical circuit design to generate functional block levels 908 and 910. Functional block level 1A 908 is created by memory instance level 2C 902 of bitcell type 1 and memory instance level 2D 904 of bitcell type 2. Functional block level 1B 910 is created by memory instance level 2E 906 of bitcell type 2.

[0074] However, by using user-defined hierarchy rules, the hierarchy tree 1100 does not have to follow the DUT design hierarchy. In the illustrated example of FIG. 11 , the user-specified test configuration generator 808 groups the memory circuits 902, 904, 906 by their configuring bit cell types 1102, 1104. This grouping summarizes the yields 1106, 1108 by different bit cell types 1102, 1104. The user-defined levels (e.g., bit cell type levels 1102, 1104) do not have to follow the design hierarchy. The user-defined levels 1102, 1104 can be any grouping of circuit blocks 902, 904, 906 that satisfies any custom yield analysis.

[0075] 12 , a user may invoke the user-specified test configuration generator 808 to create an HRY hierarchy tree 1200 that combines multiple DFT features, such as memory test instances 902, 904, and 906, and scan diagnostic tests 1202 in the same DUT yield health indicator 1204. In some examples, memory test instances and scan tests cannot be executed simultaneously on the same test instance in a test program. However, by using the HRY hierarchy tree 1200, the user-specified test configuration generator 808 may combine the results of the memory test instances 902, 904, and 906 and the scan diagnostic tests 1202 to generate additional insight into overall yield health across multiple DFT criteria. The user-specified test configuration generator 808 may combine the test results of memory instances at Level 2C 902 and Level 2D 904 to create a yield health indicator for the function block Level 1A 1206 to track the health of the memory in the function block Level 1A 1206. The user-specified test configuration generator 808 can combine the memory instance level 2E 906 and the scan diagnostic coverage test level 2F 1202 to create the function block level 1B 1208. The function block level 1B 1208 tracks the health of the memory instances 906 and the scan tests 1202. The HRY hierarchy tree 1200 model reduces the complexity of the YA by enabling different combinations of test performance information and providing quick access to summary data or yield health indicators 1204 at design hierarchy levels (e.g., DUT level 1204, functional circuit block levels 1206, 1208, base layer circuit instance levels 902, 904, 906, and / or user-defined levels).

[0076] In some examples, the human-readable test data analysis device 112 includes means for combining test outcomes based on a user definition. For example, the means for combining test outcomes based on a user definition may be implemented by user-specified test configuration generator circuitry, such as the user-specified test configuration generator 808. In some examples, the user-specified test configuration generator 808 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the user-specified test configuration generator 808 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least blocks 1922, 1924, 1926, and 1928 of FIG. 19. In some examples, the user-specified test configuration generator 808 may be instantiated by hardware logic circuitry, such as may be implemented by the ASIC, XPU, or FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the user-specified test configuration generator 808 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the user-specified test configuration generator 808 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier (op-amp), logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other structures are equally suitable.

[0077] The example indicator generator 810 generates test indicators for groupings of circuit blocks and generates combined indicators based on combinations of one or more test indicators. For example, as described above in conjunction with FIG. 10 , the indicator generator 810 generates human-readable yield indicators from the TraceID and HRY_RAWSTR data strings decoded by the data string decoder 804. The data string decoder 804 uses the TraceID data string to decode the test results of a test instance or the multi-result HRY_RAWSTR data string 1014 to decode the test outcomes of multiple circuit instances. The decoded HRY_RAWSTR test outcome indicators 1002, 1004, 1006 can be combined to create higher-level summary test yield health indicators 1008, 1010. The indicator generator 810 combines the low-level indicators 1002, 1004, 1006 and the HRY tree-based hierarchical indicators 1008, 1010 using Boolean operators (eg, AND, OR, NOT) to create a combined yield indicator.

[0078] Exemplary test performance decoded by the exemplary data string decoder 804 is shown in Table 3. Table 3 shows exemplary circuit instance-level test performance from decoding an exemplary multi-result HRY_RAWSTR data string. Table 3 includes rows representing test performance of various memory instances in DUT1 and DUT2. The columns in Table 3 correspond to the yield health of memory instance 1, memory instance 2, and memory instance 3. Circuit instance level test results [Table 3] Table 3

[0079] Exemplary test results decoded by data string decoder 804 are shown in Table 4. Table 4 shows exemplary test instance level test results from decoding an exemplary TraceID. Table 4 includes rows representing test results for test instances of DUT1 and DUT2. The columns in Table 4 correspond to yield health indicators for Functional Test 1 and Functional Test 2. Test instance level test results [Table 4] Table 4

[0080] By utilizing the decoded test outcomes in Table 3, a combined yield indicator can be created using the HRY framework. The indicator generator 810 combines the test outcomes and test results using Boolean operators (e.g., AND, OR, NOT). The low-level yield indicators including the test results decoded from the TraceID and the test outcomes decoded from the HRY_RAWSTR data string can be combined. The indicator generator 810 combines these low-level yield indicators using Boolean operators to create a combined yield indicator.

[0081] The exemplary combined indicator reduces the complexity of tracking multiple indicators separately and allows a user to simply track one combined or composite indicator. For example, a user may invoke the indicator generator 810 to create combined yield indicator 1. Combined yield indicator 1 may be defined as passing if (functional test 1: passing) AND (functional test 2: passing) AND (memory instance 2: passing). Combined yield indicator 1 may be defined as failing if any of the test instances in the definition are not passing.

[0082] When applying the above definition of Combinatorial Yield Indicator 1 to DUT1 shown in Tables 3 and 4, DUT1 has (Functional Test 1: Fail) AND (Functional Test 2: Pass) AND (Memory Instance 2: Fail). Therefore, DUT1 is categorized as Fail. When applying the definition of Combinatorial Yield Indicator 1 to DUT2, DUT2 has (Functional Test 1: Pass) AND (Functional Test 2: Pass) AND (Memory Instance 2: Pass). Therefore, DUT2 is categorized as Pass because all three test instances of DUT2 have a Pass status.

[0083] Combined indicators reduce the complexity of test tracking by allowing tracking of one combined indicator instead of tracking multiple indicators separately. When a combined indicator indicates a change, the user can easily access the underlying indicators to find the root cause causing the change to yield health. Combined indicators can be further extended to create combinations of combined indicators. The example shown in Table 5 illustrates how the indicator generator 810 creates an indicator of a combined indicator. [Table 5] Table 5 Table 5 includes rows corresponding to combined yield indicators 1, 2, and 3. The columns in Table 5 represent combinations of passed and failed test instances for combined yield indicators 1, 2, and 3. The indicator generator 810 defines combined yield indicators 1 and 2 based on the test results and test performance data. The indicator generator 810 defines combined yield indicator 3 based on the combination of combined yield indicators 1 and 2. Combining combined yield indicators allows users to track a smaller set of indicators when monitoring for any changes across a larger set of sub-indicators. When a change is detected in the aggregate-level indicator of a combined indicator, the HRY framework has the ability to quickly drill down and find which lower-level indicator is causing the change in the higher-level aggregate indicator. The indicator generator 810's ability to combine different indicators and combined indicators allows users to create and track different test elements. Users can highlight issues or track the most important yield problems.

[0084] For example, FIG. 13 illustrates an exemplary application of machine learning to identify top indicators for a DUT. The exemplary data string decoder 804 decodes the TraceID and HRY_RAWSTR data strings to create a group of lower-level, fine-grained indicators 1302. The hierarchical test generator 806 combines test results and test outcomes to form an HRY hierarchical tree-based summary indicator 1304. The user-specified test configuration generator 808 combines test indicators based on user-defined rules to create a combined indicator 1306. A user may invoke the human-readable test data analyzer 112 to apply data science techniques such as principal component analysis (PCA) or machine learning techniques such as feature selection to identify top test instance indicators 1310 that result in an overall DUT yield health indicator from a given set of lower-level yield health indicators. The exemplary indicator generator 810 creates a combined indicator 1312 based on the top test instance indicators identified by the above data science techniques.

[0085] FIG. 14 shows an example graph 1400 illustrating the yield health of a DUT (e.g., die) over different time frames. Dashed line 1402 shows good die count, and solid line 1404 shows die count based on a combined indicator created by an automated system that identifies the top indicator using the data science techniques described above in conjunction with FIG. 13 . The automated system may compare the overall good die yield to the die yield based on the combined indicator created using the top indicator. For example, in FIG. 14 , in time frame 1406, the graph shows that the overall good die yield and the die yield based on the combined indicator are consistent, and the automated system detects no deviation in the overall yield health of the DUT (e.g., die). In time frame 1408, the system detects that the overall yield health based on the combined indicator has deviated from the overall yield health of the good die. The automated system may trigger a process to find a new combined indicator based on the new top yield indicator. Utilizing a yield automation system with the HRY framework allows users to create artificial intelligence (AI)-based yield tracking applications. The AI ​​can track the combination indicators and the deterioration between overall yield from a good die count. If deviations are detected, the automation system can create new combination indicators. The automation system can evaluate the effectiveness of top indicators identified by the data science techniques described above in conjunction with FIG. 13.

[0086] The ability to create indicators of combined indicators is well suited to yield automation systems that use machine learning and other AI-based techniques. Aggregating indicators also facilitates screening for anomalous DUTs, where filtering at the aggregated indicator level can detect anomalies across a wide range of lower-level yield indicators. This aggregated indicator reduces the complexity of YA by combining information from thousands of indicators into a handful of combined indicators that can be easily tracked and understood.

[0087] In some examples, the human-readable test data analysis device 112 includes means for determining test indicators. For example, the means for determining may be implemented by indicator generation device circuitry, such as indicator generation device 810. In some examples, the indicator generation device 810 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the indicator generation device 810 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least blocks 1920, 1930, and 1932 of FIG. 19. In some examples, the indicator generation device 810 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or the FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the indicator generation device 810 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the indicator generating device 810 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier (op-amp), logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other configurations are equally suitable.

[0088] The exemplary bin category generator 812 generates category values ​​for testing the DUT based on the definition of DUT quality. The HRY framework can generate a variety of indicators. Such indicators include lower-level indicators, HRY tree hierarchy summary indicators, and / or combination indicators. These indicators are binary in nature (e.g., they represent either a pass or fail value for the indicator). Bin categories provide category values ​​for these indicators. For example, as shown in FIG. 15 , bin categories (e.g., Best 1502, Better 1504, Good 1506, and Fail 1508) are created by applying logic-based rules to test instances. For example, the Best bin category 1502 requires that indicators for Test 1, Test 2, and Test 3 have a pass status. The bin category generator 812 creates bin categories using test results from multiple test modules (e.g., memory integrity test, scan logic integrity test, etc.). During the test development or debug phase of the DUT, bin categories are assigned to monitor the DUT quality. If the definition of DUT quality changes, a new bin category may be assigned. However, the bin category generator 812 may perform offline binning if the definition of DUT quality changes. The bin category generator 812 creates new bin categories using the HRY indicator. The bin category generator 812 creates bin categories based on the collected and combined test data without retesting the device (e.g., die) and without collecting additional test data, and generates a new YA indicator for the new bin category.

[0089] FIG. 16 illustrates an offline binning process incorporated into an exemplary manufacturing flow 1600. The manufacturing flow 1600 is divided into two separate manufacturing processes: an exemplary test program data collection process 1602 and an exemplary offline binning process 1604. First, the test program data collection process 1602 is performed to generate data capturing HRY framework-specific data (e.g., TraceID and HRY_RAWSTR) and other parametric data. Next, the offline binning process 1604 is performed as a virtual operation by the bin category generator 812. The data string decoder 804 extracts HRY framework elements, such as TraceID and HRY_RAWSTR data strings, and the indicator generator 810 creates different levels of HRY indicators. The test result generator 814 rearranges other test data, such as parametric data, into an optimized data structure. Finally, the bin category generator 812 executes new bin classification rules and assigns new bins to each DUT offline. The bin category generator 812 obtains the second hierarchical rules after determining the first test report including the TraceID and the HRY_RAWSTR data string. Advantages of the offline binning process 1604 using the HRY framework are test time savings and ease of use. If new bin classifications are created based on already collected test data, the offline binning process 1604 is performed without running the first manufacturing process 1602. A new test program does not need to be released, and the first manufacturing process 1602 does not need to be performed again. A second test report based on the second hierarchical rules and data strings is output without running additional test instances on the device under test.

[0090] In some examples, the human-readable test data analysis device 112 includes means for generating classifications of the DUTs offline. For example, the means for generating classifications may be implemented by bin category generator circuitry, such as the bin category generator 812. In some examples, the bin category generator 812 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the bin category generator 812 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least blocks 1940, 1942 of FIG. 19. In some examples, the bin category generator 812 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or the FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the bin category generator 812 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the bin category generator 812 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier (op-amp), logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other configurations are equally suitable.

[0091] The example test results generator 814 generates a DUT yield health or test report based on various test indicators decoded from the TraceID and HRY_RAWSTR data string. The test report includes customized indicators or bin category values ​​for the DUT 102. The DUT yield health or test report displays test results at each level of a hierarchical data structure. The test results displayed at each hierarchical level are based on either physical or logical groupings of various tests. Data analysts have the option of working with various levels of data granularity, from the lowest level corresponding to individual circuit block test results at the highest granularity, to higher intermediate levels including test results for one or more combinations of circuit blocks, and to the highest level corresponding to the lowest granularity test results at the DUT level.

[0092] In some examples, the human-readable test data analysis device 112 includes means for outputting a test report based on the test indicators. For example, the means for outputting the test report may be implemented by test result generation device circuitry, such as the test result generation device 814. In some examples, the test result generation device 814 may be instantiated by programmable circuitry, such as the example programmable circuitry 2012 of FIG. 20. For example, the test result generation device 814 may be instantiated by the example microprocessor 2100 of FIG. 21 executing machine-executable instructions, such as those implemented by at least blocks 1936 and 1938 of FIG. 19. In some examples, the test result generation device 814 may be instantiated by hardware logic circuitry, such as may be implemented by an ASIC, an XPU, or the FPGA circuitry 2200 of FIG. 22, configured and / or constructed to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the test result generation device 814 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the test result generating device 814 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier (op-amp), logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other configurations are equally suitable.

[0093] The exemplary hierarchical tree database 816 stores a design hierarchical tree or a user-defined hierarchical tree specified by the hierarchical test generator 806 using a physical grouping of circuit blocks or a logical grouping of circuit blocks based on the physical design of the DUT. The user-defined hierarchical tree is generated by the exemplary user-specified test configuration generator 808 based on the user definition.

[0094] 1 is shown in Figure 2, and the human-readable test data analysis apparatus 112 of Figure 1 is shown in Figure 8, one or more of the elements, processes, and / or devices shown in Figures 2 and 8 may be combined, divided, rearranged, omitted, removed, and / or implemented in any other manner. Furthermore, the example test results interface 210, the example test instance trace encoder 212, the parallel test execution encoder 214, the extended test report generator, and / or, more generally, the example human-readable test data generator 110 of Figure 2, the example data string encoder 804, the example hierarchical test generator 806, the example user-specified test configuration generator 808, the indicator generator 810, the bin category generator 812, the test result generator 814, and / or, more generally, the example human-readable test data analysis apparatus 112 may be implemented by hardware alone or by hardware in combination with software and / or firmware.Thus, for example, any of the example test results interface 210, the example test instance trace encoder 212, the parallel test execution encoder 214, the extended test report generator, and / or more generally, any of the example human-readable test data generator 110, the example data string encoder 804, the example hierarchical test generator 806, the example user-specified test configuration generator 808, the indicator generator 810, the bin category generator 812, the test result generator 814, and / or more generally, the example human-readable test data analysis apparatus 112 may be implemented by programmable circuitry in combination with machine-readable instructions (e.g., firmware or software), processor circuitry, analog circuitry, digital circuitry, logic circuitry, programmable processors, programmable microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs), ASICs, programmable logic devices (PLDs), and / or field programmable logic devices (FPLDs), such as FPGAs. Still further, the example human-readable test data generator 110 of FIG. 2 and the example human-readable test data analyzer 112 of FIG. 8 may include one or more elements, processes, and / or devices in addition to or instead of those shown in FIGS. 2 and 8, and / or may include more than one of any or all of the elements, processes, and devices shown.

[0095] Flowcharts representing example machine-readable instructions that may be executed by programmable circuitry to implement and / or instantiate the human-readable test data generator 110 of FIG. 2 and the human-readable test data analyzer 112 of FIG. 8 and / or example operations that may be performed by programmable circuitry to implement and / or instantiate the human-readable test data generator 110 of FIG. 2 are shown in FIGS. 17-18, and the human-readable test data analyzer 112 of FIG. 8 is shown in FIGS. 17 and 19. The machine-readable instructions may be one or more executable programs or portions of one or more executable programs for execution by programmable circuitry, such as the programmable circuitry 2012 shown in the example processor platform 2000 described below in connection with FIG. 20, and / or may be one or more functions or portions of functions performed by the example programmable circuitry (e.g., FPGA) described below in connection with FIGS. 21 and / or 22. In some instances, the machine-readable instructions cause an action, task, etc. to be performed and / or carried out in the real world in an automated manner. As used herein, "automated" means without human involvement.

[0096] The program may be embodied in instructions (e.g., software and / or firmware) stored on one or more non-transitory computer-readable and / or machine-readable storage media, such as cache memory, a magnetic storage device or disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical storage device or disk (e.g., a Blu-ray disk, a compact disk (CD), a digital versatile disk (DVD), etc.), a redundant array of independent disks (RAID), registers, ROM, a solid-state drive (SSD), SSD memory, non-volatile memory (e.g., an electrically erasable programmable read-only memory (EEPROM), flash memory, etc.), volatile memory (e.g., any type of random access memory (RAM), etc.), and / or any other storage device or storage disk. The instructions of the non-transitory computer-readable and / or machine-readable medium may be programmed and / or executed by programmable circuitry located in one or more hardware devices, although the entire program and / or portions thereof may alternatively be executed and / or instantiated by one or more hardware devices other than programmable circuitry and / or embodied in dedicated hardware. The machine-readable instructions may be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., a server and a client hardware device). For example, a client hardware device may be implemented by an end-point client hardware device (e.g., a hardware device associated with a human and / or machine user) or an intermediate client hardware device gateway (e.g., a radio access network (RAN) that may facilitate communication between a server and an end-point client hardware device). Similarly, a non-transitory computer-readable storage medium may include one or more media.Furthermore, although the exemplary programs are described with reference to the flowcharts shown in Figures 17-19, many other ways of implementing the exemplary human-readable test data generator 110 and human-readable test data analyzer 112 may alternatively be used. For example, the order of execution of the flowchart blocks may be changed, and / or some of the described blocks may be modified, eliminated, or combined. Additionally or alternatively, any or all of the flowchart blocks may be implemented by one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. The programmable circuitry may be distributed across different network locations and / or local to one or more hardware devices (e.g., single-core processors (e.g., single-core CPUs), multi-core processors (e.g., multi-core CPUs, XPUs, etc.)). For example, the programmable circuitry may be a CPU and / or FPGA located in the same package (e.g., the same integrated circuit (IC) package or two or more separate housings), one or more processors in a single machine, multiple processors distributed across multiple servers in a server rack, multiple processors distributed across one or more server racks, and / or any combination thereof.

[0097] The machine-readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. The machine-readable instructions described herein may be stored as data (e.g., computer-readable data, machine-readable data, one or more bits (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), a bitstream (e.g., a computer-readable bitstream, a machine-readable bitstream, etc.)), or a data structure (e.g., a portion of instructions, code, a representation of code, etc.) that can be utilized to create, manufacture, and / or produce machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored on one or more storage devices, disks, and / or computing devices (e.g., servers) located at the same or different locations of a network or collection of networks (e.g., cloud, edge devices, etc.). Machine-readable instructions may require one or more of installing, modifying, adapting, updating, combining, supplementing, configuring, decrypting, decompressing, unpacking, distributing, reassigning, compiling, etc. to render them directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, machine-readable instructions may be stored in multiple portions that are individually compressed, encrypted, and / or stored on separate computing devices, where the portions, when decrypted, decompressed, and / or combined, form a set of computer-executable and / or machine-executable instructions that implement one or more functions and / or operations that may together form a program such as those described herein.

[0098] In another example, machine-readable instructions may be stored in a manner that allows them to be read by programmable circuitry, but requires the addition of a library (e.g., a dynamic link library (DLL)), software development kit (SDK), application programming interface (API), etc., to execute the machine-readable instructions on a particular computing device or other device. In another example, machine-readable instructions may need to be configured (e.g., stored settings, data input, recorded network addresses, etc.) before the machine-readable instructions and / or corresponding program can be executed in whole or in part. Thus, as used herein, machine-readable, computer-readable, and / or machine-readable medium may include instructions and / or programs regardless of the particular format or state of the machine-readable instructions and / or programs.

[0099] The machine-readable instructions described herein may be expressed in any past, present, or future command language, scripting language, programming language, etc. For example, the machine-readable instructions may be expressed using any of the following languages: C, C++, Java®, C#, Perl, Python, JavaScript®, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift®, etc.

[0100] 17-19 may be implemented using executable instructions (e.g., computer-readable and / or machine-readable instructions) stored on one or more non-transitory computer-readable and / or machine-readable media. As used herein, the terms non-transitory computer-readable medium, non-transitory computer-readable storage medium, non-transitory machine-readable medium, and / or non-transitory machine-readable storage medium are expressly defined to include any type of computer-readable storage device and / or storage disk and to exclude propagating signals and to exclude transmission media. Examples of such non-transitory computer-readable medium, non-transitory computer-readable storage medium, non-transitory machine-readable medium, and / or non-transitory machine-readable storage medium include optical storage devices, magnetic storage devices, HDDs, flash memory, read-only memory (ROM), CDs, DVDs, caches, any type of RAM, registers, and / or any other storage device or storage disk in which information is stored for any length of time (e.g., long-term, permanent, short-term, temporary buffering, and / or caching of information). As used herein, the terms "non-transitory computer-readable storage device" and "non-transitory machine-readable storage device" are defined to include any physical (mechanical, magnetic, and / or electrical) hardware for holding information over a period of time, but to exclude propagating signals and to exclude transmission media. Examples of non-transitory computer-readable storage devices and / or non-transitory machine-readable storage devices include any type of random access memory, any type of read-only memory, solid-state memory, flash memory, optical disks, magnetic disks, disk drives, and / or redundant array of independent disks (RAID) systems.As used herein, the term "device" refers to a physical structure, such as mechanical and / or electrical equipment, hardware, and / or circuitry, that may or may not be configured with and / or manufactured to execute computer-readable instructions, machine-readable instructions, etc.

[0101] FIG. 17 is a flowchart depicting example machine-readable instructions and / or example operations 1700 that may be executed, instantiated, and / or performed by programmable circuitry to generate a semiconductor product yield indicator. The example machine-readable instructions and / or example operations 1700 of FIG. 17 begin at block 1702, where the example human-readable test data generator 110 generates test data including one or more encoded data strings, as described further below in conjunction with FIG. 18. At block 1704, the example human-readable test data analyzer 112 analyzes the test data by decoding the received test data and performing further analysis on the test data, as described further below in conjunction with FIG. 19. At block 1706, the example output unit 108 outputs a test report. As described above in conjunction with FIG. 1, the test report includes results and data obtained from testing the DUT 102. The test report summarizes the results of the testing process.

[0102] 18 is a flowchart representing example machine-readable instructions and / or example operations 1702 that may be executed, instantiated, and / or performed by programmable circuitry to generate test data. The example machine-readable instructions and / or example operations 1702 of FIG. 18 begin at block 1802, where the example test results interface 210 receives test results for tests executed on the DUT 102. At block 1804, the example test instance trace encoder 212 encodes the test results into a first data string (e.g., a TraceID) that represents the test results for the sequence of test instances executed by the test program 204.

[0103] In block 1806, the example human-readable test data generator 110 selects test instances in the first data string (e.g., TraceID) to check for parallel test execution. In block 1808, the human-readable test data generator 110 determines whether there is parallel testing at the circuit block level for any of the test instances in the data string. If the example human-readable test data generator 110 determines that there is no parallel testing at the circuit block level for the test instances (block 1808: NO), the example human-readable test data generator 110 determines whether all test instances in the first data string have been processed (block 1812). If the example human-readable test data generator 110 determines that there is parallel testing at the circuit block level for one or more of the test instances (block 1808: YES), the example parallel test execution encoder 214 encodes the multiple circuit block test outcomes as a second data string (e.g., HRY_RAWSTR) for the test instance (block 1810).

[0104] In block 1812, the human-readable test data generator 110 determines whether all test instances in the first data string (e.g., TraceID) have been processed. If the human-readable test data generator 110 determines that all test instances in the data string have not been processed (block 1812: No), control returns to block 1806, where the human-readable test data generator 110 selects test instances in the first data string for checking parallel test execution.

[0105] If the human-readable test data generator 110 determines that all test instances have been processed (block 1812: YES), the example extended test report generator 216 outputs the first data string and the second data string to the second device (block 1814). Control returns to the example instructions and / or operations of block 1704 of Figure 17, and the example instructions and / or operations 1702 of Figure 18 end.

[0106] 19 is a flowchart depicting example machine-readable instructions and / or example operations 1704 that may be executed, instantiated, and / or performed by programmable circuitry to analyze test data. The example machine-readable instructions and / or example operations 1704 of FIG. 19 begin at block 1902, where the example test report receiver 802 receives test results in the form of one or more data strings for the DUT 102. At block 1904, the example data string decoder 804 decodes the data strings. As described above in conjunction with FIG. 8, the data string decoder 804 decodes the TraceID data string and the HRY_RAWSTR data string to provide detailed test result information for the tested circuit instance.

[0107] At block 1906, the data string decoder 804 generates test outcomes from the decoded data strings. At block 1908, the example hierarchical test generator 806 combines the test outcomes based on rules. In some examples, the rules may be hierarchical rules. As described above in conjunction with FIG. 8, the hierarchical rules may be based on a physical grouping of two or more circuit blocks based on the physical design of the DUT 102 or a user-defined logical grouping of two or more circuit blocks.

[0108] In block 1910, the hierarchical test generator 806 determines whether the hierarchical rule specifies a physical grouping or a logical grouping. If the exemplary hierarchical test generator 806 determines that the hierarchical rule specifies a physical grouping (block 1910: physical), the exemplary hierarchical test generator 806 generates a physical hierarchical tree (block 1912). In block 1914, the hierarchical test generator 806 creates a first level of the hierarchical tree based on the test outcomes of the test instances implemented on the circuit blocks of the DUT. In block 1916, the hierarchical test generator 806 forms a second level of the hierarchical tree based on the physical grouping of the test outcomes of the circuit blocks. In block 1918, the hierarchical test generator 806 forms a summary level of the hierarchical tree based on a combination of the first and second levels of the hierarchical tree. In block 1920, the exemplary indicator generator 810 generates an overall test indicator.

[0109] If the exemplary hierarchical test generator 806 determines that the hierarchical rules specify a logical grouping (block 1910: logical), the user-specified test configuration generator 808 generates a logical hierarchy (block 1922). In block 1924, the user-specified test configuration generator 808 creates a first level of the hierarchical tree based on the test outcomes of the test instances implemented on the circuit blocks. In block 1926, the user-specified test configuration generator 808 forms a second level of the hierarchical tree based on the logical grouping of the test outcomes of the circuit blocks. The logical grouping is defined by the user. In block 1928, the user-specified test configuration generator 808 forms an abstraction level of the hierarchical tree based on a combination of the first and second levels of the hierarchical tree. In block 1930, the indicator generator 810 generates user-defined test indicators.

[0110] In block 1932, the indicator generator 810 determines whether the user created a user-defined combination indicator. If the indicator generator 810 determines that the user did not create a combination indicator (block 1932: No), the test results generator 814 generates a test report using the overall test indicators from the physical hierarchy tree and / or the user-defined test indicators from the logical hierarchy tree (block 1936).

[0111] If the indicator generator 810 determines that the user has created a combined indicator (block 1932: yes), the indicator generator 810 combines the various test indicators based on the user definition (block 1934). In block 1938, the test results generator 814 generates a test report based on the combined indicators.

[0112] In block 1940, the bin category generator 812 determines whether to create a new classification for the DUT 102. A new bin classification is created if the DUT quality definition has changed. If the bin category generator 812 determines that a new bin classification needs to be created (block 1940: Yes), the bin category generator 812 generates a new classification for the DUT by combining the test indicators and the test outcomes (block 1942). As described above in conjunction with Figures 15 and 16, the new bin classification may be performed offline using the test results from the test indicators and the collected test outcomes. Control returns to the example instructions and / or operations of block 1706 of Figure 17, and the example instructions and / or operations 1704 of Figure 19 end.

[0113] If the bin category generator 812 determines that no new bin classifications need to be created (block 1940: No), control returns to the example instructions and / or operations of block 1706 of FIG. 17, and the example instructions and / or operations 1704 of FIG. 19 end.

[0114] Figure 20 is a block diagram of an example programmable circuitry configuration platform 2000 configured to execute and / or instantiate the example machine-readable instructions and / or example operations of Figures 17-19 to implement the human-readable test data generator 110 of Figure 2 and the human-readable test data analyzer 112 of Figure 8. The programmable circuitry configuration platform 2000 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a mobile phone, a smartphone, a tablet such as an iPad®), a personal digital assistant (PDA®), an internet appliance, or any other type of computing and / or electronic device.

[0115] The programmable circuitry platform 2000 of the illustrated example includes programmable circuitry 2012. The programmable circuitry 2012 of the illustrated example is hardware. For example, the programmable circuitry 2012 may be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers from any desired family or manufacturer. The programmable circuitry 2012 may be implemented by one or more semiconductor-based (e.g., silicon-based) devices. In this example, the programmable circuitry 2012 implements an exemplary test results interface 210, an exemplary test instance trace encoder 212, an exemplary parallel test execution encoder 214, an exemplary extended test report generator 216, and / or more generally a human-readable test data generator 110, an exemplary test report receiver 802, an exemplary data string decoder 804, an exemplary hierarchical test generator 806, an exemplary user-specified test configuration generator 808, an exemplary indicator generator 810, an exemplary bin category generator 812, an exemplary test result generator 814, and / or more generally a human-readable test data analyzer 112.

[0116] The programmable circuitry 2012 of the illustrated example includes local memory 2013 (e.g., cache, registers, etc.). The programmable circuitry 2012 of the illustrated example communicates with main memory 2014, 2016, which includes volatile memory 2014 and non-volatile memory 2016, via a bus 2018. The volatile memory 2014 may be implemented with synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of RAM device. The non-volatile memory 2016 may be implemented with flash memory and / or any other desired type of memory device. Access to the main memory 2014, 2016 of the illustrated example is controlled by a memory controller 2017. In some examples, the memory controller 2017 may be implemented by one or more integrated circuits, logic circuits, microcontrollers, or any other type of circuitry from any desired family or manufacturer to manage the flow of data to and from the main memory 2014, 2016. In the example of FIG. 20, the exemplary hierarchical tree database 816 may be implemented by the main memory 2014, 2016 and / or the mass storage disk or device 2028.

[0117] The programmable circuit configuration platform 2000 of the depicted example also includes interface circuitry 2020. The interface circuitry 2020 may be implemented by hardware according to any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB) interface, a Bluetooth® interface, a Near Field Communication (NFC) interface, a Peripheral Component Interconnect (PCI) interface, and / or a Peripheral Component Interconnect Express (PCIe) interface. In the example of FIG. 20 , the example connection system 114 and the example communication interface 116 may be implemented by the interface circuitry 2020.

[0118] In the illustrated example, one or more input devices 2022 are connected to the interface circuitry 2020. The input devices 2022 allow a user (e.g., a human user, a machine user, etc.) to input data and / or commands into the programmable circuitry 2012. The input devices 2022 may be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, buttons, a mouse, a touchscreen, a trackpad, a trackball, an isopoint device, and / or a voice recognition system.

[0119] One or more output devices 2024 are also connected to the interface circuitry 2020 of the illustrated example. The output device(s) 2024 may be implemented by, for example, a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-plane switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. The interface circuitry 2020 of the illustrated example therefore typically includes a graphics driver card, a graphics driver chip, and / or graphics processor circuitry such as a GPU.

[0120] The interface circuitry 2020 of the illustrated example also includes communication devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external equipment (e.g., any type of computing device) over the network 2026. Communication may be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a non-line-of-sight wireless communication system, a line-of-sight wireless communication system, a cellular telephone system, an optical connection, etc.

[0121] The programmable circuit configuration platform 2000 of the depicted example also includes one or more mass storage disks or devices 2028 for storing firmware, software, and / or data. Examples of such mass storage disks or devices 2028 include magnetic storage devices (e.g., floppy disks, drives, HDDs, etc.), optical storage devices (e.g., Blu-ray disks, CDs, DVDs, etc.), RAID systems, and / or solid-state storage disks or devices such as flash memory devices and / or SSDs.

[0122] Machine-readable instructions 2032 that may be implemented by the machine-readable instructions of Figures 17-19 may be stored on at least one non-transitory computer-readable storage medium, such as mass storage device 2028, volatile memory 2014, non-volatile memory 2016, and / or a CD or DVD, which may be removable.

[0123] FIG. 21 is a block diagram of an exemplary implementation of the programmable circuitry 2012 of FIG. 20. In this example, the programmable circuitry 2012 of FIG. 20 is implemented by a microprocessor 2100. For example, the microprocessor 2100 may be a general-purpose microprocessor (e.g., general-purpose microprocessor circuitry). The microprocessor 2100 executes some or all of the machine-readable instructions of the flowcharts of FIGS. 17-19 to effectively instantiate the circuitry of FIG. 2 as logic circuitry and perform operations corresponding to those machine-readable instructions. In some such examples, the circuitry of FIGS. 2 and 8, in combination with machine-readable instructions, is instantiated by hardware circuitry of the microprocessor 2100. For example, the microprocessor 2100 may be implemented by multi-core hardware circuitry, such as a CPU, DSP, GPU, XPU, etc. While any number of the exemplary cores 2102 (e.g., one core) may be included, the microprocessor 2100 of this example is a multi-core semiconductor device including N cores. The cores 2102 of the microprocessor 2100 may operate independently or may cooperate to execute machine-readable instructions. For example, a firmware program, an embedded software program, or machine code corresponding to a software program may be executed by one of the cores 2102 or may be executed at the same or different times by multiple cores 2102. In some examples, the firmware program, the embedded software program, or machine code corresponding to a software program is divided into threads and executed in parallel by two or more cores 2102. The software program may correspond to some or all of the machine-readable instructions and / or operations represented by the flowcharts of FIGS. 17-19.

[0124] The cores 2102 may communicate via a first exemplary bus 2104. In some examples, the first bus 2104 may be implemented by a communication bus to achieve communications associated with one of the cores 2102. For example, the first bus 2104 may be implemented by at least one of an Inter-Integrated Circuit (I2C) bus, a Serial Peripheral Interface (SPI) bus, a PCI bus, or a PCIe bus. Additionally or alternatively, the first bus 2104 may be implemented by any other type of computing or electrical bus. The cores 2102 may obtain data, instructions, and / or signals from one or more external devices via exemplary interface circuitry 2106. The cores 2102 may output data, instructions, and / or signals to one or more external devices via the interface circuitry 2106. The cores 2102 in this example include exemplary local memory 2120 (e.g., a level 1 (L1) cache, which may be divided into an L1 data cache and an L1 instruction cache), but the microprocessor 2100 also includes exemplary shared memory 2110 (e.g., a level 2 (L2) cache) that may be shared by the cores for fast access to data and / or instructions. Data and / or instructions may be transferred (e.g., shared) by writing to and / or reading from the shared memory 2110. The local memories 2120 of each of the cores 2102 and the shared memory 2110 may be part of a hierarchy of storage devices that includes multiple levels of cache memories and main memories (e.g., main memories 2014, 2016 in FIG. 20). Typically, higher-level memories in the hierarchy exhibit slower access times and smaller storage capacities than lower-level memories. Changes in the cache hierarchy at various levels are managed (e.g., coordinated) by a cache coherency policy.

[0125] Each core 2102 may be referred to as a CPU, DSP, GPU, etc., or any other type of hardware circuitry. Each core 2102 includes control unit circuitry 2114, arithmetic and logic (AL) circuitry (sometimes referred to as an ALU) 2116, multiple registers 2118, local memory 2120, and a second exemplary bus 2122. Other configurations may exist. For example, each core 2102 may include vector unit circuitry, single instruction multiple data (SIMD) unit circuitry, load / store unit (LSU) circuitry, branch / jump unit circuitry, floating point unit (FPU) circuitry, etc. The control unit circuitry 2114 includes semiconductor-based circuitry configured to control (e.g., coordinate) data movement within the corresponding core 2102. The AL circuitry 2116 includes semiconductor-based circuitry configured to perform one or more mathematical and / or logical operations on data within the corresponding core 2102. In some examples, the AL circuitry 2116 performs integer-based operations. In other examples, the AL circuitry 2116 also performs floating-point operations. In yet other examples, the AL circuitry 2116 may include first AL circuitry that performs integer-based operations and second AL circuitry that performs floating-point operations. In some examples, the AL circuitry 2116 may be referred to as an arithmetic logic unit (ALU).

[0126] The registers 2118 are semiconductor-based structures for storing data and / or instructions, such as results of one or more of the operations performed by the AL circuitry 2116 of the corresponding core 2102. For example, the registers 2118 may include vector registers, SIMD registers, general-purpose registers, flag registers, segment registers, machine-specific registers, instruction pointer registers, control registers, debug registers, memory management registers, machine check registers, etc. The registers 2118 may be arranged in banks as shown in FIG. 21 . Alternatively, the registers 2118 may be organized in any other arrangement, format, or structure, such as by being distributed throughout the core 2102 to reduce access time. The second bus 2122 may be implemented by at least one of an I2C bus, an SPI bus, a PCI bus, or a PCIe bus.

[0127] Each core 2102 and / or, more generally, microprocessor 2100 may include additional and / or alternative structures to those shown and described above. For example, one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more convergent / common mesh stops (CMSs), one or more shifters (e.g., barrel shifters), and / or other circuitry may be present. Microprocessor 2100 is a semiconductor device fabricated to include many transistor interconnects to implement the structures described above in one or more integrated circuits (ICs) contained in one or more packages.

[0128] The microprocessor 2100 may include and / or interface with one or more accelerators (e.g., acceleration circuitry, hardware accelerators, etc.). In some examples, an accelerator is implemented by logic circuitry to perform a particular task more quickly and / or efficiently than can be performed by a general-purpose processor. Examples of accelerators include ASICs and FPGAs, such as those discussed herein. GPUs, DSPs, and / or other programmable devices may also be accelerators. An accelerator may be on-board the microprocessor 2100 in the same chip package as the microprocessor 2100 and / or in one or more packages separate from the microprocessor 2100.

[0129] Figure 22 is a block diagram of another example implementation of the programmable circuitry 2012 of Figure 20. In this example, the programmable circuitry 2012 is implemented by FPGA circuitry 2200. For example, the FPGA circuitry 2200 may be implemented by an FPGA. The FPGA circuitry 2200 may be used, for example, to perform operations that might otherwise be performed by the example microprocessor 2100 of Figure 21 executing corresponding machine-readable instructions. However, once configured, the FPGA circuitry 2200 instantiates the operations and / or functions corresponding to the machine-readable instructions in hardware and, therefore, may often perform the operations / functions faster than could be performed by a general-purpose microprocessor executing corresponding software.

[0130] More specifically, in contrast to the above-described microprocessor 2100 of FIG. 21 (which is a general-purpose device whose interconnect and logic circuitry are fixed once fabricated, though it can be programmed to execute some or all of the machine-readable instructions represented by the flowcharts of FIGS. 17-19), the example FPGA circuitry 2200 of FIG. 22 includes interconnect and logic circuitry that can be configured, structured, programmed, and / or interconnected in different ways after fabrication, e.g., to instantiate some or all of the operations / functions corresponding to the machine-readable instructions represented by the flowcharts of FIGS. 17-19. In particular, FPGA circuitry 2200 can be thought of as an array of logic gates, interconnects, and switches. The switches can be programmed to change the manner in which the logic gates are interconnected by the interconnects, effectively forming one or more dedicated logic circuits (unless FPGA circuitry 2200 is reprogrammed). The configured logic circuits allow the logic gates to cooperate in different ways to perform different operations on data received by the input circuitry. Those operations may correspond to some or all of the instructions (e.g., software and / or firmware) represented by the flowcharts of Figures 17-19. As such, FPGA circuitry 2200 may be configured and / or constructed to effectively instantiate some or all of the operations / functions corresponding to the machine-readable instructions of the flowcharts of Figures 17-19 as special-purpose logic circuitry to perform the operations / functions corresponding to those software instructions in a dedicated manner similar to an ASIC. As such, FPGA circuitry 2200 may perform the operations / functions corresponding to some or all of the machine-readable instructions of Figures 17-19 faster than a general-purpose microprocessor could.

[0131] In the example of FIG. 22 , the FPGA circuitry 2200 is configured and / or constructed in response to being programmed (and / or reprogrammed one or more times) based on a binary file. In some examples, the binary file may be compiled and / or generated based on instructions in a hardware description language (HDL), such as Lucid, Very High Speed ​​Integrated Circuit (VHSIC) Hardware Description Language (VHDL), or Verilog. For example, a user (e.g., a human user, a machine user, etc.) may write code or a program corresponding to one or more operations / functions in the HDL. The code / program may be translated into a low-level language as needed. The code / program (e.g., code / program in the low-level language) may be converted into a binary file (e.g., by a compiler, a software application, etc.). In some examples, the FPGA circuitry 2200 of FIG. 22 may access and / or load the binary file, such that the FPGA circuitry 2200 of FIG. 22 is configured and / or constructed to perform one or more operations / functions. For example, a binary file may be implemented with a bitstream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions accessible to the FPGA circuitry 2200 of FIG. 22 to cause configuration and / or construction of the FPGA circuitry 2200 of FIG. 22 or portions thereof.

[0132] In some examples, a binary file is compiled, generated, translated, and / or otherwise output from a uniform software platform utilized to program the FPGA. For example, the uniform software platform may translate first instructions (e.g., code or program) corresponding to one or more operations / functions in a high-level language (e.g., C, C++, Python, etc.) into second instructions corresponding to one or more operations / functions in an HDL. In some such examples, a binary file is compiled, generated, and / or otherwise output from the uniform software platform based on the second instructions. In some examples, the FPGA circuitry 2200 of FIG. 22 may access and / or load the binary file such that the FPGA circuitry 2200 of FIG. 22 is configured and / or constructed to perform one or more operations / functions. For example, a binary file may be implemented with a bitstream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions accessible to the FPGA circuitry 2200 of FIG. 22 to cause configuration and / or construction of the FPGA circuitry 2200 of FIG. 22 or portions thereof.

[0133] The FPGA circuitry 2200 of FIG. 22 includes exemplary input / output (I / O) circuitry 2202 for obtaining and / or outputting data from exemplary configuration circuitry 2204 and / or external hardware 2206. For example, the configuration circuitry 2204 may be implemented by interface circuitry that may obtain a binary file that may be implemented with bitstreams, data, and / or machine-readable instructions to configure the FPGA circuitry 2200 or a portion thereof. In some such examples, the configuration circuitry 2204 may obtain the binary file from a user, a machine (e.g., hardware circuitry (e.g., programmable or dedicated circuitry) that may implement an artificial intelligence / machine learning (AI / ML) model to generate the binary file, etc., and / or any combination thereof). In some examples, the external hardware 2206 may be implemented by external hardware circuitry. For example, the external hardware 2206 may be implemented by the microprocessor 2100 of FIG. 21.

[0134] The FPGA circuitry 2200 also includes an array of exemplary logic gate circuitry 2208, a plurality of exemplary configurable interconnects 2210, and exemplary storage circuitry 2212. The logic gate circuitry 2208 and configurable interconnects 2210 are configurable to instantiate one or more operations / functions that may correspond to at least a portion of the machine-readable instructions and / or other desired operations of FIGS. 17-19. The logic gate circuitry 2208 shown in FIG. 22 is assembled in blocks or groups. Each block includes semiconductor-based electrical structures that may be configured into a logic circuit. In some examples, the electrical structures include logic gates (e.g., And gates, Or gates, Nor gates, etc.) that provide the basic building blocks of a logic circuit. Electrically controllable switches (e.g., transistors) are present in each of the logic gate circuitry 2208, enabling the configuration of the electrical structures and / or logic gates to form a circuit to perform a desired operation / function. Logic gate circuitry 2208 may include other electrical structures, such as look-up tables (LUTs), registers (eg, flip-flops or latches), multiplexers, and the like.

[0135] The configurable interconnect 2210 in the illustrated example is a conductive path, trace, via, or the like that may include electrically controllable switches (e.g., transistors) whose state can be changed by programming (e.g., using an HDL instruction language) to activate or deactivate one or more connections between one or more of the logic gate circuitry 2208 to program a desired logic circuit.

[0136] The storage circuitry 2212 in the illustrated example is configured to store one or more results of the operations performed by the corresponding logic gates. The storage circuitry 2212 may be implemented by registers, etc. In the illustrated example, the storage circuitry 2212 is distributed among the logic gate circuitry 2208 for ease of access and increased execution speed.

[0137] The example FPGA circuitry 2200 of FIG. 22 also includes example dedicated operating circuitry 2214. In this example, the dedicated operating circuitry 2214 includes special-purpose circuitry 2216 that may be called upon to implement commonly used functions to avoid the need to program those functions in the field. Examples of such special-purpose circuitry 2216 include memory (e.g., DRAM) controller circuitry, PCIe controller circuitry, clock circuitry, transceiver circuitry, memory, and multiplier-accumulator circuitry. Other types of special-purpose circuitry may also be present. In some examples, the FPGA circuitry 2200 may also include example general-purpose programmable circuitry 2218, such as an example CPU 2220 and / or an example DSP 2222. There may additionally or alternatively be other general-purpose programmable circuitry 2218 that can be programmed to perform other operations, such as a GPU, XPU, etc.

[0138] 21 and 22 show two example implementations of the programmable circuitry 2012 of FIG. 20, many other approaches are envisioned. For example, the FPGA circuitry may include an on-board CPU, such as one or more of the example CPUs 2220 of FIG. 21. Thus, the programmable circuitry 2012 of FIG. 20 may additionally be implemented by combining at least the example microprocessor 2100 of FIG. 21 and the example FPGA circuitry 2200 of FIG. 22. In some such hybrid examples, to perform first operations / functions, one or more cores 2102 of FIG. 21 may execute a first portion of the machine-readable instructions represented by the flowcharts of FIGS. 17-19, the FPGA circuitry 2200 of FIG. 22 may be configured and / or constructed to perform second operations / functions corresponding to a second portion of the machine-readable instructions represented by the flowcharts of FIGS. 17-19, and / or the ASIC may be configured and / or constructed to perform third operations / functions corresponding to a third portion of the machine-readable instructions represented by the flowcharts of FIGS. 17-19.

[0139] 2 and 8 may be instantiated at the same time or at different times. For example, the same and / or different portions of microprocessor 2100 of FIG. 21 may be programmed to execute portions of the machine-readable instructions at the same time and / or at different times. In some examples, the same and / or different portions of FPGA circuitry 2200 of FIG. 22 may be configured and / or constructed to perform operations / functions corresponding to portions of the machine-readable instructions at the same time and / or at different times.

[0140] In some examples, some or all of the circuitry of Figures 2 and 8 may be instantiated in one or more threads, e.g., executing simultaneously and / or serially. For example, microprocessor 2100 of Figure 21 may execute machine-readable instructions in one or more threads, executing simultaneously and / or serially. In some examples, FPGA circuitry 2200 of Figure 22 may be configured and / or constructed to perform operations / functions simultaneously and / or serially. Also, in some examples, some or all of the circuitry of Figures 2 and 8 may be implemented within one or more virtual machines and / or containers executing on microprocessor 2100 of Figure 21.

[0141] In some examples, the programmable circuitry 2012 of Figure 20 may be in one or more packages. For example, the microprocessor 2100 of Figure 21 and / or the FPGA circuitry 2200 of Figure 22 may be in one or more packages. In some examples, an XPU may be implemented by the programmable circuitry 2012 of Figure 20, which may be in one or more packages. For example, an XPU may include a CPU (e.g., the microprocessor 2100 of Figure 21, the CPU 2220 of Figure 22, etc.) in one package, a DSP (e.g., the DSP 2222 of Figure 22) in another package, a GPU in yet another package, and an FPGA (e.g., the FPGA circuitry 2200 of Figure 22) in yet another package.

[0142] A block diagram illustrating an exemplary software distribution platform 2305 for distributing software, such as the exemplary machine-readable instructions 2032 of FIG. 20, to other hardware devices (e.g., hardware devices owned and / or operated by a third party other than the owner and / or operator of the software distribution platform) is shown in FIG. 23. The exemplary software distribution platform 2305 may be implemented by any computer server, data facility, cloud service, etc. capable of storing and transmitting software to other computing devices. The third party may be a customer of the entity owning and / or operating the software distribution platform 2305. For example, the entity owning and / or operating the software distribution platform 2305 may be a developer, seller, and / or licensor of software, such as the exemplary machine-readable instructions 2032 of FIG. 20. The third party may be a consumer, user, retailer, OEM, etc., that purchases and / or licenses the software for use and / or resale and / or sublicense. In the illustrated example, the software distribution platform 2305 includes one or more servers and one or more storage devices. The storage device stores machine-readable instructions 2032, which may correspond to the example machine-readable instructions of Figures 17-19, as described above. One or more servers of the example software distribution platform 2305 are in communication with the example network 2310, which may correspond to the Internet and / or any one or more of any of the example networks described above. In some examples, the one or more servers respond to requests to transmit software to a requestor as part of a commercial transaction. Payment for the distribution, sale, and / or licensing of the software may be processed by one or more servers of the software distribution platform and / or a third-party payment entity. The servers enable purchasers and / or licensors to download the machine-readable instructions 2032 from the software distribution platform 2305.17-19 may be downloaded to the example programmable circuit configuration platform 2000, which executes the machine-readable instructions 2032 to implement the human-readable test data generator 110 and the human-readable test data analyzer 112. In some examples, one or more servers of the software distribution platform 2305 periodically provide, transmit, and / or force updates to the software (e.g., the example machine-readable instructions 2032 of FIG. 20) to ensure that improvements, patches, updates, etc. are distributed and applied to the software at end-user devices. Although referred to above as software, the distributed "software" may alternatively be firmware.

[0143] The terms "including" and "comprising" (and all their forms and tenses) are used herein as open-ended terms. Thus, whenever a claim uses any form of "include" or "comprise" (e.g., "comprises," "includes," "comprising," "including," "having," etc.) as a preamble or within any type of claim recitation, it is to be understood that additional elements, terms, etc. may be present without departing from the scope of the corresponding claim or recitation. As used herein, the phrase "at least" is open-ended in the same way that the terms "including" and "comprising" are open-ended when used, for example, as a transitional phrase in a claim preamble. When used in the form of, for example, A, B, and / or C, the term "and / or" refers to any combination or subset of A, B, and C, such as: (1) A alone; (2) B alone; (3) C alone; (4) A and B; (5) A and C; (6) B and C; (7) A, B, and C. When used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, when used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. When used herein in the context of describing the performance or execution of a process, instruction, action, activity, etc., the phrase "at least one of A and B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.Similarly, when used herein in the context of describing the performance or execution of a process, instruction, action, activity, or the like, the phrase "at least one of A or B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0144] As used herein, singular references (e.g., "a," "an," "first," "second," etc.) do not exclude a plurality. As used herein, the term "a" or "an" object refers to one or more of that object. The terms "a" (or "an"), "one or more," and "at least one" are used interchangeably herein. Furthermore, although individually listed, multiple means, elements, or actions may be implemented, for example, by the same entity or object. In addition, although individual features may be included in different examples or claims, they may in some cases be combined, and their inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0145] As used herein, unless otherwise specified, the term "above" describes the relationship of two portions to the Earth. A first portion is above a second portion if the second portion has at least one portion between the Earth and the first portion. Similarly, as used herein, a first portion is "below" a second portion if the first portion is closer to the Earth than the second portion. As described above, a first portion can be above or below a second portion, with one or more of: with other portions therebetween; without other portions therebetween; with the first and second portions touching; or without the first and second portions directly contacting each other.

[0146] Notwithstanding the above, when referring to a semiconductor device (e.g., a transistor), a semiconductor die including a semiconductor device, and / or an integrated circuit (IC) package including a semiconductor die during fabrication or manufacturing, “above” does not refer to the earth, but instead to the underlying substrate upon which the associated components are fabricated, assembled, mounted, supported, or otherwise provided. Thus, as used herein, unless otherwise specified or implied by context, a first component (e.g., a transistor or other semiconductor device) within a semiconductor die is “above” a second component within the semiconductor die if, during fabrication / manufacturing, the first component is further away from the substrate (e.g., a semiconductor wafer) upon which the two components are fabricated or otherwise provided than the second component. Similarly, unless otherwise specified or implied by context, a first component (e.g., a semiconductor die) within an IC package is “above” a second component within the IC package if, during fabrication, the first component is further away from the printed circuit board (PCB) upon which the IC package will be mounted or attached. It is understood that a semiconductor device is often used in an orientation different from its orientation during fabrication. Thus, when referring to a semiconductor device (e.g., a transistor), a semiconductor die that includes the semiconductor device, and / or an integrated circuit (IC) package that includes the semiconductor die during use, the definition of "above" in the previous paragraph (i.e., the term "above" describes the relationship of two parts relative to the Earth) may be determined based on the context of use.

[0147] As used in this patent, a description of any part (e.g., a layer, film, area, region, or plate) being on (e.g., positioned on, located on, disposed on, or formed on, etc.) another part in any manner indicates that the referenced part is either in contact with the other part, or that the referenced part is on top of the other part with one or more intermediate parts located therebetween.

[0148] As used herein, unless otherwise specified, connection references (e.g., attached, coupled, connected, and joined) may include intermediate members between, and / or relative movement between, the elements referenced by the connection reference. As such, a connection reference does not necessarily infer that two elements are directly connected and / or in a fixed relationship to one another. As used herein, the phrase "any part in contact with" another part is defined to mean that there is no intermediate part between the two parts.

[0149] Unless otherwise specified, descriptors such as "first," "second," "third," etc. are used herein without negatively or otherwise implying any sense of priority, physical order, placement within a list, and / or ordering in any way, but are merely used as labels and / or arbitrary names to distinguish elements to facilitate understanding of the disclosed examples. In some instances, the descriptor "first" may be used to refer to an element in the detailed description, while in the claims the same element may be referred to using a different descriptor such as "second" or "third." In such instances, it should be understood that such descriptors are merely used to separately identify those elements within the context of the description (e.g., in the claims) (wherein the elements may otherwise share the same name, for example).

[0150] As used herein, "approximately" and "about" modify their objects / values ​​to recognize the existence of potential variations that occur in real-world applications. For example, "approximately" and "about" may modify dimensions that may not be exact due to manufacturing tolerances and / or other real-world imperfections, as understood by those skilled in the art. For example, "approximately" and "about" may indicate that such dimensions may be within a tolerance range of + / - 10%, unless otherwise specified herein.

[0151] As used herein, "substantially real-time" refers to occurring nearly instantaneously, although it is recognized that there may be real-world delays due to computation time, transmission, etc. Thus, unless otherwise specified, "substantially real-time" refers to real-time plus one second.

[0152] As used herein, the term "communicate" (including variations thereof) encompasses direct communication and / or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) and / or constant communication, and additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-time events.

[0153] As used herein, "programmable circuitry" is defined to include (i) one or more special-purpose electrical circuits (e.g., application-specific circuits (ASICs)) that have a structure to perform specific operations and that include one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more general-purpose semiconductor-based electrical circuits that are programmable with instructions to perform specific functions and / or operations and that include one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of programmable circuitry include programmable microprocessors such as a central processing unit (CPU) that can execute first instructions to perform one or more operations and / or functions; a field programmable gate array (FPGA) that can be programmed with second instructions to cause the configuration and / or structure of the FPGA to instantiate one or more operations and / or functions corresponding to the first instructions; a graphics processor unit (GPU) that can execute first instructions to perform one or more operations and / or functions; a digital signal processor (DSP), XPU, network processing unit (NPU) that can execute first instructions to perform one or more operations and / or functions; one or more microcontrollers that can execute first instructions to perform one or more operations and / or functions; and / or integrated circuits such as application specific integrated circuits (ASICs). For example, an XPU may be implemented by a heterogeneous computing system that includes multiple types of programmable circuitry (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more NPUs, one or more DSPs, etc., and / or any combination thereof) and an orchestration technique (e.g., an application programming interface (API)) that can assign computing tasks to any of the multiple types of programmable circuitry that are suitable for or available to perform the computing task.

[0154] As used herein, one or more circuit elements, e.g., an integrated circuit / circuitry, is defined as one or more semiconductor packages that include transistors, capacitors, inductors, resistors, current paths, diodes, etc. For example, an integrated circuit may be implemented as one or more of an ASIC, an FPGA, a chip, a microchip, a programmable circuitry, a semiconductor substrate coupling multiple circuit elements, a system on chip (SoC), etc.

[0155] From the foregoing, it will be appreciated that exemplary systems, apparatus, products, and methods for generating human-readable semiconductor product yield indicators have been disclosed. The disclosed systems, apparatus, products, and methods improve the efficiency of computing device usage by implementing a human-readable test data generation apparatus for generating human-readable test data and a customizable human-readable test data analysis apparatus for facilitating analysis of the semiconductor yield test data using the test indicators. The disclosed systems, apparatus, products, and methods are therefore related to one or more improvements in the operation of machines, such as computers or other electronic and / or mechanical devices.

[0156]

[0010] Exemplary methods, apparatus, systems, and products for generating a semiconductor product yield indicator are disclosed herein. Further examples and combinations thereof include the following:

[0011] Example 1 includes a non-transitory machine-readable storage medium comprising instructions that cause at least one processor circuit to perform at least: encoding a data string representing a sequence of test results corresponding to a plurality of test instances executed by automatic test equipment on a first device under test; and outputting the data string to a second device.

[0157] Example 2 includes the non-transitory machine-readable storage medium of example 1, wherein the data string is based on a plurality of substrings, each one of the substrings corresponding to a respective one of the test instances.

[0158] Example 3 includes the non-transitory machine-readable storage medium of example 2, wherein the instructions cause one or more of the at least one processor circuit to encode the plurality of substrings into the data string, the data string being a single alphanumeric character or a sequence of alphanumeric characters having fewer characters than the plurality of substrings.

[0159] Example 4 includes the non-transitory machine-readable storage medium of any one of Examples 2 or 3, wherein a first one of the substrings corresponds to a first one of the test instances and the first one of the substrings includes a first sequence of one or more characters to identify the first one of the test instances and a second sequence of one or more characters representing a first one of the test results corresponding to the first one of the test instances.

[0160] Example 5 includes the non-transitory machine-readable storage medium of Example 4, wherein the data string is a first data string, and the instructions cause the one or more of the at least one processor circuit to generate a second data string associated with a first of the substrings of the first data string, the second data string representing test outcomes resulting from a plurality of circuit blocks of the first device under test being tested based on the first of the test instances.

[0161] Example 6 includes the non-transitory machine-readable storage medium of example 5, wherein the second data string includes respective characters representing test outcomes for each of the plurality of circuit blocks, and a position of the respective characters in the second data string identifies a respective one of the circuit blocks.

[0162] Example 7 includes an apparatus comprising interface circuitry, machine-readable instructions, and at least one processor circuit programmed by the machine-readable instructions to encode a data string representing a sequence of test results corresponding to a plurality of test instances executed by automatic test equipment on a first device under test, and to output the data string to a second device.

[0163] Example 8 includes the apparatus of example 7, wherein the data string is based on a plurality of substrings, each one of the substrings corresponding to a respective one of the test instances.

[0164] Example 9 includes the apparatus of example 8, wherein the at least one processor circuit encodes the plurality of substrings into the data string, and the data string is a single alphanumeric character or a sequence of alphanumeric characters having fewer characters than the plurality of substrings.

[0165] Example 10 includes the apparatus of any one of Examples 7-9, wherein a first one of the substrings corresponds to a first one of the test instances and the first one of the substrings includes a first sequence of one or more characters to identify the first one of the test instances and a second sequence of one or more characters representing a first one of the test results corresponding to the first one of the test instances.

[0166] Example 11 includes the apparatus of example 10, wherein the data string is a first data string, and one or more of the at least one processor circuits generate a second data string associated with a first of the substrings of the first data string, the second data string representing a plurality of test outcomes resulting from a plurality of circuit blocks of the first device under test that are being tested based on the first of the test instances.

[0167] Example 12 includes the apparatus of example 11, wherein the second data string includes respective characters representing test outcomes for each of the plurality of circuit blocks, and a position of the respective characters in the second data string identifies the respective one of the circuit blocks.

[0168] Example 13 includes a non-transitory machine-readable storage medium comprising instructions to cause at least one processor circuit to perform at least steps of: decoding a data string including a plurality of sub-strings representing a plurality of test outcomes for a plurality of test instances performed on a plurality of circuit blocks of a device under test; combining at least two of the test outcomes based on a hierarchical rule to determine test indicators for two or more groups of the circuit blocks; and outputting a test report based on the test indicators.

[0169] Example 14 includes the non-transitory machine-readable storage medium of Example 13, wherein the hierarchical rules specify at least one of 1) a physical grouping of the two or more circuit blocks based on a physical design of the device under test, or 2) a user-defined logical grouping of the two or more circuit blocks.

[0170] Example 15 includes the non-transitory machine-readable storage medium of Example 14, wherein the rules specify the physical grouping based on a physical hierarchy, the physical hierarchy including a first hierarchical level representing a first of the test outcomes of a first of the test instances executed on the two or more of the circuit blocks of the device under test, and a second hierarchical level representing the first combination of the test outcomes used to determine the test indicator for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the physical design of the device under test.

[0171] Example 16 includes the non-transitory machine-readable storage medium of any one of Examples 14 or 15, wherein the rules specify the logical grouping based on a logical hierarchy, the logical hierarchy including a first hierarchical level representing a first of the test outcomes of a first of the test instances performed on the two or more of the circuit blocks of the device under test, and a second hierarchical level representing the first combination of the test outcomes used to determine the test indicator for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the user definition.

[0172] Example 17 includes the non-transitory machine-readable storage medium of any one of Examples 13 to 16, wherein the at least two of the test outcomes are a first group of test outcomes, the test indicator is a first test indicator, the group of two or more of the circuit blocks is a first group of the circuit blocks, and the instructions cause one or more of the at least one processor circuit to perform the following steps: combine a second group of the test outcomes based on the hierarchical rule to determine a second test indicator for the second group of circuit blocks; and determine a combined indicator based on a combination of the first test indicator and the second test indicator.

[0173] Example 18 includes the non-transitory machine-readable storage medium of Example 17, wherein the instructions cause one or more of the at least one processor circuit to perform a procedure for generating a classification of the device under test, the classification being based on the combined indicator.

[0174] Example 19 includes the non-transitory machine-readable storage medium of Example 18, wherein the instructions cause one or more of the at least one processor circuit to perform a procedure for generating a classification of the device under test offline.

[0175] Example 20 includes the non-transitory machine-readable storage medium of any one of Examples 16 to 19, wherein the hierarchical rule is a first hierarchical rule, the test report is a first test report, and the instructions cause one or more of the at least one processor circuit to perform steps of obtaining a second hierarchical rule after determining the first test report, and outputting a second test report based on the second hierarchical rule and the data string without running an additional test instance on the device under test.

[0176] Example 21 includes the apparatus of any one of Examples 18 or 19, wherein the rules specify the logical grouping based on a logical hierarchy, the logical hierarchy including a first hierarchical level representing a first of the test outcomes of a first of the test instances performed on the two or more of the circuit blocks of the device under test, and a second hierarchical level representing the first combination of the test outcomes used to determine the test indicator for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the user definition.

[0177] Example 22 includes the apparatus of any one of Examples 18 to 21, wherein the at least two of the test outcomes are a first group of the test outcomes, the test indicator is a first test indicator, the group of two or more of the circuit blocks is a first group of the circuit blocks, and the programmable circuit configuration combines the second group of test outcomes based on the hierarchical rule to determine a second test indicator for the second group of the circuit blocks, and determines a combined indicator based on a combination of the first test indicator and the second test indicator.

[0178] Example 23 includes the apparatus of any one of examples 18-22, wherein the programmable circuitry generates a classification of the device under test, the classification based on the combination indicator.

[0179] Example 24 includes the apparatus of any one of examples 18-23, wherein the programmable circuitry generates the classification of the device under test offline.

[0180] Example 25 includes a method comprising: encoding a data string representing a sequence of test results corresponding to multiple test instances performed by automatic test equipment on a first device under test; and outputting the data string to a second device.

[0181] Example 26 includes the method of example 25, wherein the data string includes a plurality of substrings separated by delimiters, each one of the substrings corresponding to a respective one of the test instances.

[0182] Example 27 includes the method of any one of Examples 25 or 26, wherein a first one of the substrings corresponds to a first one of the test instances and the first one of the substrings includes a first sequence of one or more characters to identify the first one of the test instances and a second sequence of one or more characters representing a first one of the test results corresponding to the first one of the test instances.

[0183] Example 28 includes the method of any one of Examples 25 to 27, wherein the data string is a first data string, and further includes generating a second data string for association with a first of the substrings of the first data string, the second data string representing a plurality of test outcomes resulting from a plurality of circuit blocks of the first device under test being tested based on the first of the test instances.

[0184] Example 29 includes the method of any one of Examples 25 to 28, wherein the second data string includes respective characters representing test results for each of the plurality of circuit blocks, and the position of each of the characters in the second data string identifies the respective one of the circuit blocks.

[0185] Example 30 includes a method comprising: decoding a data string including multiple sub-strings representing multiple test outcomes for multiple test instances performed on multiple circuit blocks of a device under test; combining at least two of the test outcomes based on a hierarchical rule to determine test indicators for two or more groups of the circuit blocks; and outputting a test report based on the test indicators.

[0186] Example 31 includes the method of example 30, in which the hierarchical rules specify at least one of 1) a physical grouping of the two or more circuit blocks based on a physical design of the device under test, or 2) a logical grouping of the two or more circuit blocks based on a user definition.

[0187] Example 32 includes the method of any one of Examples 30 or 31, wherein the rules specify the physical grouping based on a physical hierarchy, the physical hierarchy including a first hierarchical level representing a first of the test outcomes of a first of the test instances performed on the two or more of the circuit blocks of the device under test, and a second hierarchical level representing the first combination of the test outcomes used to determine the test indicator for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the physical design of the device under test.

[0188] Example 33 includes the method of any one of Examples 30 to 32, wherein the rules specify the logical grouping based on a logical hierarchy, the logical hierarchy including a first hierarchical level representing a first of the test outcomes of a first of the test instances performed on the two or more of the circuit blocks of the device under test, and a second hierarchical level representing a first combination of the test outcomes used to determine the test indicator for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the user definition.

[0189] Example 34 includes the method of any one of Examples 30 to 33, wherein the at least two of the test outcomes are a first group of the test outcomes, the test indicator is a first test indicator, the group of two or more of the circuit blocks is a first group of the circuit blocks, and the method further includes combining the second group of test outcomes based on the hierarchical rule to determine a second test indicator for the second group of circuit blocks, and determining a combined indicator based on a combination of the first test indicator and the second test indicator.

[0190] Example 35 includes the method of any one of examples 30-34, further including generating a classification of the device under test, the classification being based on the combined indicator.

[0191] Example 36 includes the method of example 35, further including generating the classification of the device under test.

[0192] The following claims are hereby incorporated by reference into this detailed description. Although certain exemplary systems, apparatus, products, and methods are disclosed herein, the scope of coverage of this patent is not limited thereto. To the contrary, this patent covers all systems, apparatus, products, and methods that fall even slightly within the scope of the claims. Other possible items [Item 1] At least one processor circuit includes at least: encoding a data string representing a sequence of test results corresponding to a plurality of test instances performed by the automated test equipment on the first device under test; and outputting said data string to a second device; A non-transitory machine-readable storage medium comprising instructions to cause [Item 2] Item 10. The non-transitory machine-readable storage medium of item 1, wherein the data string is based on a plurality of substrings, each one of the substrings corresponding to a respective one of the test instances. [Item 3] Item 3. The non-transitory machine-readable storage medium of item 2, wherein the instructions cause one or more of the at least one processor circuit to encode the plurality of substrings into the data string, the data string being a single alphanumeric character or a sequence of alphanumeric characters having fewer characters than the plurality of substrings. [Item 4] Item 3. The non-transitory machine-readable storage medium of item 2, wherein a first of the substrings corresponds to a first of the test instances, the first of the substrings including a first sequence of one or more characters to identify the first of the test instances and a second sequence of one or more characters representing a first of the test results corresponding to the first of the test instances. [Item 5] 5. The non-transitory machine-readable storage medium of item 4, wherein the data string is a first data string, and the instructions cause the one or more of the at least one processor circuit to generate a second data string associated with a first of the substrings of the first data string, the second data string representing a plurality of test outcomes resulting from a plurality of circuit blocks of the first device under test that are being tested based on the first of the test instances. [Item 6] Item 6. The non-transitory machine-readable storage medium of item 5, wherein the second data string includes respective characters representing test outcomes for each of the plurality of circuit blocks, and a position of the respective characters in the second data string identifies the respective one of the circuit blocks. [Item 7] Interface circuit configuration; machine-readable instructions; and Encoding a data string representing a sequence of test results corresponding to a plurality of test instances performed by the automated test equipment on the first device under test; and outputting said data string to a second device; at least one processor circuit programmed with said machine-readable instructions to An apparatus comprising: [Item 8] 8. The apparatus of claim 7, wherein the data string is based on a plurality of substrings, each one of the substrings corresponding to a respective one of the test instances. [Item 9] 9. The apparatus of claim 8, wherein the at least one processor circuit encodes the plurality of substrings into the data string, the data string being a single alphanumeric character or a sequence of alphanumeric characters having fewer characters than the plurality of substrings. [Item 10] 8. The apparatus of claim 7, wherein a first one of the substrings corresponds to a first one of the test instances, and the first one of the substrings includes a first sequence of one or more characters to identify the first one of the test instances and a second sequence of one or more characters representing a first one of the test results corresponding to the first one of the test instances. [Item 11] Item 11. The apparatus of item 10, wherein the data string is a first data string, and one or more of the at least one processor circuit generates a second data string associated with a first of the substrings of the first data string, the second data string representing a plurality of test outcomes resulting from a plurality of circuit blocks of the first device under test being tested based on the first of the test instances. [Item 12] Item 12. The apparatus of item 11, wherein the second data string includes respective characters representing test outcomes for each of the plurality of circuit blocks, and the position of the respective characters in the second data string identifies the respective one of the circuit blocks. [Item 13] At least one processor circuit includes at least: decoding a data string including a plurality of substrings representing a plurality of test outcomes for a plurality of test instances performed on a plurality of circuit blocks of the device under test; combining at least two of the test outcomes based on hierarchical rules to determine test indicators for two or more groups of the circuit blocks; and outputting a test report based on said test indicators; A non-transitory machine-readable storage medium comprising instructions to cause [Item 14] Item 14. The non-transitory machine-readable storage medium of item 13, wherein the hierarchical rules specify at least one of: 1) a physical grouping of the two or more circuit blocks based on a physical design of the device under test; or 2) a logical grouping of the two or more circuit blocks based on a user definition. [Item 15] The rules specify the physical grouping based on a physical hierarchy, the physical hierarchy being: a first hierarchical level representing a first of the test outcomes of a first of the test instances performed on the two or more of the circuit blocks of the device under test; and a second hierarchical level representing the first combination of the test outcomes used to determine the test indicators for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the physical design of the device under test; Item 15. The non-transitory machine-readable storage medium of item 14, comprising: [Item 16] The rules specify the logical groupings based on a logical hierarchy, the logical hierarchy comprising: a first hierarchical level representing a first of the test outcomes of a first of the test instances performed on the two or more of the circuit blocks of the device under test; and a second hierarchical level representing the first combination of the test outcomes used to determine the test indicators for the group of the two or more circuit blocks, the first combination of the test outcomes being based on the user definition; Item 15. The non-transitory machine-readable storage medium of item 14, comprising: [Item 17] the at least two of the test outcomes are a first group of the test outcomes, the test indicator is a first test indicator, the group of two or more of the circuit blocks is a first group of the circuit blocks, and the instructions to one or more of the at least one processor circuit: combining the second group of test outcomes based on the hierarchical rules to determine second test indicators for the second group of circuit blocks; and determining a combined indicator based on a combination of the first test indicator and the second test indicator; Item 14. The non-transitory machine-readable storage medium of item 13, [Item 18] Item 18. The non-transitory machine-readable storage medium of item 17, wherein the instructions cause one or more of the at least one processor circuit to perform a procedure for generating a classification of the device under test, the classification being based on the combined indicator. [Item 19] 20. The non-transitory machine-readable storage medium of claim 18, wherein the instructions cause one or more of the at least one processor circuit to perform a procedure for generating a classification of the device under test offline. [Item 20] the hierarchical rule is a first hierarchical rule, the test report is a first test report, and the instructions to one or more of the at least one processor circuit: obtaining a second tier rule after determining the first test report; and outputting a second test report based on the second hierarchical rule and the data string without performing an additional test instance on the device under test; Item 17. The non-transitory machine-readable storage medium of item 16,

Claims

1. At least one processor circuit includes at least: encoding a data string representing a sequence of test results corresponding to a plurality of test instances performed by the automatic test equipment on the first device under test; and outputting the data string to a second device; A computer program for executing

2. 2. The computer program product of claim 1, wherein the data string is based on a plurality of substrings, each one of the plurality of substrings corresponding to a respective one of the plurality of test instances.

3. 3. The computer program product of claim 2, further comprising: causing one or more of the at least one processor circuit to perform a procedure for encoding the plurality of substrings into the data string, the data string being a single alphanumeric character or a sequence of alphanumeric characters having fewer characters than the plurality of substrings.

4. 4. The computer program product of claim 2 or claim 3, wherein a first of the plurality of sub-strings corresponds to a first of the plurality of test instances, the first of the plurality of sub-strings including a first sequence of one or more characters to identify the first of the plurality of test instances and a second sequence of one or more characters representing a first of the test results corresponding to the first of the plurality of test instances.

5. 5. The computer program of claim 4, wherein the data string is a first data string, and the computer program causes the one or more of the at least one processor circuits to generate a second data string associated with a first of the plurality of sub-strings of the first data string, the second data string representing a plurality of test outcomes resulting from a plurality of circuit blocks of the first device under test being tested based on the first of the plurality of test instances.

6. 6. The computer program product of claim 5, wherein the second data string includes respective characters representing test outcomes for the respective ones of the plurality of circuit blocks, and wherein a position of the respective characters in the second data string identifies the respective one of the plurality of circuit blocks.

7. A machine-readable storage medium storing a computer program according to any one of claims 1 to 3.

8. Interface circuit configuration; machine-readable instructions; and Encoding a data string representing a sequence of test results corresponding to a plurality of test instances performed by the automated test equipment on the first device under test; and Outputting the data string to a second device at least one processor circuit programmed with said machine-readable instructions to An apparatus comprising:

9. 9. The apparatus of claim 8, wherein the data string is based on a plurality of substrings, each one of the plurality of substrings corresponding to a respective one of the plurality of test instances.

10. 10. The apparatus of claim 9, wherein the at least one processor circuit encodes the plurality of substrings into the data string, the data string being a single alphanumeric character or a sequence of alphanumeric characters having fewer characters than the plurality of substrings.

11. 11. The apparatus of claim 9 or claim 10, wherein a first of the plurality of sub-strings corresponds to a first of the plurality of test instances, the first of the plurality of sub-strings including a first sequence of one or more characters to identify the first of the plurality of test instances and a second sequence of one or more characters representing a first of the test results corresponding to the first of the plurality of test instances.

12. 12. The apparatus of claim 11, wherein the data string is a first data string, and one or more of the at least one processor circuit generates a second data string associated with a first of the plurality of substrings of the first data string, the second data string representing a plurality of test outcomes resulting from a plurality of circuit blocks of the first device under test being tested based on the first of the plurality of test instances.

13. 13. The apparatus of claim 12, wherein the second data string includes a respective character representing a test outcome for each of the plurality of circuit blocks, the position of the respective character in the second data string identifying the respective one of the plurality of circuit blocks.

14. At least one processor circuit includes at least: decoding a data string including a plurality of sub-strings representing a plurality of test outcomes for a plurality of test instances performed on a plurality of circuit blocks of the device under test; combining at least two of the plurality of test outcomes based on hierarchical rules to determine test indicators for two or more groups of the plurality of circuit blocks; and outputting a test report based on said test indicators; A computer program for executing

15. 15. The computer program product of claim 14, wherein the hierarchical rules specify at least one of: 1) a physical grouping of the two or more circuit blocks based on a physical design of the device under test; or 2) a user-defined logical grouping of the two or more circuit blocks.

16. The hierarchical rules specify the physical grouping based on a physical hierarchy, the physical hierarchy comprising: a first hierarchical level representing a first of the plurality of test outcomes of a first of the plurality of test instances performed on the two or more of the plurality of circuit blocks of the device under test; and a second hierarchical level representing the first combination of the plurality of test outcomes used to determine the test indicators for the group of the two or more circuit blocks, the first combination of the plurality of test outcomes being based on the physical design of the device under test; 16. The computer program of claim 15, comprising:

17. The hierarchical rules specify the logical groupings based on a logical hierarchy, the logical hierarchy comprising: a first hierarchical level representing a first of the plurality of test outcomes of a first of the plurality of test instances performed on the two or more of the plurality of circuit blocks of the device under test; and a second hierarchical level representing the first combination of the plurality of test outcomes used to determine the test indicators for the group of the two or more circuit blocks, the first combination of the plurality of test outcomes being based on the user definition; 16. The computer program of claim 15, comprising:

18. the at least two of the plurality of test outcomes are a first group of the plurality of test outcomes, the test indicator is a first test indicator, the group of two or more of the plurality of circuit blocks is a first group of the plurality of circuit blocks, and the computer program causes one or more of the at least one processor circuit to: combining the second group of the plurality of test outcomes based on the hierarchical rules to determine second test indicators for the second group of the plurality of circuit blocks; and determining a combined indicator based on a combination of the first test indicator and the second test indicator; 18. A computer program product according to any one of claims 14 to 17, for causing the computer to execute the following:

19. 20. The computer program product of claim 18, causing one or more of the at least one processor circuit to perform a procedure for generating a classification of the device under test, the classification being based on the combined indicator.

20. 20. The computer program product of claim 19, further comprising causing one or more of the at least one processor circuit to perform a procedure for generating a classification of the device under test offline.

21. the hierarchical rule is a first hierarchical rule, the test report is a first test report, and the computer program causes one or more of the at least one processor circuit to: obtaining a second hierarchical rule after determining the first test report; and outputting a second test report based on the second hierarchical rule and the data string without performing an additional test instance on the device under test; 15. The computer program product of claim 14,

22. A machine-readable storage medium storing a computer program according to any one of claims 14 to 17.

23. Interface circuit configuration; machine-readable instructions; and decoding a data string including a plurality of substrings representing a plurality of test outcomes for a plurality of test instances performed on a plurality of circuit blocks of the device under test; combining at least two of the plurality of test outcomes based on a hierarchical rule to determine test indicators for two or more groups of the plurality of circuit blocks; and outputting a test report based on the test indicators at least one processor circuit programmed with said machine-readable instructions to An apparatus comprising:

24. 24. The apparatus of claim 23, wherein the hierarchical rules specify at least one of: 1) a physical grouping of the two or more circuit blocks based on a physical design of the device under test; or 2) a user-defined logical grouping of the two or more circuit blocks.

25. the at least two of the plurality of test outcomes are a first group of the plurality of test outcomes, the test indicator is a first test indicator, the group of two or more of the plurality of circuit blocks is a first group of the plurality of circuit blocks, and one or more of the at least one processor circuit: combining the second group of the plurality of test outcomes based on the hierarchical rule to determine second test indicators for the second group of the plurality of circuit blocks; and determining a combined indicator based on a combination of the first test indicator and the second test indicator; 25. Apparatus according to claim 23 or claim 24.

26. 26. The apparatus of claim 25, wherein one or more of the at least one processor circuit generates a classification of the device under test, the classification being based on the combined indicator.

27. the hierarchical rule is a first hierarchical rule, the test report is a first test report, and one or more of the at least one processor circuit: Obtaining a second hierarchical rule after determining the first test report; and outputting a second test report based on the second hierarchical rule and the data string without performing an additional test instance on the device under test; 24. The apparatus of claim 23.

Images