Test measurement device and quality determination method of device under test

Abstract

To more quickly determine quality of a device under test (DUT).SOLUTION: A test measurement device 40 includes a port that receives a signal from DUT 10, a user interface 56 that allows a user to input data to the test measurement device 40 and receive a result, and a processor 42. The processor 42 acquires waveform data of the signal from the DUT 10, generates a measurement value for generating a thinning-out measurement value set for the signal, converts the thinning-out measurement value into a tensor, sends the tensor to a machine learning network 28, and receives a pass / rejection value from the machine learning network 28. The device generates and analyses a full sweep measurement value set of the DUT 10 to determine quality of the DUT, only when the quality cannot be clearly determined by the pass / rejection value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a test and measurement apparatus and a method using the same, and more particularly to a test and measurement apparatus and method that applies machine learning to measure characteristics of a signal from a device under test (DUT) and determine whether the DUT passes or fails. [Background technology]

[0002] When measuring the performance of a device under test (DUT), a user typically performs a "full sweep" of measurements. Examples of measurements include, but are not limited to, various measurements such as current and voltage. The term "full sweep" in this context refers to measurements of all possible measurement points in the data. Making this many measurements takes a long time. This measurement typically allows the user testing the device to determine whether the DUT has passed or failed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Pat. No. 1,192,3895 [Patent Document 2] US Patent Application Publication No. 2022 / 0373598 [Non-patent literature]

[0004] [Non-Patent Document 1] "Tektronix oscilloscope" introduction site, Tektronix, [online], [searched September 24, 2024], Internet<https: / / www.tek.com / ja / products / oscilloscopes> [Non-Patent Document 2] "Keithley Source Measure Units", Tektronix / Keithley, [Online], [Retrieved September 24, 2024], Internet<https: / / www.tek.com / ja / products / keithley / source-measure-units> [Non-Patent Document 3] "IV Curve Tracer Software", Tektronix / Keithley, [Online], [Retrieved September 24, 2024], Internet<https: / / www.tek.com / ja / keithley-iv-curve-tracer> Summary of the Invention [Problem to be solved by the invention]

[0005] Users want to reduce the number of measurements required to speed up the data collection process, but they must balance the desire for faster measurements with the need to accurately determine a pass / fail result. [Means for solving the problem]

[0006] In general, embodiments of the present application associate full sweep measurements and pass / fail results with decimated versions of the full sweep measurements. This allows users to speed up acquisition of measurement data. Rather than decimating measurements and creating pass / fail results based on the decimated measurements, the system trains a neural network with pass / fail input labels based on the slower full sweep measurement set and associates these labels with a decimated measurement set extracted from the full sweep measurement set. Utilizing a smaller set of decimated measurements while maintaining accuracy further speeds up the process.

[0007] In one embodiment of the present disclosure, a manufacturing application involves taking multiple measurements on a device under test (DUT) and then processing to indicate a classification such as pass / fail or other failure type. For example, one may wish to measure the IV (current vs. voltage) curve of a part. This requires measuring many points and can be time consuming. Additionally, measuring data such as transfer functions, such as s21 frequency response style data, requires many points and can be very time consuming.

[0008] Naturally, users and programmers of such test systems will attempt to reduce the number of measurements required to speed up the data collection process. Embodiments of the disclosed technology allow users to further reduce the data measurement set by using a neural network. These embodiments include training a machine learning system on the complete data set of a full sweep to create training labels, while associating measurements using a reduced set of measurements (data points). For the purposes of this description, the reduced set of measurements (data points) is referred to as the "thinned" measurement set. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an embodiment of a training configuration for a test and measurement system. [Diagram 2] FIG. 2 illustrates an embodiment of a run-time configuration for a test and measurement system. [Diagram 3] FIG. 3 illustrates an embodiment of a test and measurement device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] FIG. 1 illustrates an embodiment of a system configuration for training a machine learning network to provide a pass / fail result for a device under test (DUT) using decimated measurements. The block diagram illustrates a system configuration for training a neural network, according to some example embodiments. In the processing block of a neural network during training, a label is one of the inputs and is a predicted pass / fail value. The label may be assigned a value of 1.0 for pass and a value of 0 for fail. The neural network is trained to associate this label with other inputs. In machine learning, a "label" is information that indicates an expected correct answer for data. For example, for "cat image" data, the expected correct answer information is "cat", so "cat" is the label. Thus, in this example, training is performed to associate the label "cat" with the "cat image" data. As a result, at runtime, it is expected that when "cat image" data is input, "cat" is output.

[0011] In some embodiments, the DUT may be an integrated circuit die, such as a die of a silicon photonics transceiver wafer. According to some of these embodiments, other inputs to the neural network may consist of six sets of measurements obtained from the die through a test station setup. Other examples of DUTs include, but are not limited to, electronic devices, optical devices such as optical switches, etc. Examples of measurements include current-voltage (IV) curves, photonic S-parameters (s11, s12, s21, s22), or other characteristics.

[0012] Additionally, in some embodiments, other parameters may be employed as inputs to the neural network, such as ambient temperature, model number of the wafer / die format, etc. Training a neural network requires a sufficient amount of data, i.e., enough samples of DUTs to cover the expected distribution of the DUT characteristics to be seen during manufacturing. A sufficient amount of data can be considered to exist if the neural network can be trained to make predictions within a given accuracy threshold.

[0013] FIG. 1 shows an embodiment of a training configuration for a system that can use decimated measurements for pass / fail of a DUT. The test and measurement equipment forms part of a test station 12. The test station 12 is connected to a DUT 10 to receive signals from the DUT 10, and the test and measurement equipment can obtain measurement data, such as waveform data (also simply referred to as a "waveform") from the DUT 10. The test and measurement equipment performs a full sweep of measurements for each DUT to generate a full sweep measurement set in block 14. The test and measurement equipment then reduces each full sweep measurement set to a limited reduced measurement set 16, which is referred to herein as a "decimated measurement set."

[0014] In one embodiment, the tensor builder 20 converts each of the decimated measurements into either a tensor, a one-dimensional data vector, or a tensor image (e.g., a red-green-blue (RGB) image), as described in more detail in U.S. Pat. No. 1,192,3895 and U.S. Patent Application Publication No. 2022 / 0373598, both of which are incorporated herein by reference.

[0015] If the tensor builder 20 produces a tensor image, then no extra normalization block is needed. If the tensor builder 20 produces a single input vector, then a normalization block 22 is needed to represent the data in the range 0 to 1 for input to the first layer of the neural network. For label inputs, if the label is 1 for pass and 0 for fail, then an extra normalization block 26 is not needed. In this case, the output of the label vector block 24 is fed directly to the neural network. However, if it is desired for the neural network to predict other values ​​in addition to pass / fail, then a normalization block may be needed, depending on the nature of the variables.

[0016] Other information such as the temperature of the DUT under test, the model number of the DUT and other information may also be added or embedded or encoded into the tensors generated by the tensor builder 20. The full sweep measurements corresponding to the decimated measurements are analyzed in block 18 to ascertain whether the DUT has passed or failed. The analysis block 18 analyzes the full sweep measurement (data point) set and applies the necessary algorithms to determine what the pass / fail label is.

[0017] The pass or fail labels are also converted to a label vector in block 24. This may be normalized in block 26 for input to the neural network 28. As mentioned above, depending on the values ​​used for pass / fail, the normalization block 26 may not be necessary. The decimated measurement tensor is assigned a pass / fail label. This allows the machine learning network 28 to associate the set of decimated measurements with a pass or fail label. This ultimately results in the machine learning network (neural network) 28 being able to determine pass / fail (pass / fail) from the set of decimated measurements at run time.

[0018] In summary, the training process involves characterizing a sufficient number of DUTs (e.g., semiconductor dies) and performing a full sweep of data points for each DUT. Each set of data points from the full sweep is analyzed to identify a pass or fail. The full sweep set of measurements is decimated to generate a decimated set of measurements. The decimated and other data are sent to a tensor builder 20 for normalization as needed. The resulting tensors and label tensors are sent to the neural network 28 for training.

[0019] FIG. 2 shows an embodiment of a configuration for the run-time of the test system. In the configuration of FIG. 2, a full sweep measurement set is utilized as an option for further analysis of a suspected failing DUT. The main process flow includes creating a decimated measurement set (data points), building a tensor, and sending the tensor to the input of the neural network. In this configuration, a full sweep measurement set is not collected. The test station blocks only spend time collecting decimated data measurement sets. Upon receiving the decimated data (measurements), the neural network 28 provides a value indicating pass or fail (PF).

[0020] The PF results may take various forms and may be further analyzed to make them more accurate and unambiguous. The results may or may not be un-normalized in block 30 depending on whether the pass / fail range is between 0 and 1, as described above. Further analysis may include determining a single threshold value to indicate pass or fail, or alternatively, determining one threshold value for pass and one threshold value for fail. The labels output from the neural network may be represented based on the recurrent layer of the neural network. In cases where the expected value output for pass is 1.0 and the expected value output for fail is 0.0, or in embodiments where the expected value ranges from 0 to 1, the un-normalization block 30 may not be necessary. The neural network may output a PF value as a decimal value of any value between 0 and 1, representing the degree of pass or fail.

[0021] In the embodiment of Figure 2, the pass threshold is if the PF value is greater than or equal to 0.8 at block 32, then the DUT passes. If the PF value is less than or equal to 0.2 at block 34, then the DUT fails. Another option is to have a single threshold, above which values ​​pass and below which values ​​fail.

[0022] Problems can arise when multiple separate thresholds are used and the PF value falls between the separate thresholds, resulting in an unclear or "fuzzy" result. For example, in the example of FIG. 2, the DUT did not pass at decision block 32 and did not fail at decision block 34. At that point, there is the option to generate a full sweep measurement set as in block 14 of FIG. 1. The full sweep measurement set is then analyzed in detail in block 18 to provide a final result of pass or fail.

[0023] The pass / fail output from the neural network then determines whether to package the die or move the DUT through the manufacturing process. If the result is a fail, the die or DUT is discarded. If the result is a pass, the die is packaged or the DUT is moved through the manufacturing process. In summary, threshold levels may be assigned such that a label greater than 0.8 is a pass and less than 0.2 is a fail, and thresholds other than 0.8 and 0.2 may be used. Limiting the use of the full swept measurement set to only those results that are ambiguous makes the process much faster.

[0024] A test and measurement instrument captures signals generated by a device under test (DUT) as waveform data (also simply referred to as "waveform"). FIG. 3 illustrates an embodiment of such a test and measurement instrument 40. FIG. 3 illustrates an embodiment of a test setup in an example of a test and measurement instrument 40 measuring or testing a DUT 10. The test setup may include a test and measurement instrument 40, which may be an oscilloscope or a source measure unit (SMU). The test and measurement instrument 40 receives signals from the DUT 10 directly or through a probe 46. In the case of an optical transmitter (Tx) DUT, the probe typically includes an optical fiber coupled to an optical / electrical conversion circuit 46, not shown, which provides signals to the test and measurement instrument 40 through one or more ports 43. Two ports may be used for differential signaling. Also, one port may be used for single-ended signaling. The signals are sampled and digitized by the test and measurement instrument 40 into waveform data. The clock recovery unit (CRU) 50 recovers a clock signal from the data signal when the test and measurement equipment 40 includes, for example, a sampling oscilloscope. Software clock recovery can be used in real-time oscilloscopes. In addition, when the test and measurement equipment 40 is an SMU, for example, a current or voltage may be supplied to the device under test (DUT) 10, and the current or voltage generated in the DUT 10 may be measured in response to the current or voltage to generate waveform data. In this case, a set of full sweep measurements may be generated by sequentially changing (full sweeping) the supplied current or voltage over the entire range required to measure the characteristics of the DUT, and measuring the current or voltage generated at that time to generate waveform data. To generate data for training the machine learning network 28, multiple DUTs may be sequentially replaced and a similar full sweep process may be performed to generate a set of full sweep measurements for each of the multiple DUTs.

[0025] The test and measurement instrument 40 includes one or more processors represented by processor 42, memory 52, and a user interface 56. The memory 52 may store executable instructions in the form of code that, when executed by the processor 42, cause the processor to perform tasks. One such example may be IV curve tracer software (see Non-Patent Document 3) in the case where the test and measurement instrument 40 includes an SMU. The user interface 56 of the test and measurement instrument 40 allows a user to interactively operate / configure the test and measurement instrument 40, including entering settings, configuring tests, etc. The test and measurement instrument 40 may also include a reference equalizer and analysis module 54.

[0026] The embodiments of the present application employ machine learning in the form of a machine learning network 28, such as a deep learning network. The machine learning network 28 may include a processor programmed to implement the machine learning network functions, which may be part of or accessible to the test and measurement instrument 40. As test and measurement instrument capabilities and processors evolve, one or more processors (such as 42) may include both processors (one for operation of the test and measurement instrument and one for the machine learning network). The machine learning system may take the form of a programmed model running on one or more processors. The processor may also be configured to execute code (programs) for performing the methods of the embodiments.

[0027] The present embodiment provides a test and measurement device and a generalized method for predicting / judging pass / fail based on a decimated measurement set using machine learning. The machine learning network is trained using pass / fail labels derived from a full sweep measurement set. Meanwhile, at run-time, measurement time can be reduced by collecting only the data of the decimated sweep measurement set. Only for DUTs for which the data of the decimated measurement set does not clearly predict / judgment pass / fail, a full sweep measurement is performed to generate a full sweep measurement set, which is then analyzed to make a final accurate prediction / judgment of pass / fail. DUTs below the pass threshold may be discarded.

[0028] Aspects of the disclosed technology can operate on specially created hardware, firmware, digital signal processors, or specially programmed general-purpose computers, including processors that operate according to programmed instructions. The term "controller" or "processor" in this application contemplates microprocessors, microcomputers, ASICs, and dedicated hardware controllers, among others. Aspects of the disclosed technology can be implemented in computer-available data and computer-executable instructions, such as one or more program modules, executed by one or more computers (including a monitoring module) or other devices. Generally, program modules include routines, programs, objects, components, data structures, and the like, which, when executed by a processor in a computer or other device, perform particular tasks or implement particular abstract data formats. The computer-executable instructions may be stored in computer-readable storage media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, and the like. As will be appreciated by those skilled in the art, the functionality of the program modules may be combined or distributed as desired in various embodiments. Furthermore, such functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc. Certain data structures may be used to more effectively implement one or more aspects of the disclosed techniques, and such data structures are considered to be within the scope of the computer-executable instructions and computer-usable data described herein.

[0029] The disclosed aspects may, in some cases, be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried by or stored on one or more computer-readable media that may be read and executed by one or more processors. Such instructions may be referred to as computer program products. A computer-readable medium as described herein means any medium that can be accessed by a computing device. By way of example, and not limitation, computer-readable media may include computer storage media and communication media.

[0030] Computer storage media means any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital video disc (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any other volatile or non-volatile removable or non-removable media implemented in any technology. Computer storage media excludes signals themselves and transitory forms of signal transmission.

[0031] A communication medium refers to any medium capable of communicating computer readable information. By way of example, and not limitation, communication media may include coaxial cables, fiber optic cables, air, or any other medium suitable for communicating electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals. Working Example

[0032] In the following, examples useful for understanding the technology disclosed in the present application are presented. The embodiment of the technology may include one or more of the examples described below and any combination thereof.

[0033] Example 1 is a test and measurement instrument comprising one or more ports for enabling the test and measurement instrument to receive signals from a device under test (DUT), a user interface for enabling a user to provide input to the test and measurement instrument and receive results, and one or more processors, the one or more processors being configured to execute programs that cause the one or more processors to perform the following processes: acquire waveform data of signals from the DUT; generate measurements to generate a set of decimated measurements for the signals; convert the set of decimated measurements into tensors; send the tensors to a machine learning network; and receive a pass / fail value for the DUT from the machine learning network.

[0034] Example 2 is the test and measurement device of Example 1, wherein the program that causes the one or more processors to process receiving a pass / fail value from the machine learning network causes the one or more processors to process receiving the pass / fail value as a value between 1.0 and 0.0.

[0035] Example 3 is a test and measurement apparatus of either Example 1 or 2, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of comparing the pass / fail value with a threshold value to determine whether the DUT has passed or failed.

[0036] Example 4 is the test and measurement apparatus of Example 3, wherein the DUT passes when the pass / fail value is 0.8 or greater, and fails when the pass / fail value is 0.2 or less.

[0037] Example 5 is a test and measurement instrument of any of Examples 1 to 4, wherein the one or more processors are further configured to execute a program that causes the one or more processors to generate a full swept measurement set for the DUT when a clear pass / fail determination cannot be made.

[0038] Example 6 is the test and measurement instrument of example 5, wherein a user provides an input through the user interface selecting to perform a full sweep of measurements.

[0039] Example 7 is the test and measurement instrument of any of Examples 1 to 6, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process for training the machine learning network.

[0040] Example 8 is the test and measurement instrument of Example 7, wherein the program for causing the one or more processors to perform the process of training the machine learning network includes a program for causing the one or more processors to perform the following processes: characterizing the sufficient number of DUTs by generating full sweep measurement sets for each of the sufficient number of DUTs to generate a sufficient amount of measurements (data points) for training; analyzing each of the full sweep measurement sets to determine a pass or fail result for the full sweep measurement set; using the pass or fail result as a label for training the machine learning network; thinning each of the full sweep measurement sets to generate a thinned measurement set corresponding to each of the full sweep measurement sets; and training the machine learning network to associate each of the thinned measurement sets with the pass or fail result, which is a label for the corresponding full sweep measurement set.

[0041] Example 9 is the test and measurement device of Example 8, wherein the one or more processors are further configured to execute a program that causes the one or more processors to convert the thinned measurement set into tensors prior to training the machine learning network.

[0042] Example 10 is a method comprising the steps of acquiring waveform data of a signal from a device under test (DUT), generating measurements to generate a set of decimated measurements for the signal, converting the set of decimated measurements into a tensor, sending the tensor to a machine learning network, and receiving a pass / fail value from the machine learning network.

[0043] Example 11 is the method of example 10, wherein receiving the pass / fail value from the machine learning network comprises receiving the pass / fail value as a value between 1.0 and 0.0.

[0044] Example 12 is the method of any of examples 10 or 11, further comprising comparing the pass / fail value to a threshold to determine whether the DUT has passed or failed.

[0045] Example 13 is the method of example 12, wherein the DUT passes if the pass / fail value is greater than or equal to 0.8 and fails if the pass / fail value is less than or equal to 0.2.

[0046] Example 14 is the method of any of examples 10 to 13, further comprising performing a full sweep of measurements for the DUT if a clear pass / fail decision cannot be made.

[0047] Example 15 is the method of example 14, wherein a user provides an input through the user interface selecting to perform a full sweep of measurements.

[0048] Example 16 is the method of any of examples 10 to 13, further comprising training the machine learning network.

[0049] Example 17 is the method of Example 16, wherein the process of training the machine learning network includes a process of evaluating characteristics of a sufficient number of DUTs by generating full sweep measurement sets for each of the DUTs to generate a sufficient amount of measurements (data points) for training, a process of analyzing each of the full sweep measurement sets to determine a pass or fail outcome for the full sweep measurement set, a process of using the pass or fail outcome as a label for training the machine learning network, a process of thinning each of the full sweep measurement sets to generate a thinned measurement set corresponding to each of the full sweep measurement sets, and a process of training the machine learning network to associate each of the thinned measurement sets with the pass or fail outcome, which is a label for the corresponding full sweep measurement set.

[0050] Example 18 is the method of example 17, further comprising converting the thinned measurement set into a tensor before training the machine learning network.

[0051] In addition, the present description refers to specific features. It should be understood that the disclosure herein includes all possible combinations of those specific features. When a specific feature is disclosed in connection with a particular aspect or embodiment, that feature can also be used in connection with other aspects and embodiments, to the extent possible.

[0052] Furthermore, when the application refers to a method having two or more defined steps or processes, those defined steps or processes may be performed in any order or simultaneously, unless the context precludes such a possibility.

[0053] All features disclosed in the specification, claims, abstract and drawings, and all steps in any disclosed method or process, may be combined in any combination, except where at least some of such features or steps are mutually exclusive combinations. Each feature disclosed in the specification, abstract, claims and drawings may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0054] For convenience of illustration, specific embodiments of the disclosed technology have been illustrated and described, but it will be understood that various modifications can be made therein without departing from the spirit and scope of the invention. Accordingly, the disclosed technology should not be limited, except as by the appended claims. [Explanation of symbols]

[0055] 10 Device Under Test (DUT) 28 Machine Learning Networks 40 Test and measurement equipment 42 processors 43 Port 46 Test Optical Fiber / Probe 50 Clock Recovery Unit 52 Memory 54 Reference Equalizer and Analysis Module 56 User Interface

Claims

1. 1. A test and measurement device comprising: one or more ports for enabling the test and measurement instrument to receive signals from a device under test (DUT); a user interface that allows a user to provide inputs to and receive results from the test and measurement instrument; One or more processors Equipped with the one or more processors A process of acquiring waveform data of a signal from the DUT; generating measurements to generate a set of decimated measurements on the signal; converting the set of thinned measurements into a tensor; and Sending the tensor to a machine learning network; A process of receiving a pass / fail value from the machine learning network. a test and measurement instrument configured to execute a program that causes the one or more processors to:

2. 2. The test and measurement instrument of claim 1, wherein the one or more processors are further configured to execute a program that causes the one or more processors to compare the pass / fail value to a threshold value to determine whether the DUT has passed or failed.

3. 3. The test and measurement instrument of claim 2, wherein the thresholds comprise a first threshold for determining whether the DUT has passed, and a second threshold for determining whether the DUT has failed.

4. 2. The test and measurement instrument of claim 1, wherein the one or more processors are further configured to execute a program that causes the one or more processors to generate a full swept measurement set for the DUT if a definitive pass / fail determination of the DUT cannot be made.

5. The one or more processors: characterizing a sufficient number of DUTs by generating a full sweep measurement set for each of the sufficient number of DUTs to generate a sufficient amount of measurements for training; performing an analysis of each of the full sweep measurement sets to determine a pass or fail result for the full sweep measurement set; using the pass or fail results as labels for training the machine learning network; decimating each of the full sweep measurement sets to generate a corresponding set of decimated measurements; training the machine learning network to associate each of the thinned measurement sets with a pass or fail outcome that is a label for the corresponding full sweep measurement set; 2. The test and measurement instrument of claim 1, further configured to execute a program that causes the one or more processors to:

6. A process of acquiring waveform data of a signal from a device under test (DUT); generating measurements to generate a set of decimated measurements on the signal; converting the set of thinned measurements into a tensor; and Sending the tensor to a machine learning network; receiving a pass / fail value for the DUT from the machine learning network; A method for determining whether a device under test passes or fails, comprising:

7. 7. The method of claim 6, further comprising comparing the pass / fail value to a threshold value to determine whether the DUT has passed or failed.

8. 7. The method of claim 6, further comprising generating a full swept measurement set for said DUT if a definite pass / fail decision cannot be made.

9. characterizing a sufficient number of DUTs by generating a full sweep measurement set for each of the sufficient number of DUTs to generate a sufficient amount of measurements for training; performing an analysis of each of the full sweep measurement sets to determine a pass or fail result for the full sweep measurement set; using the pass or fail results as labels for training the machine learning network; decimating each of the full sweep measurement sets to generate a corresponding set of decimated measurements; training the machine learning network to associate each of the thinned measurement sets with a pass or fail outcome that is a label for the corresponding full sweep measurement set; 7. The method of claim 6, further comprising:

Images