Determination device, test system, determination method and recording medium

Abstract

A determination device is provided, comprising: a result acquisition unit that acquires test results of a plurality of items of tests performed on a device under test; and a first determination unit that determines whether to retest the device under test that failed the test; and the first determination unit makes the determination based on the reproducibility of the test results when the plurality of tests were previously performed a plurality of times on the plurality of devices under test.

Description

Technical Field

[0001] The present invention relates to a determination device, a test system, a determination method and a recording medium. Background Art

[0002] When testing a device under test, there is a possibility that the test may be erroneously determined to have failed due to poor contact, etc. Therefore, conventionally, the failed device under test is retested to prevent a decrease in yield. Summary of the Invention

[0003] In a first aspect of the present invention, a determination device is provided. The determination device may include a result acquisition unit that acquires test results of multiple items of tests performed on a device under test. The determination device may include a first determination unit that determines whether to retest a device under test that failed a test. The first determination unit may make the determination based on the reproducibility of test results from multiple tests previously performed on multiple devices under test.

[0004] The determination device may further include a calculation unit that calculates reproducibility for each of the plurality of items. The first determination unit may determine whether to retest a device under test that has failed a test for a corresponding item based on the reproducibility calculated by the calculation unit.

[0005] The first determination unit may perform determination using a learning model that has learned reproducibility.

[0006] The calculation unit may include a learning model that outputs a predicted result of a retest in response to input test results of a plurality of items. The calculation unit may include a supply unit that supplies the test results of the plurality of items acquired by the result acquisition unit to the learning model. The calculation unit may include a reproducibility acquisition unit that acquires reproducibility from the predicted result of the retest output by the learning model in response to the test results of the plurality of items being supplied to the learning model.

[0007] The determination device may further include a learning processing unit that executes a learning process of the learning model using learning data including an item ID (identifier) of at least a failed item among the plurality of items and a result of a retest.

[0008] The determination device may further include a second determination unit that determines whether to relearn the learning model using the result of the retest and the determination result of the first determination unit. The learning processing unit may execute a learning process of the learning model based on the determination result of the second determination unit.

[0009] The calculation unit may calculate the reproducibility based on test results of a plurality of tests performed on a plurality of devices under test and test results of a plurality of retests.

[0010] The determination device may further include a second determination unit that determines whether to update the reproducibility using the result of the retest and the determination result of the first determination unit. The calculation unit may update the reproducibility based on the determination result of the second determination unit.

[0011] The determination device may further include a third determination unit that determines to retest the devices under test regardless of the determination result of the first determination unit, based on performing a test on the devices under test included in each batch of a reference number of batches. The second determination unit may use the result of the retest performed by the third determination unit to make the determination.

[0012] The determination device may further include a lower limit value acquisition unit that acquires an allowable lower limit value for the proportion of devices under test that passed the test. The determination device may further include a prediction unit that calculates, based on the reproducibility of each item, a predicted number of devices under test that passed the corresponding item, which are predicted to pass at least the retest of the item. The first determination unit may use a value based on the reproducibility as a determination threshold when accumulating the corresponding predicted number of passed devices in order of magnitude of the reproducibility of each item, the value corresponding to the predicted number of passed devices reaching the allowable lower limit value or just before reaching the corresponding number of passed devices.

[0013] The determination device may further include an extraction unit that acquires a data file containing test results and extracts the test results for each item from the data file. The determination device may further include a storage unit that stores the test results extracted by the extraction unit. The result acquisition unit may acquire the test results for a plurality of items from the storage unit.

[0014] According to a second aspect of the present invention, there is provided a test system that may include the determination device according to the first aspect. The test system may include a test device that performs a plurality of tests on a device under test.

[0015] In a third aspect of the present invention, a determination method is provided, which may include a result acquisition stage for acquiring test results of a plurality of items of testing performed on a device under test. The determination method may include a first determination stage for determining whether to retest a device under test that failed a test. In the first determination stage, the determination may be made based on the reproducibility of test results obtained from a plurality of tests previously performed on the plurality of devices under test.

[0016] In a fourth aspect of the present invention, a recording medium is provided that stores a determination program. The determination program is executed by a computer and causes the computer to function as a result acquisition unit that acquires test results of multiple items of tests performed on a device under test. The determination program is executed by the computer and causes the computer to function as a first determination unit that determines whether to retest a device under test that failed a test. The first determination unit can make this determination based on the reproducibility of test results from multiple tests previously performed on multiple devices under test.

[0017] Furthermore, the above summary of the invention does not list all the features of the present invention. In addition, sub-combinations of these feature groups may also constitute inventions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A test system 1 according to this embodiment is shown.

[0019] Figure 2 The test apparatus 200 is shown together with the wafer 101 .

[0020] Figure 3 The operation of the determination device 300 is shown together with the operation of the testing device 200 .

[0021] Figure 4 Indicates the first determination process.

[0022] Figure 5 A determination device 300A according to a modified example is shown together with the test device 200 and the device under test 100 .

[0023] Figure 6 An example of a computer 2200 is shown in which all or part of the various aspects of the present invention may be embodied. DETAILED DESCRIPTION

[0024] The present invention will be described below by way of embodiments of the invention, but the following embodiments are not intended to limit the invention of the claims. In addition, not all combinations of features described in the embodiments are essential to the solution of the invention.

[0025] [1. Test system 1]

[0026] Figure 1 The test system 1 of this embodiment is shown. The test system 1 includes a device under test 100, a test apparatus 200, and a determination apparatus 300. The test apparatus 200 and the determination apparatus 300 may be integrated into a single apparatus.

[0027] [1-1. Device under test 100]

[0028] The device under test 100 may be an electronic device such as a semiconductor or a micro-electromechanical system (MEMS). A plurality of devices under test 100 may be formed on a single wafer. The wafer may be diced and singulated into bare chips, or may be sealed and packaged.

[0029] [1-2. Test device 200]

[0030] The testing apparatus 200 tests one or more devices under test 100. The testing apparatus 200 can test a single item or multiple items (for example, 1,000 to 2,000 items) simultaneously or sequentially. For example, if the device under test 100 is a packaged electronic device, any two or more of the multiple items included in the test can be from the same category, such as audio functions. In this embodiment, as an example, the testing apparatus 200 sequentially tests multiple items on multiple devices under test 100 on a wafer.

[0031] The test device 200 can be a device test device such as a system LSI tester, an analog tester, a logic tester, or a memory tester. The test device 200 provides various test signals to the device under test 100 and obtains response signals from the device under test 100. The test device 200 can provide the data files obtained from the tests of each item to the judgment device 300 via wired or wireless communication. In this embodiment, the test data file provided by the test device 200 can be in the form of a standard test data format (STDF), as an example, and can include the batch number and wafer number of the device under test 100, the device ID of the device under test 100, the item ID of the test item, the measurement value of each item, and the test result of each item (in this embodiment, as an example, the pass / fail judgment value "0", "1"), etc. The data file can also be in the form of a data log (Datalog) and a test log.

[0032] Here, during the test conducted by the test apparatus 200, erroneous test results may be caused by factors such as surface contamination of the device under test 100 and poor contact of the test apparatus 200. Even a device under test 100 that fails the first test (also called initial inspection) may pass the second test (also called re-inspection or retest). Therefore, in conventional test systems, devices under test that fail the initial inspection are retested to prevent a decrease in yield.

[0033] However, if all DUTs that fail the initial inspection are retested, useless retests will be conducted on defective products that are unlikely to pass in the first place. Therefore, in the test system 1 of this embodiment, the determination device 300 determines whether the DUT 100 that failed the initial inspection should be retested. Furthermore, a test failure means that at least one item included in the test has failed. Retesting can be so-called re-probing, which is to contact the test device 200 with the DUT 100, make corrections, and then retest. After the DUT 100 is fixed to the test device 200, when the probe is brought into contact with the DUT 100, the fixed state may or may not be released after the initial inspection.

[0034] [1-3. Determination Device 300]

[0035] The determination device 300 includes an extraction unit 301 , a storage unit 303 , a result acquisition unit 305 , a calculation unit 307 , a lower limit acquisition unit 309 , a prediction unit 311 , a threshold determination unit 313 , and a first determination unit 315 .

[0036] [1-3-1. Extraction Unit 301]

[0037] The extraction unit 301 obtains a test data file (in this embodiment, an STDF file, for example) from the test apparatus 200 and extracts the test results for each item (in this embodiment, a pass / fail determination value, for example) from the data file. In addition to the test results, the extraction unit 301 can also pre-extract the item ID of the corresponding test, the device ID and lot number of the device under test 100, and convert them into a CSV (comma-separated values) file. The extraction unit 301 can then supply the extracted data to the storage unit 303.

[0038] [1-3-2. Storage Unit 303]

[0039] The storage unit 303 stores the test results (in this embodiment, for example, pass / fail values) extracted by the extraction unit 301. The storage unit 303 can associate the test item ID, the device ID of the device under test 100, and the lot number with the test results and store them. The data stored in the storage unit 303 can be read by the result acquisition unit 305.

[0040] [1-3-3. Result Acquisition Unit 305]

[0041] The result acquisition unit 305 acquires test results for a plurality of items performed on the device under test 100. In this embodiment, as an example, the result acquisition unit 305 may acquire the test results from the storage unit 303. The result acquisition unit 305 may acquire the item ID of each item and the device ID of the device under test 100 from the storage unit 303, along with the test results for each item. The result acquisition unit 305 may provide preliminary inspection-related test results and other information to the calculation unit 307 and the first determination unit 315.

[0042] [1-3-4. Calculation Unit 307]

[0043] When multiple tests (two tests in this embodiment, for example) are performed on multiple devices under test 100 , the calculation unit 307 calculates the reproducibility of the test results. The calculation unit 307 can supply the calculated reproducibility value to the first determination unit 315 and the prediction unit 311 .

[0044] Reproducibility can be an indicator that shows how often the same test result (in this embodiment, for example, a pass / fail judgment value) can be reproduced. For example, it shows how often a failed test result in an initial test is reproduced in a retest. The higher the reproducibility, the more likely it is that the time required for retesting will be wasted due to a failed retest. Therefore, reproducibility can be an indicator of the ineffectiveness of retesting.

[0045] As an example, reproducibility can be the ratio of the reproducibility of failed test results (also called the reproducibility rate). Alternatively, reproducibility can be the number of failures predicted in the retest (also called the retest predicted failure number). The retest predicted failure number can be the value obtained by subtracting the number of qualified devices predicted in the retest from the number of DUTs 100 that failed in the initial test (also called the initial test failure number). It can also be the value obtained by multiplying the number of DUTs 100 that failed in the initial test by the reproducibility rate for a reference number of DUTs 100. The reference number can be the number of DUTs 100 included in the wafer, and as an example, can be 100 to 10,000.

[0046] Reproducibility can be a value for each of multiple items, indicating that if the initial test result for one item fails, the retest result will also fail. In this embodiment, as an example, retest failure means that any item included in the test fails, or it can also mean that the same item as the initial test fails.

[0047] In this embodiment, as an example, the calculation unit 307 calculates reproducibility using the learning model 371. The calculation unit 307 includes the learning model 371, a supply unit 373, and a reproducibility acquisition unit 375.

[0048] [1-3-5(1). Learning Model 371]

[0049] The learning model 371 responds to the input test results of a plurality of items and outputs a prediction result of a retest.

[0050] The learning model 371 can be generated through a learning process using learning data, which includes the test results of retests performed on devices under test 100 that failed the initial inspection. The learning data may include the item ID of at least the item that failed among the various items tested, and the retest results. If a test of an item in the initial inspection failed, the retest results may represent the results of only that item, the results of the tests of all items, or the results of the entire test. The learning data may further include the position of the device under test 100 on the wafer, the deviation between the measured value and the ideal value, and so on.

[0051] In this embodiment, as an example, the learning model 371 is a random forest machine learning algorithm, but it can also be other machine learning algorithms such as support vector machine (SVM), K-nearest neighbor method, logistic regression, etc.

[0052] The learning model 371 can learn the results of each item of the test by setting a pass to 0 and a fail to 1. The higher the probability that the retest result will be a failure, the closer the output value to 1. Thus, when a test result indicating failure (i.e., 1) is input for a test of a certain item, the higher the probability that the retest result will be a failure again, the closer the output value to 1. Therefore, the output prediction result can indicate the reproducibility of each test item.

[0053] [1-3-5(2). Supply unit 373]

[0054] The supply unit 373 supplies the test results of a plurality of items (in this embodiment, the test results of the preliminary test as an example) acquired by the result acquisition unit 305 to the learning model 371. The supply unit 373 can supply the test results of each item together with the item ID of the item.

[0055] [1-3-5(3). Reproducibility acquisition unit 375]

[0056] Supply unit 373 supplies test results for a plurality of items to learning model 371. In response, reproducibility acquisition unit 375 can acquire reproducibility from the retest prediction results output by learning model 371. Reproducibility acquisition unit 375 can acquire reproducibility by performing arithmetic operations on the values of the retest prediction results, or can acquire the prediction result values themselves as reproducibility.

[0057] The reproducibility acquisition unit 375 supplies the acquired reproducibility to the prediction unit 311 and the first determination unit 315. In this embodiment, as an example, regardless of the test results of the preliminary inspection, the reproducibility of each item is supplied to the prediction unit 311, and the reproducibility of the items that failed the preliminary inspection (also referred to as preliminary inspection failure items) is supplied to the first determination unit 315.

[0058] [1-3-6. Lower Limit Value Acquisition Unit 309]

[0059] The lower limit value acquisition unit 309 acquires the permissible lower limit value for the proportion of devices under test 100 that pass the test. The permissible lower limit value can be set by the operator of the determination device 300 based on, for example, the yield rate of devices under test 100 that should pass the test. As an example, the permissible lower limit value can be 10%. The lower limit value acquisition unit 309 supplies the acquired permissible lower limit value to the threshold value determination unit 313.

[0060] [1-3-6. Prediction Unit 311]

[0061] Based on the reproducibility of each item, the prediction unit 311 calculates the predicted number of devices under test 100 that passed at least one item of the test (in this embodiment, the initial test, for example) that failed the test. For example, the prediction unit 311 may calculate the predicted number of devices under test 100 that passed the retest.

[0062] The prediction unit 311 may calculate the predicted number of good products based on the reproducibility supplied from the calculation unit 307. The prediction unit 311 may associate the reproducibility with the predicted number of good products for each item and supply the associated information to the threshold determination unit 313.

[0063] [1-3-6. Threshold Determination Unit 313]

[0064] The threshold determination unit 313 determines a threshold value for determining whether to conduct a retest based on the reproducibility of each item and the predicted number of qualified items. In this embodiment, as an example, the threshold value may be a reproducibility value. The threshold determination unit 313 may supply the determined threshold value to the first determination unit 315.

[0065] [1-3-6. First Determination Unit 315]

[0066] The first determination unit 315 determines whether to retest a device under test 100 that failed a test (in this embodiment, for example, an initial test). A test failure may mean that at least one item included in the test failed. The first determination unit 315 can make this determination based on reproducibility.

[0067] For example, based on the reproducibility of each item, the first determination unit 315 can determine whether to retest a device under test 100 that failed a test for that item. In this embodiment, as an example, the first determination unit 315 can make this determination using the test results for each item, the item ID for that item, and the device ID of the device under test 100 supplied from the result acquisition unit 305. The first determination unit 315 can use a learning model 371 that has learned reproducibility and make the determination based on the reproducibility output from the learning model 371. The first determination unit 315 can use the threshold value determined by the threshold determination unit 313 for the determination.

[0068] When a retest is determined, the first determination unit 315 can provide the device ID of the device under test 100 to be retested to the test apparatus 200. This allows the test apparatus 200 to retest the target device under test 100. The retest can be performed again on all test items.

[0069] The determination device 300 described above determines whether to retest a failed device 100 based on the reproducibility of the test results. This saves the trouble of wasting time and effort on retesting a device 100 that still fails after retesting, and allows for efficient retesting.

[0070] Furthermore, whether or not to retest the device under test 100 that failed the test for the corresponding item is determined based on the reproducibility of each item, thereby reliably eliminating unnecessary retests.

[0071] Furthermore, since the judgment is performed using the learning model 371 that has learned the reproducibility of each item, the potential correlation between the test results of the items can be reflected in the judgment, thereby improving the accuracy of the judgment and effectively performing re-inspection.

[0072] Furthermore, by inputting test results of a plurality of items into the learning model 371 and obtaining a retest prediction result indicating reproducibility, it is possible to reliably obtain high-precision reproducibility and perform determination.

[0073] Furthermore, the test results for each item are extracted from the test data file and stored in the storage unit 303, and the stored test results can be retrieved by the result acquisition unit 305. Therefore, unlike the case where the result acquisition unit 305 directly retrieves the data file from the testing device 200 and then extracts the test results, the processing speed of the result acquisition unit 305 can be improved.

[0074] [2. Test device 200]

[0075] Figure 2 A test apparatus 200 is shown together with a wafer 101. The test apparatus 200 includes a tester main body 201, a test head 203, and a prober 205.

[0076] The tester main body 201 is the main body of the test apparatus 200 and controls various tests. For example, the tester main body 201 can perform an initial test on the device under test 100 and retest the device under test 100 based on a signal from the determination device 300 .

[0077] The test head 203 is configured to be drivable between a test position, in which it is connected to the tester main body 201 via a cable and performs tests on the device under test 100, and a retreat position, in which it does not perform tests. During a test, the test head 203 transmits a test signal to the device under test 100 at the test position under control of the tester main body 201. After receiving a response signal from the device under test 100, the response signal is relayed to the tester main body 201. The test head 203 may include a plurality of probes 231 that make electrical contact with the device under test 100.

[0078] The plurality of probes 231 are arranged corresponding to each of a plurality of electrode pads of a plurality of devices under test 100 formed on the wafer 101 (four in this embodiment).

[0079] The prober 205 transfers the wafer 101 and places it on the stage, and aligns the wafer 101 with the test head 203 .

[0080] [3.Action]

[0081] Figure 3 The operation of the determination device 300 is shown together with the operation of the test device 200. The determination device 300 performs the processing of steps S11 to S25 to determine whether to retest the device under test 100. The dotted-line frame in the figure shows the processing executed by the test device 200.

[0082] In step S11, the testing apparatus 200 tests each of the plurality of devices under test 100. In this embodiment, as an example, the testing apparatus 200 tests a reference number of devices under test 100, i.e., all devices under test 100 on the wafer 101. When the process of step S11 is performed for the second time or later, the testing apparatus 200 can perform preliminary inspections on the plurality of devices under test 100 that have not yet been tested.

[0083] In step S13, the extraction unit 301 obtains a data file (in this embodiment, an STDF file, for example) from the test apparatus 200 and extracts the test results for each item. The test results are then stored in the storage unit 303. In this embodiment, the extraction unit 301 may further extract the device ID of the device under test 100 and the test item ID, for example, and store them in the storage unit 303.

[0084] In step S15, the result acquisition unit 305 acquires the test result from the storage unit 303. The result acquisition unit 305 may further acquire the test result, the device ID of the device under test 100, the item ID of the test, and the like.

[0085] In step S17, the first determination unit 315 determines whether all the devices under test 100 of the reference number tested in step S11 have passed the test. In this embodiment, as an example, the first determination unit 315 determines whether all the devices under test 100 on the wafer 101 have passed all the test items. When it is determined that all the devices under test 100 have passed the test in step S17 (step S17; yes), the process moves to step S11. Thus, the devices under test 100 of the next wafer 101 are tested by the test apparatus 200. When it is determined that all the devices under test 100 have not passed the test in step S17 (step S17; no), the process moves to step S19.

[0086] In step S19, the first determination unit 315 determines whether to retest the device under test 100 that failed the test in step S11. The details of the process in step S19 will be described later.

[0087] In step S21, the first determination unit 315 determines whether to perform retesting. If retesting is not performed for all devices under test 100 (step S21: Yes), the process proceeds to step S11. As a result, the devices under test 100 on the next wafer 101 are tested by the test apparatus 200. If retesting is performed for at least one device under test 100 in step S21 (step S21: Yes), the process proceeds to step S23.

[0088] In step S23, the test apparatus 200 retests the one or more devices under test 100 that have been determined to be retested. The retest can be performed on each item of the test.

[0089] When the testing apparatus 200 is capable of testing multiple DUTs 100 simultaneously, the testing apparatus 200 may test only the DUTs 100 that are determined to be undergoing retesting. For example, when the testing apparatus 200 has multiple probes 231 capable of contacting each DUT 100, the testing apparatus 200 may cause each probe 231 to contact a corresponding position on the DUT 100 and flow current only through the probes 231 contacting the DUT 100 undergoing retesting.

[0090] In step S25, similar to step S13 described above, the extraction unit 301 retrieves the data file (in this embodiment, an STDF file, for example) from the test apparatus 200, extracts the test results for each item, and the storage unit 303 stores these test results. After the processing of step S25 is completed, the process can move to step S11. Thus, the device under test 100 on the next wafer 101 is tested using the test apparatus 200.

[0091] [4-1. First determination process]

[0092] Figure 4 The first determination process is shown. The determination apparatus 300 determines whether or not to retest the device under test 100 by performing the processes from steps S101 to S107.

[0093] In step S101, the lower limit value acquisition unit 309 acquires a preset allowable lower limit value from the operator of the determination device 300. When the allowable lower limit value is stored internally in the determination device 300, the lower limit value acquisition unit 309 can acquire the allowable lower limit value.

[0094] In step S102 , the calculation unit 307 calculates reproducibility for each of the plurality of items. The calculation unit 307 may calculate reproducibility using the learning model 371 .

[0095] In step S103, the prediction unit 311 calculates the predicted number of qualified tests for each item based on the reproducibility of each item. As an example, when the reproducibility indicates the recurrence rate of failure, the prediction unit 311 may calculate the predicted number of qualified tests for each item according to the following equation (1). When the reproducibility indicates the number of predicted failures in retests, the prediction unit 311 may calculate the predicted number of qualified tests for each item according to the following equation (2).

[0096] Predicted number of qualified products = number of initial inspection failures × (1-reproduction rate) (1)

[0097] Number of predicted qualified samples = number of initial inspection failures - number of re-inspection prediction failures (2)

[0098] In step S105, the threshold determination unit 313 determines a threshold value for determining whether to retest based on the reproducibility of each item and the predicted number of qualified candidates. For example, the threshold determination unit 313 may accumulate the corresponding predicted number of qualified candidates in order of reproducibility for each item and detect the predicted number of qualified candidates when the accumulated result reaches the corresponding number of qualified candidates at the lower limit of the allowable range. Alternatively, the threshold determination unit 313 may detect the predicted number of qualified candidates just before the accumulated result reaches the corresponding number of qualified candidates at the lower limit of the allowable range. The threshold determination unit 313 may use the value based on the reproducibility corresponding to the detected predicted number of qualified candidates as the threshold value.

[0099] Here, accumulating the predicted number of good tests in order of reproducibility can mean accumulating the predicted number of good tests in ascending or descending order of reproducibility. In this embodiment, as an example, the predicted number of good tests is accumulated in ascending order of reproducibility. The value based on reproducibility can be a value obtained by performing arithmetic operations on the reproducibility value, but in this embodiment, as an example, the reproducibility value itself. In this case, using the comparison threshold and the reproducibility of each item, it is possible to determine whether to retest a device under test 100 that failed that item.

[0100] In step S107, the first determination unit 315 determines, for each item, whether to retest the device under test 100 that failed the test for that item. The first determination unit 315 can make this determination using the reproducibility for each item that failed the initial test, provided by the reproducibility acquisition unit 375 of the calculation unit 307, and the reproducibility threshold value provided by the threshold determination unit 313.

[0101] When the reproducibility of an item that failed the initial inspection is greater than a threshold, the first determination unit 315 may determine not to retest the device under test 100 that failed that item. When the reproducibility of an item that failed the initial inspection is less than the threshold, the first determination unit 315 may determine to retest the device under test 100 that failed that item. The first determination unit 315 may detect the device IDs of the device under test 100 that failed the item whose reproducibility of the item that failed the initial inspection is less than the threshold among the device IDs acquired by the result acquisition unit 305, and supply them to the testing apparatus 200 as targets for retesting.

[0102] According to the first determination process described above, when the predicted number of qualified devices for each item is accumulated sequentially according to the magnitude of each reproducibility, when the number corresponding to the acceptable lower limit of the qualified ratio is detected, or when the reproducibility corresponding to the previously predicted number of qualified devices is about to be reached, a threshold value is detected that satisfies the acceptable lower limit and does not require retesting. Therefore, by using this threshold value to determine whether to retest, the proportion of devices under test 100 that pass the test can be kept above the acceptable lower limit.

[0103] Furthermore, the reproducibility of the test varies depending on the processing steps applied to the device under test 100. For example, the reproducibility may differ between a device under test 100 processed at a low temperature and a device under test 100 processed at a high temperature. Therefore, when the test targets include devices under test 100 that have undergone different processing steps, it is preferable to perform the first determination process using different learning models 371 for each processing step.

[0104] [5. Modifications]

[0105] Figure 5A determination device 300A of a modified example is shown together with the test apparatus 200 and the device under test 100. The determination device 300A further includes a first determination unit 315A, a result acquisition unit 305A, a third determination unit 317, a second determination unit 319, and a learning processing unit 321. Figure 1 The judgment devices 300 shown are substantially the same as those shown, and the same reference numerals are assigned, and the description thereof will be omitted.

[0106] [5-2. First Determination Unit 315A]

[0107] The first determination unit 315A has the same configuration as the first determination unit 315 described above, but also supplies a determination result to the second determination unit 319 .

[0108] [5-3. Result Acquisition Unit 305A]

[0109] Result acquisition unit 305A has the same configuration as result acquisition unit 305 described above, but further acquires test results and the lot number of device under test 100 from storage unit 303. Result acquisition unit 305A associates the lot number with the device ID and supplies it to third determination unit 317. Result acquisition unit 305A also associates the initial test results and retest results corresponding to the device ID supplied by third determination unit 317 with the item ID of the test item, acquires them from storage unit 303, and supplies them to second determination unit 319.

[0110] [5-4. Third Determination Unit 317]

[0111] The third determination unit 317 determines whether to re-test the device under test 100. The third determination unit 317 can determine to re-test the device under test 100 regardless of the determination result of the first determination unit 315. The third determination unit 317 performs the test on the devices under test 100 included in each batch of a reference number (in this embodiment, as an example, 4), thereby determining whether to re-test. For example, when the third determination unit 317 performs the initial inspection on each device under test 100 included in the batch next to the reference number (in this embodiment, as an example, the 5th batch) of the batch, it can determine whether to re-test the device under test 100. As an example, the third determination unit 317 can make a determination between the above-mentioned steps S15 and S17.

[0112] The third determination unit 317 can detect the change in lot number based on the lot number supplied by the result acquisition unit 305A and count the number of tested lots. The third determination unit 317 can determine whether the device under test 100 should be retested after the number of tested lots reaches a reference number (4 in this embodiment) and then undergoes initial testing until the number reaches the next reference number (5 in this embodiment). The third determination unit 317 can extract the device ID of the device under test 100 to be retested from the device ID supplied by the result acquisition unit 305A and supply it to the test apparatus 200. This allows the test apparatus 200 to retest the target device under test 100.

[0113] The third determination unit 317 may also supply the device ID of the device under test 100 to be retested to the result acquisition unit 305A. The initial test results and retest results corresponding to the device ID are then associated with the item IDs of the test items and supplied to the second determination unit 319.

[0114] [5-5. Second Determination Unit 319]

[0115] Second determiner 319 uses the retest result and the determination result of first determiner 315 to determine whether to retrain learning model 371. The retest result used by second determiner 319 may be the result of the retest performed by third determiner 317, or may be the result of the retest performed by first determiner 315.

[0116] The second determination unit 319 can calculate TP (True Positive), TN (True Negative), FP (False Positive), and FN (False Negative) values from the following two results: the determination result of the first determination unit 315 (in this embodiment, as an example, the device ID of the device under test 100 determined to be subject to retesting) and the retest test results for each device ID provided by the result acquisition unit 305A. TP can be the number of devices under test 100 that were determined to be subject to retesting and passed the retest. TN can be the number of devices under test 100 that were determined not to be subject to retesting and failed the retest. FP can be the number of devices under test 100 that were determined to be subject to retesting and failed the retest. FN can be the number of devices under test 100 that were determined not to be subject to retesting and passed the retest.

[0117] If at least one of the conditions for preventing yield reduction (also referred to as yield conditions) and the conditions for reducing the number of retests (also referred to as retest reduction conditions) are not met, the second determination unit 319 may perform a relearning determination. Furthermore, the yield rate may be the ratio of devices under test 100 that pass the test (or retest) among all devices under test 100 that are tested.

[0118] The yield condition can be: when all devices under test 100 are retested, the ratio of the number of devices under test 100 that pass (FN + TP) to the number of devices under test 100 that pass the retest (TP) is greater than a threshold. In other words, the yield condition can be expressed using a threshold (TH1) as TP / (FN + TP) ≥ TH1.

[0119] The retest reduction condition can be: the ratio of the number of DUTs 100 that failed the initial test (TN+FP+FN+TP) to the number of DUTs 100 that did not undergo retest (TN+FN) is greater than a threshold. In other words, the retest reduction condition can be expressed as (TN+FN) / (TN+FP+FN+TP) ≥ TH2 using a threshold (TH2).

[0120] When it is determined that re-learning has been performed, the second determination unit 319 may supply the learning data extracted from the data acquired by the result acquisition unit 305A to the learning processing unit 321 .

[0121] [5-6. Learning Processing Unit 321]

[0122] The learning processing unit 321 performs learning processing on the learning model 371 using learning data. This learning data includes the item ID of at least one item that failed among the various items tested, as well as the results of the retest. The learning processing unit 321 can perform learning processing on the learning model 371 based on the determination result made by the second determination unit 319. As an example, the learning processing unit 321 can perform learning processing based on the learning data supplied by the second determination unit 319. The learning processing unit 321 can further perform learning processing using learning data including the position of the device under test 100 on the wafer and the deviation between the measured value and the ideal value. The learning processing unit 321 can obtain this data from the result acquisition unit 305A and the second determination unit 319.

[0123] According to the above-described determination device 300A, since the learning processing unit 321 performs the learning process of the learning model 371 , the accuracy of reproducibility can be further improved.

[0124] Furthermore, whether or not to relearn the learning model 371 is determined using the results of the recheck by the second determination unit 319 and the determination result by the first determination unit 315. Therefore, if the determination accuracy of the first determination unit 315 is low, relearning the learning model 371 can improve the determination accuracy of the first determination unit 315.

[0125] Furthermore, since re-inspection is mandatory every time a reference number of lots of devices under test 100 are tested, the accuracy of reproducibility and the learning accuracy of the learning model 371 can be reliably maintained at a high level.

[0126] [6. Other Modifications]

[0127] Furthermore, in the above-described embodiment and variations, the determination device 300 is described as including the calculation unit 307, the lower limit value acquisition unit 309, the prediction unit 311, and the threshold determination unit 313. However, these components may not be included. For example, the determination device 300 may obtain the calculation results of reproducibility, the predicted number of qualified devices, and the threshold value from the externally connected calculation unit 307, the prediction unit 311, and the threshold determination unit 313. Furthermore, the determination device 300 may perform the determination by the first determination unit 315 without using the predicted number of qualified devices and the threshold value. As an example, the first determination unit 315 may not retest a device under test that fails a benchmark number of items with a high reproducibility.

[0128] In addition, although the description has been made of the result acquisition unit 305 acquiring test results and the like from the testing apparatus 200 via the extraction unit 301 and the storage unit 303, the result acquisition unit 305 may also acquire the test results and the like directly from the testing apparatus 200. In this case, the result acquisition unit 305 may acquire test data in a data file (for example, an STDF file) and then extract the test results and the like from the data file. In this case, the determination apparatus 300 may not include the extraction unit 301.

[0129] The predicted number of qualified devices for each item calculated by the prediction unit 311 may be described as the number of DUTs 100 predicted to pass the retest for that item among those that failed the initial test for that item, but may also be a number representing another value. For example, the predicted number of qualified devices for each item may be the number of DUTs 100 predicted to pass the retest for that item among those that failed the initial test for that item.

[0130] In addition, although the description above shows that each test item is represented by a reproducibility value, a value corresponding to the entire test can also be represented. If the initial test result fails, the reproducibility value can indicate that the retest result has failed.

[0131] In addition, the calculation of reproducibility by the calculation unit 307 using the learning model 371 is described, but reproducibility can also be calculated without using the learning model 371. For example, the calculation unit 307 can calculate reproducibility from the following two results: the test results of a plurality of tests on a plurality of devices under test 100, and the test results of a plurality of retests. As an example, the calculation unit 307 can use the results of the initial inspection and retest of a plurality of devices under test 100, and calculate the reproducibility from the ratio of the number of failures in the initial inspection and the number of failures in the retest of each item. In this way, a statistical reproducibility can be obtained, which is based on the results of the initial inspection of a plurality of devices under test 100. When calculating reproducibility without using the learning model 371, since it is not necessary to input the test results of the initial inspection of each device under test 100 into the learning model 371 every time, the reproducibility can be calculated in advance for each item and stored internally in the determination device 300. Furthermore, the second determination unit 319 can use the results of the retest and the determination result of the first determination unit 315 to determine whether to update the calculated reproducibility. In this case, the calculation unit 307 can update the reproducibility based on the determination result of the second determination unit 319. Furthermore, the determination method of the second determination unit 319 can be the same as that of the above-mentioned modified example.

[0132] Furthermore, at least some of the functional units of the test device 200 and the determination devices 300 and 300A of the test system 1 can be implemented by a computer executing a program. The computer can perform calculations and logical processing according to a pre-created program. For example, it can be a special-purpose computer with a microprocessor, a general-purpose computer, or other computers.

[0133] Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where a block may represent: (1) a stage of a process in which an operation is performed, or (2) a section of a device having the task of performing an operation. Specific stages and sections may be implemented by dedicated circuits, programmable circuits supplied with computer-readable instructions stored on a computer-readable medium, and / or processors supplied with computer-readable instructions stored on a computer-readable medium. Dedicated circuits may include digital and / or analog hardware circuits, as well as integrated circuits (ICs) and / or discrete circuits. Programmable circuits may include reconfigurable hardware circuits, including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGAs), and memory components such as programmable logic arrays (PLAs).

[0134] A computer-readable medium may include any tangible device capable of storing instructions for execution by a suitable device. Consequently, a computer-readable medium having instructions stored therein provides a product containing executable instructions to create the means required for the operations specified in the flowchart or block diagram. Examples of computer-readable media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, and semiconductor storage media. More specific examples of computer-readable media include floppy disks (registered trademark), magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile discs (DVD), Blu-ray discs (RTM), memory sticks, integrated circuit cards, and the like.

[0135] The computer-readable instructions may include any source code or object code described in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or object-oriented programming languages such as Smalltalk, JAVA (registered trademark), C++, and conventional sequential programming languages such as the "C" programming language or similar programming languages.

[0136] Computer-readable instructions may be provided to a processor or programmable circuit of a programmable data processing device such as a general-purpose computer, a special-purpose computer, or other computer locally or via a wide-area network (WAN) such as a local area network (LAN) or the Internet to create the means necessary to perform the operations specified in the flowchart or block diagram and execute the computer-readable instructions. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, and microcontrollers.

[0137] Figure 6 This illustrates an example of a computer 2200 that can fully or partially embody various aspects of the present invention. A program installed on computer 2200 can cause computer 2200 to function as an apparatus or one or more sections of an apparatus associated with an embodiment of the present invention; or can execute such an operation or one or more sections, and / or can cause computer 2200 to execute a process or stage of such a process associated with an embodiment of the present invention. Such a program can cause computer 2200, via CPU 2212, to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0138] The computer 2200 of this embodiment includes a CPU 2212, RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected via a host controller 2210. The computer 2200 also includes a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an input / output unit such as an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes a ROM 2230 and conventional input / output units such as a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0139] The CPU 2212 controls each unit by operating according to programs stored in the ROM 2230 and the RAM 2214. The graphics controller 2216 obtains image data generated by the CPU 2212 from a frame buffer provided in the RAM 2214 or from the graphics controller itself, and displays the image data on the display device 2218.

[0140] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data to be used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0141] The ROM 2230 stores a startup program to be executed by the computer 2200 when it is started, and / or stores programs that depend on the hardware of the computer 2200. The input / output chip 2240 can also connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0142] The program is provided via a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from the computer-readable medium and installed in, for example, a hard disk drive 2224, RAM 2214, or ROM 2230, and is executed by the CPU 2212. The information processing described in the program is read into the computer 2200, enabling collaboration between the program and the various types of hardware resources described above. The apparatus or method can be configured such that information operations or processing are implemented based on the use of the computer 2200.

[0143] For example, when communication is performed between computer 2200 and an external device, CPU 2212 executes a communication program loaded into RAM 2214 and, based on the processing described in the communication program, commands communication interface 2222 to perform communication processing. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer provided in a recording medium such as RAM 2214, hard disk drive 2224, DVD-ROM 2201, or an IC card, and transmits the read transmission data to the network, or writes received data from the network into a receive buffer provided in the recording medium.

[0144] Furthermore, the CPU 2212 can read all or a necessary portion of files or databases stored in an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), or an IC card into the RAM 2214, and perform various types of processing on the data in the RAM 2214. The CPU 2212 then writes the processed data back to the external recording medium.

[0145] Various types of programs, data, tables, and various types of information, such as databases, can be stored in a recording medium and subjected to information processing. CPU 2212 can perform various types of processing, including various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, and information retrieval / replacement, specified by the program's instruction sequence, on the data read from RAM 2214, and write the results back to RAM 2214. Furthermore, CPU 2212 can search for information in files, databases, and the like within the recording medium. For example, when a plurality of entries having attribute values for a first attribute, each associated with an attribute value for a second attribute, are stored in the recording medium, CPU 2212 searches for an entry that meets the conditions specified by the attribute value for the first attribute from the plurality of entries and reads the attribute value for the second attribute stored within the entry. This allows the CPU to obtain the attribute value for the second attribute associated with the first attribute that satisfies the predetermined conditions.

[0146] The programs or software modules described above may be stored in a computer-readable medium on or near the computer 2200. Furthermore, a hard disk or a RAM-like recording medium provided in a server system connected to a dedicated communication network or the Internet may be used as a computer-readable medium, thereby providing the program to the computer 2200 via the network.

[0147] While the present invention has been described above using embodiments, the scope of protection of the present invention is not limited to the aforementioned embodiments. Those skilled in the art will appreciate that various modifications or improvements may be made to the aforementioned embodiments. As will be apparent from the claims, such modifications or improvements are also encompassed within the scope of protection of the present invention.

[0148] It should be noted that the order of execution of actions, procedures, steps, and stages, etc., in the apparatuses, systems, programs, and methods described in the claims, specifications, and drawings may be any order unless specifically indicated by "before," "before," etc., and unless the output of a previous process is used in a subsequent process. The use of phrases such as "first," "next," etc. to describe the flow of actions in the claims, specifications, and drawings for convenience does not necessarily imply that the actions must be performed in this order.

[0149] Description of Reference Numerals

[0150] 1: Test system

[0151] 100: Device under test

[0152] 101: Wafer

[0153] 200:Test device

[0154] 201: Tester body

[0155] 203:Test head

[0156] 205: Detector

[0157] 231: Probe

[0158] 300: Determination device

[0159] 301: Extraction Department

[0160] 303: Storage

[0161] 305: Result Acquisition Unit

[0162] 307: Computing Department

[0163] 309: Lower limit value acquisition unit

[0164] 311: Forecast Department

[0165] 313: Threshold determination unit

[0166] 315: First Judgment Unit

[0167] 317: Judgment Section 3

[0168] 319: Second Judgment Section

[0169] 321: Learning Processing Department

[0170] 371: Learning Model

[0171] 373: Supply Department

[0172] 375: Reproducibility acquisition unit

[0173] 2200: Computer

[0174] 2201:DVD-ROM

[0175] 2210:Host Controller

[0176] 2212:CPU

[0177] 2214:RAM

[0178] 2216: Graphics Controller

[0179] 2218: Display device

[0180] 2220: Input / Output Controller

[0181] 2222: Communication interface

[0182] 2224: Hard Drive

[0183] 2226:DVD-ROM drive

[0184] 2230:ROM

[0185] 2240: Input / Output Chip

[0186] 2242:Keyboard

Claims

1. A determination device comprising: a result acquisition unit that acquires test results of a plurality of items of tests performed on the device under test; a calculation unit for calculating, for each of the plurality of items, reproducibility of test results obtained when the test is previously performed a plurality of times on a plurality of devices under test, the calculation unit comprising: A learning model that corresponds to the test results of the plurality of items input and outputs a predicted result of the retest; a supply unit that supplies the learning model with the test results of the plurality of items acquired by the result acquisition unit; and, a reproducibility acquisition unit that acquires the reproducibility from a prediction result of a retest output by the learning model in response to the test results of the plurality of items being supplied to the learning model; and The first determination unit determines whether to retest a device under test that failed a test of a corresponding item based on the reproducibility calculated by the calculation unit for each of the plurality of items using a learning model that has learned the reproducibility.

2. The determination device according to claim 1, wherein: The system further includes a learning processing unit that executes a learning process of the learning model using learning data, the learning data including an item ID of at least a failed item among the plurality of items and a result of a retest.

3. The determination device according to claim 2, wherein: further comprising a second determination unit that determines whether to relearn the learning model using a result of the retest and a determination result of the first determination unit; Furthermore, the learning processing unit executes a learning process of the learning model based on the determination result made by the second determination unit.

4. The determination device according to claim 1, wherein: The calculation unit calculates the reproducibility based on test results of a plurality of the tests performed on a plurality of devices under test and test results of a plurality of retests.

5. The determination device according to claim 4, wherein: further comprising a second determination unit for determining whether to update the reproducibility using a result of the retest and a determination result of the first determination unit; Furthermore, the calculation unit updates the reproducibility based on the determination result of the second determination unit.

6. The determination device according to claim 3 or 5, wherein: further comprising a third determination unit that determines to retest the devices under test regardless of a determination result of the first determination unit, based on executing the test for the devices under test included in every reference number of lots; Furthermore, the second determination unit performs determination using the result of the retest executed by the third determination unit.

7. The determination device according to claim 1, wherein: Also features: an extraction unit that acquires a data file containing test results and extracts the test results of each item from the data file; and a storage unit for storing the test results extracted by the extraction unit; Furthermore, the result acquisition unit acquires the test results of the plurality of items from the storage unit. 8 . A test system comprising: the determination device according to claim 1 ; and a test device that performs a plurality of tests on a device under test.

9. A determination method comprising: A result acquisition phase, which acquires test results of a plurality of items of tests performed on the device under test; The calculation stage calculates, for each of the plurality of items, the reproducibility of the test results when the test is previously performed multiple times on the plurality of devices under test. The calculation stage includes: In the output stage, the learning model outputs the predicted results of the retest based on the test results of the plurality of items input; a supplying stage, which supplies the learning model with the test results of the plurality of items acquired in the result acquiring stage; and, a reproducibility acquisition phase, corresponding to supplying the test results of the plurality of items to the learning model, acquiring the reproducibility from the prediction results of the retest output by the learning model; and The first determination stage determines whether to retest a device under test that failed a test of a corresponding item based on the reproducibility calculated for each of the plurality of items in the calculation stage using a learning model that has learned the reproducibility.

10. A recording medium recording a determination program, wherein the determination program is executed by a computer and causes the computer to function as: A result acquisition unit acquires test results of a plurality of items of tests performed on a device under test; and a calculation unit calculates reproducibility for each of the plurality of items, comprising: A learning model that corresponds to the test results of the plurality of items input and outputs a predicted result of the retest; a supply unit that supplies the learning model with the test results of the plurality of items acquired by the result acquisition unit; and, a reproducibility acquisition unit that acquires the reproducibility from a prediction result of a retest output by the learning model in response to the test results of the plurality of items being supplied to the learning model; and A first determination unit determines whether to retest a device under test that has failed a test of a corresponding item based on the reproducibility calculated by the calculation unit using a learning model that has learned the reproducibility.

Images