Evaluation Method, System, Wafer Testing System and Storage Medium for Test Data

Abstract

An embodiment of the present application discloses a method, a system, a wafer testing system and a storage medium for evaluating test data. The method for evaluating test data includes: obtaining test data of multiple test programs; each test program includes multiple test items; for each of the test programs, calculating a correlation coefficient of each of the test items according to the test data; drawing a difference analysis graph for every two of the multiple test programs according to the correlation coefficients between the respective test items in different test programs; wherein, the horizontal axis and the vertical axis of the difference analysis graph respectively correspond to one test program; and evaluating the differences of the respective test items in two different test programs according to the difference analysis graph.

Description

Technical Field

[0001] The present application relates to the field of semiconductor technology, and particularly to a method, a system, a wafer test system and a storage medium for evaluating test data. Background Art

[0002] After the wafer production is completed, wafer electrical testing (Circuit Probing) needs to be performed, and a large amount of wafer test data is generated. In actual analysis, it is necessary to analyze the correlation of the wafer test data obtained from different electrical tests to find the correlation between different test items. In actual mass production, the amount of these data is huge, including thousands of test items, and each test item will have millions of data.

[0003] However, during the production process, the test program often needs to be updated and optimized. Therefore, when detecting the correlation of the wafer test data, it is more difficult to detect the abnormal correlation between the wafer test data brought by different test programs. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a method, a system, a wafer test system and a storage medium for evaluating test data to solve at least one problem existing in the prior art.

[0005] To achieve the above object, the technical solution of the embodiments of the present application is implemented as follows:

[0006] In a first aspect, embodiments of the present application provide a method for evaluating test data, the method including:

[0007] Obtain test data of multiple test programs; each test program includes multiple test items;

[0008] For each of the test programs, calculate the correlation coefficient of each of the test items according to the test data;

[0009] Draw a difference analysis chart for every two of the multiple test programs according to the correlation coefficients between the test items in different test programs; wherein, the horizontal axis and the vertical axis of the difference analysis chart respectively correspond to a test program;

[0010] Evaluate the differences of each of the test items in two different test programs according to the difference analysis chart.

[0011] In an optional implementation manner, before calculating the correlation coefficient of each of the test items, the method further includes:

[0012] According to the test data of each of the said test items in different test procedures, a box plot is drawn for each of the said test items;

[0013] For each of the said test items, the abnormal test data is removed by using the box plot.

[0014] In an alternative embodiment, the calculating the correlation coefficient of each of the said test items according to the test data includes:

[0015] According to the test data after removing the abnormal test data, the correlation coefficient between each of the said test items is calculated.

[0016] In an alternative embodiment, each test procedure includes the same test items.

[0017] In an alternative embodiment, the method further includes:

[0018] Through the box plot, the differences of each of the said test items in different test procedures are evaluated.

[0019] In an alternative embodiment, the evaluating the differences of each of the said test items in different test procedures includes:

[0020] For each of the said test items, if the median of one test procedure among multiple test procedures is greater than the upper quartile of another test procedure or less than the lower quartile of another test procedure, it is evaluated that there are significant differences in this test item among different test procedures.

[0021] In an alternative embodiment, the calculating the correlation coefficient of each of the said test items includes:

[0022] The correlation coefficient between each of the said test items is calculated by using the Pearson correlation coefficient calculation formula.

[0023] In an alternative embodiment, for the first quadrant and the third quadrant of the difference analysis graph, the evaluating the differences of each of the said test items in two different test procedures includes:

[0024] For every two test items among multiple test items, if the difference value between the correlation coefficient of the two test items in one test procedure and the correlation coefficient of the two test items in another test procedure is greater than the first preset value, it is evaluated that there are significant differences in the two test items in two different test procedures.

[0025] In an alternative embodiment, for the second quadrant and the fourth quadrant of the difference analysis graph, the evaluating the differences of each of the said test items in two different test procedures further includes:

[0026] For every two test items among multiple test items, if the distance from the coordinate point formed by the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program to the origin of the difference analysis graph is greater than a second preset value, it is evaluated that there is a significant difference between the two test items in the two different test programs.

[0027] In an alternative embodiment, the first preset value is greater than the second preset value.

[0028] In an alternative embodiment, the obtaining of the test data of multiple test programs includes:

[0029] Determine the target product corresponding to multiple test programs according to the set parameters;

[0030] Obtain the test data of the preset test categories of the target product in the test database;

[0031] Wherein, the preset test categories include multiple test categories, and each test category corresponds to multiple test items.

[0032] In an alternative embodiment, before calculating the correlation coefficient of each of the test items according to the test data, the method further includes:

[0033] Merge the test data of the preset test categories according to the set parameters to generate a merged data table.

[0034] In an alternative embodiment, the set parameters include a test program identifier, a product identifier, a wafer identifier, a die identifier, and a process step identifier.

[0035] In an alternative embodiment, a difference report is generated according to the differences of each of the test items evaluated in the two different test programs;

[0036] Regularly upload the difference report.

[0037] In an alternative embodiment, the test data is the test data generated in the wafer electrical test stage.

[0038] In a second aspect, an embodiment of the present application provides an evaluation system for test data, and the system includes:

[0039] A data acquisition module, configured to acquire the test data of multiple test programs; each test program includes multiple test items;

[0040] A calculation module, configured to calculate the correlation coefficient of each of the test items according to the test data for each of the test programs;

[0041] A first drawing module, configured to draw a difference analysis diagram for every two of a plurality of test programs according to the correlation coefficients between the respective test items in different test programs; wherein, the horizontal axis and the vertical axis of the difference analysis diagram respectively correspond to one test program.

[0042] A first evaluation module, configured to evaluate the differences between the respective test items in two different test programs according to the difference analysis diagram.

[0043] In a third aspect, an embodiment of the present application provides a wafer test system, including an evaluation system for test data as described in the second aspect.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, storing a computer program, where when the computer program is executed by a processor, the evaluation method for test data described in any item of the first aspect is implemented.

[0045] In the technical solution provided by the present application, an evaluation method for test data is provided. In this method, a difference analysis diagram is drawn for every two of a plurality of test programs according to the correlation coefficients between the respective test items in different test programs, and the differences between the respective test items in two different test programs are evaluated according to the difference analysis diagram. Thus, through the evaluation method for test data provided by the present application, the difference evaluation of each test item in different test programs is realized. According to this difference evaluation, the abnormal correlation differences between the respective test items brought by different test programs can be obtained. Description of the Drawings

[0046] In the drawings, unless otherwise specified, the same reference numerals throughout the several views denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in accordance with the present application and should not be regarded as limiting the scope of the present application.

[0047] Figure 1 It is a schematic flowchart of the implementation of an evaluation method for test data provided by an embodiment of the present application.

[0048] Figure 2 It is a test data table obtained in a specific embodiment of the present application.

[0049] Figure 3A It is a box plot of D_VDLY_DQ_DC provided by an embodiment of the present application.

[0050] Figure 3B It is a box plot of IFSB_DC provided by an embodiment of the present application.

[0051] Figure 3CThe box plot of M_DELAY_DC provided by the embodiments of the present application.

[0052] Figure 4 The schematic diagram of the difference analysis chart provided by the embodiments of the present application;

[0053] Figure 5 For Figure 4 The data table corresponding to the coordinate points with significant differences in the difference analysis chart shown;

[0054] Figure 6A The schematic diagram of the correlation between D_LBIAS_DQ_DC and D_VDLY_DQ_DC provided by the embodiments of the present application;

[0055] Figure 6B The schematic diagram of the correlation between D_LBIAS_DQ_DC and IDD2P_DC provided by the embodiments of the present application;

[0056] Figure 6C The schematic diagram of the correlation between D_LBIAS_DQ_DC and IDD3P_DC provided by the embodiments of the present application;

[0057] Figure 6D The schematic diagram of the correlation between D_LBIAS_DQ_DC and IFSB_DC provided by the embodiments of the present application;

[0058] Figure 7 The structural schematic diagram of an evaluation system for test data provided by the embodiments of the present application. Detailed implementation manners

[0059] The following description gives a lot of specific details to provide a more thorough understanding of the present application. However, it is obvious to those skilled in the art that the present application can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present application, some well-known technical features are not described; that is, not all features of the actual embodiments are described here, and the well-known functions and structures are not described in detail.

[0060] In addition, the drawings are only schematic diagrams of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0061] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all steps. For example, some steps can be further decomposed, while some steps can be combined or partially combined. Therefore, the actual execution order may be changed according to the actual situation.

[0062] The purpose of the terms used herein is only to describe specific embodiments and is not a limitation of the present application. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "comprising" and / or "including", when used in this specification, specify the presence of the stated features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups. As used herein, the term "and / or" includes any and all combinations of the related listed items.

[0063] During the integrated circuit manufacturing process, a large amount of data is generated in real time, such as electrical test data, etc., and these data need to be analyzed. The relevant situations of the wafer test data obtained from different electrical tests are analyzed in a timely manner to find out the correlation between different test items, so as to make corresponding adjustments or optimizations to the test items, etc., and then ensure that the produced products have a high yield and reliability. However, during the production process, in addition to analyzing the correlation between different test items, it is also necessary to analyze the correlation between different test items brought by different test procedures. At present, the analysis of these data is mostly carried out manually by engineers. However, with the increasing amount of test procedures and test items, the current manual analysis method can no longer process these big data comprehensively and quickly, and it also takes a long time to view the results. Engineers spend a lot of time on data analysis that has little correlation with the key parameters of subsequent process improvement or the electrical performance of the final device. They are not sensitive enough to judge abnormal data, cannot quickly and accurately find abnormal data, and thus cannot timely discover the abnormal conditions of test procedures and test items, resulting in the inability to timely make corresponding adjustments or optimizations to test procedures and test items, etc., and causing the performance of the final device to be unable to be improved for a long time. Obviously, the current manual data analysis method can no longer meet the analysis requirements of big data in integrated circuit manufacturing.

[0064] Therefore, the following embodiments of the present application are proposed.

[0065] An embodiment of the present application provides a method for evaluating test data. Figure 1 For the implementation flow schematic diagram of a method for evaluating test data provided by an embodiment of the present application, as Figure 1 shown, the method includes the following steps:

[0066] Step 110: Obtain the test data of multiple test programs; each test program includes multiple test items.

[0067] In some embodiments, the test data is the test data generated in the wafer electrical test stage. In other embodiments, the test data can also be the test data generated in other wafer test stages, such as wafer acceptance test, fault test, failure pattern analysis test, etc.

[0068] It should be noted that the following takes the test data generated in the wafer electrical test stage as an example for illustration.

[0069] In the embodiments of the present application, step 110 specifically includes: determining the target product corresponding to multiple test programs according to the set parameters; obtaining the test data of the preset test categories of the target product in the test database; where the preset test categories include multiple test categories, and each test category corresponds to multiple test items. Here, the set parameters include test program identifier (PROGRAM ID), product identifier (PRODUCT ID), wafer identifier (WAFER ID), die identifier (CHIPID), and process step identifier (STEP ID). In a specific implementation manner, for each test program, according to the set test program identifier, product identifier, and process step identifier, randomly retrieve the test data of 400 wafers in the wafer-level database, that is, each test program retrieves the test data of 400 wafers. It should be noted that the number of wafers retrieved can be determined according to actual needs.

[0070] In a specific implementation manner, for each test program, continue to retrieve all the test items of the three test categories of Data Collection (DC), Fail Region Count (FRC), and Redundance Data (RD) from the test data of the 400 wafers retrieved. Here, the preset test categories include DC, FRC, and RD.

[0071] Step 120: For each of the test programs, calculate the correlation coefficient of each of the test items according to the test data.

[0072] In the embodiments of the present application, before step 120, the evaluation method of the test data further includes: merging the test data of the preset test categories according to the set parameters to generate a merged data table.

[0073] In a specific embodiment, the test data of the test items of the three test categories of DC, FRC, and RD are merged into a data table through wafer identification, die identification, and test procedures. In the embodiment of the present application, when obtaining test data, the test data is first merged according to the classification of test items, so that in the subsequent process, the test data can be analyzed and evaluated faster and more conveniently.

[0074] Figure 2 This is the test data table obtained in a specific embodiment of the present application. From the Figure 2 table shown, for the product with the full product name of DQRMANACXX and the product identification of DQRMA, the test data of test procedures E0684 and E0685 are retrieved under the PRE_HT process step. Among them, for test procedure E0684, the test data of 125 wafers with the probe card identification of DQRMAFB0767P0004 and the tester of CPTA140 are retrieved; the test data of 84 wafers with the probe card identification of DQRMAFB0767P0005 and the tester of CPTA107 are retrieved; the test data of 40 wafers with the probe card identification of DQRMAFB0767P0008 and the tester of CPTA136 are retrieved; the test data of 151 wafers with the probe card identification of DQRMAFB0767P0010 and the tester of CPTA137 are retrieved, totaling 400 wafers of test data. For test procedure E0685, the test data of 75 wafers with the probe card identification of DQRMAFB0767P0003 and the tester of CPTA120 are retrieved; the test data of 100 wafers with the probe card identification of DQRMAFB0767P0004 and the tester of CPTA140 are retrieved; the test data of 148 wafers with the probe card identification of DQRMAFB0767P0005 and the tester of CPTA107 are retrieved; the test data of 77 wafers with the probe card identification of DQRMAFB0767P0010 and the tester of CPTA137 are retrieved, totaling 400 wafers of test data.

[0075] In the embodiment of the present application, before step 120, the test data evaluation method further includes: drawing a box plot for each test item according to the test data of each test item in different test procedures; for each test item, using the box plot to remove abnormal test data. The abnormal test data exceeding the lower bound (Q1 - 1.5×IQR) and the upper bound (Q3 + 1.5×IQR) is removed through the box plot. Among them, Q1 is the lower quartile, Q3 is the upper quartile, IQR is the interquartile range, and IQR = Q3 - Q1.

[0076] Here, an example is given to illustrate obtaining the test data of two test programs. The two test programs are E0684 and E0685 respectively. Each test program includes the same test items. In some embodiments, the test items included in each test program can be divided into three test categories, namely DC, FRC, and RD; among them, the test items included in DC are D_LBIAS_DQ_DC, D_VDLY_DQ_DC, IDD2P_DC, IDD3P_DC, IFSB_DC, M_CMDDLY_DC, M_DELAY_DC, M_ODP_DC, M_OSC_DC, M_TAA_DC, PWR_SHORT_DC, VDDSHT1_DC, VDDSHT2_DC, VDDSHT4_DC, VDDSHT5_DC, etc.; the test items included in RD are p_RD, S_RD, z_RD, etc.; the test items included in FRC are LZ_BLS1_FRC, S_PAUSB_288_FRC, etc.

[0077] Here, an example is given to illustrate with three test items: D_VDLY_DQ_DC, IFSB_DC, and M_DELAY_DC. Figure 3A This is the box plot of D_VDLY_DQ_DC provided by the embodiment of the present application. Figure 3B This is the box plot of IFSB_DC provided by the embodiment of the present application. Figure 3C This is the box plot of M_DELAY_DC provided by the embodiment of the present application. As Figures 3A to 3C shown, the abscissa of the box plot is different test programs, and the ordinate is the test data.

[0078] In the embodiment of the present application, through the box plot, the differences of each test item in different test programs are evaluated. As Figures 3A to 3C shown, according to the box plots corresponding to different test items, the differences of each test item in the two test programs E0684 and E0685 can be roughly evaluated.

[0079] In the embodiment of the present application, for each test item, if the median of one test program among multiple test programs is greater than the upper quartile of another test program or less than the lower quartile of another test program, it is evaluated that there are significant differences in this test item among different test programs. As Figure 3AAs shown, the median of the test item D_VDLY_DQ_DC in the test program E0684 is 850, the upper quartile is 860, and the lower quartile is 835; the median of the test item D_VDLY_DQ_DC in the test program E0685 is 865, the upper quartile is 875, and the lower quartile is 855. The median (850) of the test item D_VDLY_DQ_DC in the test program E0684 is less than the lower quartile (855) of the test program E0685, and the median (865) of the test item D_VDLY_DQ_DC in the test program E0685 is greater than the upper quartile (860) of the test program E0684. Therefore, it is evaluated that there is a significant difference in the test item D_VDLY_DQ_DC between the test programs E0684 and E0685.

[0080] As Figure 3B shown, the median of the test item IFSB_DC in the test program E0684 is 33, the upper quartile is 38, and the lower quartile is 29; the median of the test item IFSB_DC in the test program E0685 is 29, the upper quartile is 33, and the lower quartile is 25. The median (33) of the test item IFSB_DC in the test program E0684 is not greater than the upper quartile (33) of the test program E0685, and the median (29) of the test item IFSB_DC in the test program E0685 is not less than the lower quartile (29) of the test program E0684. Therefore, it is evaluated that there is no significant difference in the test item IFSB_DC between the test programs E0684 and E0685.

[0081] As Figure 3C shown, the median of the test item M_DELAY_DC in the test program E0684 is 3.21, the upper quartile is 3.32, and the lower quartile is 3.11; the median of the test item M_DELAY_DC in the test program E0685 is 3.3, the upper quartile is 3.4, and the lower quartile is 3.24. The median (3.21) of the test item M_DELAY_DC in the test program E0684 is less than the lower quartile (3.24) of the test program E0685, and the median (3.3) of the test item M_DELAY_DC in the test program E0685 is not greater than the upper quartile (3.32) of the test program E0684. Therefore, it is evaluated that there is a significant difference in the test item M_DELAY_DC between the test programs E0684 and E0685.

[0082] In the embodiments of the present application, through the box plot of each test item, the distribution of each test item in different test programs can be statistically analyzed, and whether there is a significant difference in each test item in different test programs can be evaluated through the box plot.

[0083] In the embodiment of the present application, the specific process of step 120 is as follows: Calculate the correlation coefficient between each of the test items according to the test data after removing the abnormal test data.

[0084] In the embodiment of the present application, the correlation coefficient between each of the test items is calculated using the Pearson correlation coefficient calculation formula. According to the Pearson correlation coefficients between each of the test items in different test programs, a difference analysis graph is plotted for every two test programs among multiple test programs; wherein, the horizontal axis and the vertical axis of the difference analysis graph respectively correspond to one test program.

[0085] The Pearson correlation coefficient is used to measure the linear relationship between variables. The calculation formula of the Pearson correlation coefficient is:

[0086]

[0087] The Pearson correlation coefficient formula is defined as: The Pearson correlation coefficient ρ of two variables (x, y) x,y is equal to the covariance cov(x, y) between them divided by the product of their respective standard deviations σ x σ y .

[0088] In some embodiments, the calculated correlation coefficient includes any one of the Pearson correlation coefficient, the Spearman correlation coefficient, or the Kendall correlation coefficient. In other embodiments, the calculated correlation coefficient may also be other correlation coefficients in the art.

[0089] Step 130: According to the correlation coefficients between each of the test items in different test programs, plot a difference analysis graph for every two test programs among multiple test programs; wherein, the horizontal axis and the vertical axis of the difference analysis graph respectively correspond to one test program.

[0090] Step 140: Evaluate the differences between each of the test items in two different test programs according to the difference analysis graph.

[0091] In the embodiment of the present application, for the first quadrant and the third quadrant of the difference analysis graph, step 140 includes: For every two test items among multiple test items, if the difference value between the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program is greater than a first preset value, it is evaluated that there is a significant difference between the two test items in the two different test programs.

[0092] Figure 4 is a schematic diagram of the difference analysis graph provided by the embodiment of the present application. It should be noted thatFigure 4 Taking the test programs E0684 and E0685 as examples for illustration. Figure 4 In the difference analysis graph, the abscissa is the test program E0684, and the ordinate is the test program E0685. Figure 4 The coordinate points represented by the hollow dots in the figure are the coordinate points corresponding to the two test items with significant differences. As Figure 4 shown, for the first and third quadrants of the difference analysis graph, if the difference value between the correlation coefficients of two test items in the test program E0684 and the correlation coefficients of these two test items in the test program E0685 is greater than 0.5, it is evaluated that there are significant differences between the two test items in the test programs E0684 and E0685.

[0093] In the embodiments of the present application, for the second and fourth quadrants of the difference analysis graph, the step 140 further includes: for every two test items among multiple test items, if the distance from the coordinate point formed by the correlation coefficients of the two test items in one test program and the correlation coefficients of these two test items in another test program to the origin of the difference analysis graph is greater than a second preset value, it is evaluated that there are significant differences between the two test items in the two different test programs. As Figure 4 shown, for the second and fourth quadrants of the difference analysis graph, if the distance from the coordinate point formed by the correlation coefficients of the two test items in the test program E0684 and the correlation coefficients of these two test items in the test program E0685 to the origin of the difference analysis graph is greater than 0.4, it is evaluated that there are significant differences between the two test items in the test programs E0684 and E0685.

[0094] Here, in practical applications, the first preset value can be 0.5, the second preset value can be 0.4, and the first preset value is greater than the second preset value. Since the values of the correlation coefficients corresponding to the coordinate points in the first and third quadrants are of the same sign (either all positive or all negative), it indicates that the correlation of the test items corresponding to the coordinate points in the first and third quadrants in two different test programs is similar. Therefore, the first preset value is set to a value greater than the second preset value here. For example, the first preset value is set to be greater than or equal to 0.5, in order to reflect that the two correlation coefficients differ greatly by increasing the difference, and further indicate that the corresponding test items have significant differences in two different test programs. And the values of the correlation coefficients corresponding to the coordinate points in the second and fourth quadrants are of different signs (one positive and one negative), indicating that the correlation of the test items corresponding to the coordinate points in the second and fourth quadrants in two different test programs has a certain difference. So as long as the second preset value is greater than or equal to 0.4, it is sufficient to indicate that the corresponding test items have significant differences in two different test programs.

[0095] Figure 5 For Figure 4 the data table corresponding to the coordinate points with significant differences in the difference analysis diagram shown. Figure 5 In it, R1 represents the correlation coefficient between the X1 test item and the X2 test item in the test program E0684, R2 represents the correlation coefficient between the X1 test item and the X2 test item in the test program E0685, and difference represents the difference value between R1 and R2. From Figure 5 it can be seen that the coordinate points corresponding to the difference values with significant differences are all in the first quadrant and the third quadrant, that is, the coordinate points where the difference value between R1 and R2 is greater than 0.5, or in the second quadrant and the fourth quadrant, that is, the coordinate points formed by R1 and R2 whose distance to the origin is greater than 0.4.

[0096] Here, take X1 as D_LBIAS_DQ_DC and X2 as D_VDLY_DQ_DC as an example for illustration. Figure 6A It is a schematic diagram of the correlation between D_LBIAS_DQ_DC and D_VDLY_DQ_DC provided by the embodiment of the present application. Figure 6A In the schematic diagram of the correlation, the abscissa is the test data of D_LBIAS_DQ_DC, and the ordinate is the test data of D_VDLY_DQ_DC. It should be noted that Figure 6A in it, the correlation coefficient between D_LBIAS_DQ_DC and D_VDLY_DQ_DC in the test program E0684 is 0.602, and the correlation coefficient between D_LBIAS_DQ_DC and D_LBIAS_DQ_DC in the test program E0685 is 0.048. As Figure 6A shown, the test data distributions of the test items D_LBIAS_DQ_DC and D_VDLY_DQ_DC in the test programs E0684 and E0685 have great differences, and thus it can be evaluated that there are significant differences in the test items D_LBIAS_DQ_DC and D_VDLY_DQ_DC in the test programs E0684 and E0685.

[0097] Here, take X1 as D_LBIAS_DQ_DC and X2 as IDD2P_DC as an example for illustration. Figure 6B It is a schematic diagram of the correlation between D_LBIAS_DQ_DC and IDD2P_DC provided by the embodiment of the present application. Figure 6B In the schematic diagram of the correlation, the abscissa is the test data of D_LBIAS_DQ_DC, and the ordinate is the test data of IDD2P_DC. It should be noted that Figure 6BThe correlation coefficient between D_LBIAS_DQ_DC and IDD2P_DC in test program E0684 is -0.454, and the correlation coefficient between D_LBIAS_DQ_DC and IDD2P_DC in test program E0685 is 0.056. As Figure 6B shown, there are significant differences in the test data distributions of test items D_LBIAS_DQ_DC and IDD2P_DC in test programs E0684 and E0685. Thus, it can be evaluated that there are significant differences in test items D_LBIAS_DQ_DC and IDD2P_DC in test programs E0684 and E0685.

[0098] Here, take X1 as D_LBIAS_DQ_DC and X2 as IDD3P_DC for illustration. Figure 6C This is the correlation diagram of D_LBIAS_DQ_DC and IDD3P_DC provided by the embodiment of the present application. Figure 6C In the correlation diagram, the abscissa is the test data of D_LBIAS_DQ_DC, and the ordinate is the test data of IDD3P_DC. It should be noted that Figure 6C the correlation coefficient between D_LBIAS_DQ_DC and IDD3P_DC in test program E0684 is -0.573, and the correlation coefficient between D_LBIAS_DQ_DC and IDD3P_DC in test program E0685 is -0.053. As Figure 6C shown, there are significant differences in the test data distributions of test items D_LBIAS_DQ_DC and IDD3P_DC in test programs E0684 and E0685. Thus, it can be evaluated that there are significant differences in test items D_LBIAS_DQ_DC and IDD3P_DC in test programs E0684 and E0685.

[0099] Here, take X1 as D_LBIAS_DQ_DC and X2 as IFSB_DC for illustration. Figure 6D This is the correlation diagram of D_LBIAS_DQ_DC and IFSB_DC provided by the embodiment of the present application. Figure 6D In the correlation diagram, the abscissa is the test data of D_LBIAS_DQ_DC, and the ordinate is the test data of IFSB_DC. It should be noted that Figure 6D the correlation coefficient between D_LBIAS_DQ_DC and IFSB_DC in test program E0684 is -0.455, and the correlation coefficient between D_LBIAS_DQ_DC and IFSB_DC in test program E0685 is 0.022. As Figure 6DAs shown, there are significant differences in the test data distributions of the test items D_LBIAS_DQ_DC and IFSB_DC in test programs E0684 and E0685. Thus, it can be evaluated that there are significant differences in the test items D_LBIAS_DQ_DC and IFSB_DC in test programs E0684 and E0685.

[0100] Thus, by calculating the correlation coefficients between the test items in different test programs, plotting a difference analysis graph for every two of the multiple test programs, and based on this difference analysis graph, it is possible to evaluate the differences between the various test items in two different test programs. Based on this difference evaluation, it is possible to obtain the abnormal differences in the correlations between the various test items caused by different test programs. Thus, engineers can timely make corresponding adjustments or optimizations to the test programs or test items based on this abnormal difference, and ultimately improve the device performance.

[0101] In the embodiment of the present application, after step 140, the test data evaluation method further includes: generating a difference report based on the evaluated differences between the various test items in two different test programs; and uploading the difference report regularly. In a specific implementation, regularly uploading the difference report can be to regularly send the difference report by email to the engineer.

[0102] In the technical solution provided by the present application, test data is automatically extracted from the database by setting parameters, and the test data of the preset test categories is merged to facilitate subsequent data analysis. Then, box plots are plotted for each test item by statistical methods, and the differences between each test item in different test programs are evaluated through the box plots, so as to find out the test items with significant differences in different test programs. And based on the correlation coefficients between the various test items in different test programs, a difference analysis graph is plotted for every two of the multiple test programs, and the differences between the various test items in two different test programs are evaluated based on the difference analysis graph, so as to find out the pairs of test items with significant differences in different test programs. Then, the generated difference report is regularly sent to the engineer by an automated method, so that the engineer can timely find and fix problems through the difference report.

[0103] Through the test data evaluation method provided by the present application, the differences between the various test items in different test programs are evaluated. It can also help engineers determine whether the abnormal changes in the correlations between the various test items are related to different test programs. Based on the differences between the various test items in different test programs, the abnormal differences in the correlations between the various test items caused by different test programs can be obtained. Thus, engineers can timely make corresponding adjustments or optimizations to the test programs or test items based on this abnormal difference, and ultimately improve the device performance.

[0104] Based on the same technical concept as the evaluation method for the foregoing test data, an embodiment of the present application provides an evaluation system for test data. Figure 7 As shown in the structural schematic diagram of an evaluation system for test data provided by an embodiment of the present application, Figure 7 as shown, the evaluation system 700 for test data includes:

[0105] A data acquisition module 710, configured to acquire test data of multiple test programs; each test program includes multiple test items;

[0106] A calculation module 720, configured to calculate the correlation coefficient of each test item for each test program according to the test data.

[0107] A first plotting module 730, configured to plot a difference analysis graph for every two test programs among multiple test programs according to the correlation coefficients between the test items in different test programs; wherein, the horizontal axis and the vertical axis of the difference analysis graph respectively correspond to a test program.

[0108] A first evaluation module 740, configured to evaluate the differences of each test item in two different test programs according to the difference analysis graph.

[0109] In some embodiments, the evaluation system 700 for test data further includes: a second plotting module 750, configured to plot a box plot for each test item according to the test data of each test item in different test programs; for each test item, abnormal test data is removed by using the box plot.

[0110] In some embodiments, the calculation module 720 is specifically configured to calculate the correlation coefficient between each test item according to the test data after removing the abnormal test data.

[0111] In some embodiments, each test program includes the same test items.

[0112] In some embodiments, the evaluation system 700 for test data further includes: a second evaluation module 760, configured to evaluate the differences of each test item in different test programs through the box plot.

[0113] In some embodiments, the second evaluation module 760 is specifically configured to, for each test item, if the median of a test program among multiple test programs is greater than the upper quartile of another test program or less than the lower quartile of another test program, it is evaluated that there is a significant difference in this test item in different test programs.

[0114] In some embodiments, the computing module 720 is specifically configured to calculate the correlation coefficient between each of the test items by using the Pearson correlation coefficient calculation formula.

[0115] In some embodiments, for the first quadrant and the third quadrant of the difference analysis graph, the first evaluation module 740 is specifically configured to, for every two test items among a plurality of test items, if the difference value between the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program is greater than a first preset value, it is evaluated that there is a significant difference between the two test items in the two different test programs.

[0116] In some embodiments, for the second quadrant and the fourth quadrant of the difference analysis graph, the first evaluation module 740 is specifically configured to, for every two test items among a plurality of test items, if the distance from the coordinate point formed by the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program to the origin of the difference analysis graph is greater than a second preset value, it is evaluated that there is a significant difference between the two test items in the two different test programs.

[0117] In some embodiments, the first preset value is greater than the second preset value.

[0118] In some embodiments, the data acquisition module 710 is specifically configured to determine a target product corresponding to a plurality of test programs according to set parameters;

[0119] Obtain the test data of a preset test category of the target product in a test database;

[0120] Wherein, the preset test category includes a plurality of test categories, and each test category corresponds to a plurality of test items.

[0121] In some embodiments, the data acquisition module 710 is further configured to merge the test data of the preset test category according to the set parameters to generate a merged data table.

[0122] In some embodiments, the set parameters include a test program identifier, a product identifier, a wafer identifier, a die identifier, and a process step identifier.

[0123] In some embodiments, the evaluation system 700 for the test data further includes: an upload module 770, configured to generate a difference report according to the differences of each of the test items in two different test programs obtained by evaluation; and upload the difference report regularly.

[0124] In some embodiments, the test data is the test data generated in the wafer electrical test stage.

[0125] It should be noted that in the embodiments of the present application, the first drawing module 730 and the second drawing module 750 can execute the corresponding drawing functions in parallel, and the first evaluation module 740 and the second evaluation module 760 can also execute the corresponding evaluation functions in parallel.

[0126] The embodiments of the present application further provide a wafer testing system, and the wafer testing system includes the evaluation system for the above test data.

[0127] In the embodiments of the present application, each component can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software function module.

[0128] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0129] Therefore, the embodiments of the present application provide a storage medium that stores a computer program, and when the computer program is executed by at least one processor, the steps described in the above embodiments are implemented.

[0130] It should be noted that the computer storage medium shown in this application can be a computer signal medium, a computer storage medium, or any combination of the two. The computer storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, the computer storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. And in this application, the computer signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the program code of the computer. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer signal medium can also be any computer storage medium other than the computer storage medium, and this computer storage medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer storage medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0132] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0133] As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An evaluation method for test data, characterized in that, The method includes: Obtaining test data of multiple test programs; each test program includes multiple test items; For each of the test programs, calculating the correlation coefficient between each of the test items according to the test data; Drawing a difference analysis graph for every two test programs among the multiple test programs according to the correlation coefficient between each of the test items in different test programs; wherein, the horizontal axis and the vertical axis of the difference analysis graph respectively correspond to a test program; Evaluating the differences of each of the test items in two different test programs according to the difference analysis graph; For the first quadrant and the third quadrant of the difference analysis graph, the evaluating the differences of each of the test items in two different test programs includes: For every two test items among the multiple test items, if the difference value between the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program is greater than a first preset value, it is evaluated that there are significant differences between the two test items in the two different test programs.

2. The method according to claim 1, wherein Before calculating the correlation coefficient between each of the test items, the method further includes: Drawing a box plot for each of the test items according to the test data of each of the test items in different test programs; For each of the test items, removing abnormal test data by using the box plot.

3. The method according to claim 2, wherein The calculating the correlation coefficient between each of the test items according to the test data includes: Calculating the correlation coefficient between each of the test items according to the test data after removing the abnormal test data.

4. The method according to claim 2, characterized in that Each test program includes the same test items.

5. The method according to claim 4, wherein The method further includes: Evaluating the differences of each of the test items in different test programs through the box plot.

6. The method according to claim 5, wherein The evaluating the differences of each of the test items in different test programs includes: For each of the test items, if the median of one test program among the multiple test programs is greater than the upper quartile of another test program or less than the lower quartile of another test program, it is evaluated that there are significant differences of this test item in different test programs.

7. The method according to claim 1, characterized in that The calculating the correlation coefficient between each of the test items includes: Calculating the correlation coefficient between each of the test items by using the Pearson correlation coefficient calculation formula.

8. The method according to claim 1, wherein For the second quadrant and the fourth quadrant of the difference analysis graph, the evaluating the differences of each of the test items in two different test programs further includes: For every two test items among the multiple test items, if the distance from the coordinate point formed by the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program to the origin of the difference analysis graph is greater than a second preset value, it is evaluated that there are significant differences between the two test items in the two different test programs.

9. The method according to claim 8, wherein The first preset value is greater than the second preset value.

10. The method according to claim 1, wherein The obtaining the test data of multiple test programs includes: Determining a target product corresponding to multiple test programs according to set parameters; Obtain the test data of the preset test categories of the target product in the test database; Among them, the preset test categories include multiple test categories, and each test category corresponds to multiple test items.

11. The method according to claim 10, wherein Before calculating the correlation coefficients between the respective test items according to the test data, the method further includes: Merge the test data of the preset test categories according to the set parameters to generate a merged data table.

12. The method according to claim 10 or 11, characterized in that The set parameters include a test program identifier, a product identifier, a wafer identifier, a die identifier, and a process step identifier.

13. The method according to claim 1, characterized in that The method further includes: Generate a difference report based on the differences of the respective test items obtained through evaluation in two different test programs; Regularly upload the difference report.

14. The method according to claim 1, characterized in that The test data is the test data generated in the wafer electrical test stage.

15. An evaluation system for test data, characterized in that, The system includes: A data acquisition module for acquiring the test data of multiple test programs; each test program includes multiple test items; A calculation module for, for each of the test programs, calculating the correlation coefficients between the respective test items according to the test data; A first plotting module for plotting a difference analysis graph for every two test programs among the multiple test programs according to the correlation coefficients between the respective test items in different test programs; wherein, the horizontal axis and the vertical axis of the difference analysis graph respectively correspond to a test program; A first evaluation module for evaluating the differences of the respective test items in two different test programs according to the difference analysis graph; For the first quadrant and the third quadrant of the difference analysis graph, the evaluating the differences of the respective test items in two different test programs includes: For every two test items among the multiple test items, if the difference value between the correlation coefficient of the two test items in one test program and the correlation coefficient of the two test items in another test program is greater than a first preset value, it is evaluated that there are significant differences between the two test items in the two different test programs.

16. A wafer testing system, characterized in that, An evaluation system including the test data according to claim 15.

17. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the test data evaluation method according to any one of claims 1 to 14.

Images