Matching evaluation method, system, device and storage medium for wats test bench

Abstract

The present application belongs to the technical field of semiconductor manufacturing test, and discloses a matching evaluation method, system, device and storage medium of a WAT test machine. The method comprises: obtaining target and reference machine test data, generating comparability conditions based on product, site and other information to perform data alignment and preprocessing, and outputting aligned data sets and quality indicators; calculating a multi-dimensional feature vector containing distribution, risk, trend and wafer space characteristics for each test item based on the aligned data; fusing the feature vector and the aligned quality indicators, calculating a difference score, and combining an adaptive threshold dynamically generated based on historical data and parameter importance to perform evaluation; finally outputting a difference parameter list and an evaluation result, and simultaneously introducing an AI model for comprehensive judgment, and continuously optimizing through feedback loop. Through the above-mentioned mode, automation, quantification and intelligentization of machine matching evaluation are realized, and evaluation efficiency, consistency and accuracy are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of semiconductor manufacturing testing technology, and in particular to a matching evaluation method, system, equipment, and storage medium for a WAT ​​testing machine. Background Technology

[0002] In semiconductor manufacturing, wafer acceptance testing (WAT) is a critical step in ensuring chip performance and reliability. After introducing new testing equipment or repairing and calibrating existing equipment, the test results need to be compared with a stable set of benchmark equipment to assess consistency and stability. Currently, this evaluation process relies heavily on manual operation: engineers need to sift through a large amount of test data to find comparable data between the target equipment and the benchmark set, drawing and comparing cumulative distribution plots, box plots, trend charts, or wafer maps for each test item, and making judgments based on visual inspection and experience. This method has significant drawbacks: First, the large amount of data makes manual comparison laborious and inefficient, and the conclusions are influenced by subjective factors, resulting in poor consistency; second, different test parameters (such as leakage current and open-circuit resistance) have different tolerances for data fluctuations, and traditional methods lack an adaptive, quantitative difference tolerance mechanism, making it difficult to intelligently and efficiently screen out the truly critical difference parameters that need attention, leading to long evaluation cycles and insufficient accuracy.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a matching evaluation method, system, device, and storage medium for WAT testing equipment, aiming to solve the technical problems of large workload, low efficiency, poor consistency, and inability to adaptively and intelligently handle the tolerance of different parameter differences in WAT equipment matching evaluation.

[0005] To achieve the above objectives, the present invention provides a matching evaluation method for WAT testing equipment, the matching evaluation method for WAT testing equipment comprising the following steps:

[0006] Acquire test data from the target test bench and the benchmark test bench group, align the test data based on predefined comparability conditions, output the aligned target dataset and benchmark dataset, and generate alignment confidence and coverage indicators. The predefined comparability conditions are generated based on at least one of the following information: product, test site, test program version, and test recipe.

[0007] Based on the aligned target dataset and benchmark dataset, a multidimensional feature vector is determined for each test item. The multidimensional feature vector includes distribution features, risk features, trend features, and wafer space features.

[0008] The difference score for each test item is determined by fusing the multidimensional feature vector, the alignment confidence, and the coverage, and a corresponding dynamic evaluation threshold is generated for each test item based on an adaptive tolerance mechanism.

[0009] The difference score is compared with the corresponding dynamic evaluation threshold, and a list of difference parameters and the corresponding evaluation results are output.

[0010] In one embodiment, the process further includes, before aligning the test data based on predefined comparability conditions:

[0011] The fields from different data sources in the test data are standardized, the naming of test items, the structure of specifications and the pass or fail flags are unified, and a unified composite primary key is constructed. The composite primary key includes at least the product identifier, site identifier, test program identifier, test recipe identifier, machine identifier, batch identifier, wafer identifier, chip coordinates and test item identifier.

[0012] For missing test values ​​in the test data, robust statistics are used to fill in or mark them as missing within a comparable sample set. For records where test values ​​conflict with the original pass or fail markers, they are unified according to a preset priority strategy.

[0013] Convert data from different units for the same test item to a unified unit, and then normalize the converted values ​​based on a robust scale.

[0014] For each test item, the test data is used to calculate the data distribution range on the reference machine group data, and the values ​​of the target machine and the reference machine that exceed the distribution range are marked as extreme outliers. The distribution range is determined based on the interquartile range or the absolute deviation of the median.

[0015] In one embodiment, aligning the test data based on predefined comparability conditions and outputting the aligned target dataset and benchmark dataset includes:

[0016] When the target data and candidate benchmark data are not completely consistent in terms of test item set, test recipe, or time window, exact matching is given priority.

[0017] If exact matching does not meet the conditions, the comprehensive distance between the target data and each candidate benchmark data is calculated. The comprehensive distance is the weighted sum of the test item set distance, test recipe distance, and time window distance. The candidate benchmark data with the smallest comprehensive distance is selected as the benchmark dataset aligned with the target dataset, and the matched and aligned target dataset and benchmark dataset are output.

[0018] In one embodiment, the process of acquiring the wafer space-type features includes acquiring wafer map data of the target test instrument and the benchmark test instrument group, and calculating the difference map;

[0019] Based on the difference map, the ring anomaly score, stripe anomaly score, and hotspot anomaly score are calculated by quantization respectively;

[0020] The ring anomaly score, stripe anomaly score, and hotspot anomaly score are weighted and summed according to preset weight coefficients to generate a spatial anomaly evaluation value, which is then used as the wafer space class feature in the multidimensional feature vector.

[0021] In one embodiment, the generation of a corresponding dynamic evaluation threshold for each test item based on the adaptive tolerance mechanism includes:

[0022] Determine the baseline threshold based on the historical difference score distribution of the test items during the stable period;

[0023] The basic threshold is adjusted based on the historical failure rate index, historical near-specification sensitivity index, and preset parameter importance weights of the test item to obtain the dynamic evaluation threshold. The adjustment formula is as follows:

[0024]

[0025] Among them, T base (item) is the base threshold, Rfail(item) is the historical failure rate index, Rnear(item) is the historical near-size sensitivity index, Imp(item) is the preset parameter importance weight, and λ1, λ2, λ3 are configurable coefficients or obtained by supervised learning in labeled scenarios.

[0026] In one embodiment, the method further includes:

[0027] The multidimensional feature vector is input into the trained AI classification model to obtain the AI ​​evaluation result output by the AI ​​classification model. The AI ​​evaluation result is then combined with the evaluation result obtained by comparing the AI ​​evaluation result with a threshold to obtain the target evaluation result. The training samples of the AI ​​classification model are based on a single evaluation task, and different feature input subsets or feature weight templates are configured for different types of test items according to the differences in the parameter types of the test items.

[0028] In one embodiment, the method further includes:

[0029] Collect user feedback on the output list of difference parameters and evaluation report, and associate and store the feedback information with the corresponding evaluation task metadata;

[0030] Based on the accumulation of feedback information and system monitoring indicators, at least one update operation is triggered, including parameter calibration of the adaptive tolerance mechanism, incremental learning of the AI ​​model, or full retraining, to achieve continuous optimization of the system.

[0031] Furthermore, to achieve the above objectives, this invention also proposes a matching evaluation system for WAT testing equipment. This matching evaluation system is applied to the matching evaluation method for WAT testing equipment described above. The system includes:

[0032] The acquisition module is used to acquire test data from the target test machine and the benchmark test machine group, align the test data based on predefined comparability conditions, output the aligned target dataset and benchmark dataset, and generate alignment confidence and coverage indicators. The predefined comparability conditions are generated based on at least one of the following: product, test site, test program version, and test recipe.

[0033] The extraction module is used to determine a multi-dimensional feature vector for each test item based on the aligned target dataset and the benchmark dataset. The multi-dimensional feature vector includes distribution features, risk features, trend features, and wafer space features.

[0034] The fusion module is used to fuse the multidimensional feature vector, the alignment confidence and coverage to determine the difference score of each test item, and generate a corresponding dynamic evaluation threshold for each test item based on the adaptive tolerance mechanism;

[0035] The evaluation module is used to compare the difference score with the corresponding dynamic evaluation threshold, output a list of difference parameters, and generate an evaluation report that includes machine-level health score and release recommendations.

[0036] Furthermore, to achieve the above objectives, the present invention also proposes a matching evaluation device for a WAT ​​test machine. The matching evaluation device for the WAT test machine includes: a memory, a processor, and a matching evaluation program for the WAT test machine stored in the memory and executable on the processor. The matching evaluation program for the WAT test machine is configured to implement the steps of the matching evaluation method for the WAT test machine as described above.

[0037] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a matching evaluation program for a WAT ​​test equipment, wherein when the matching evaluation program for the WAT test equipment is executed by a processor, the steps of the matching evaluation method for the WAT test equipment as described above are implemented.

[0038] This invention acquires test data from target and benchmark equipment, generates comparability conditions based on product and site information, performs data alignment and preprocessing, and outputs an aligned dataset and quality indicators. Based on the aligned data, a multi-dimensional feature vector containing distribution, risk, trend, and wafer space characteristics is calculated for each test item. The feature vector and alignment quality indicators are fused to calculate a difference score, which is then evaluated using an adaptive threshold dynamically generated based on historical data and parameter importance. Finally, a list of difference parameters and evaluation results are output, and an AI model is introduced for comprehensive judgment, with continuous optimization possible through a feedback loop. This approach automates, quantifies, and intelligentizes equipment matching evaluation, significantly improving evaluation efficiency, consistency, and accuracy. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the first embodiment of the matching evaluation method for WAT testing equipment of the present invention.

[0040] Figure 2 This is a structural block diagram of the first embodiment of the matching evaluation system for the WAT testing machine of the present invention.

[0041] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0042] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0043] This invention provides a matching evaluation method for WAT testing equipment, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the matching evaluation method for a WAT ​​testing machine according to the present invention.

[0044] In this embodiment, the matching evaluation method for the WAT testing machine includes the following steps:

[0045] Step S10: Obtain test data from the target test machine and the benchmark test machine group, align the test data based on predefined comparability conditions, output the aligned target dataset and benchmark dataset, and generate alignment confidence and coverage metrics.

[0046] In this embodiment, the execution entity is the matching and evaluation device of the WAT test machine. The matching and evaluation device of the WAT test machine has functions such as data processing, data communication and program execution. The matching and evaluation device of the WAT test machine can be a computer terminal device or other network device, or other devices with similar functions. This embodiment does not limit this.

[0047] It is important to note that in semiconductor manufacturing, wafer acceptance testing (WAT) is a critical step in ensuring chip performance and reliability. After introducing new testing equipment or repairing and calibrating existing equipment, the test results need to be compared with a stable set of benchmark equipment to assess consistency and stability. Currently, this evaluation process mainly relies on manual operation: engineers need to sift through a large amount of test data to find comparable data between the target equipment and the benchmark set, drawing and comparing cumulative distribution plots, box plots, trend charts, or wafer maps for each test item, and making judgments based on visual inspection and experience. This method has significant drawbacks: First, the large amount of data makes manual comparison laborious and inefficient, and the conclusions are influenced by subjective factors, resulting in poor consistency; second, different test parameters (such as leakage current and open-circuit resistance) have different tolerances for data fluctuations, and traditional methods lack an adaptive, quantitative difference tolerance mechanism, making it difficult to intelligently and efficiently screen out the truly critical difference parameters that need attention, leading to long evaluation cycles and insufficient accuracy.

[0048] To address the aforementioned technical challenges, this embodiment acquires test data from the target and benchmark equipment. Based on product and site information, it generates comparability conditions for data alignment and preprocessing, outputting an aligned dataset and quality indicators. For each test item, a multi-dimensional feature vector containing distribution, risk, trend, and wafer space characteristics is calculated based on the aligned data. The feature vectors and alignment quality indicators are fused to calculate a difference score, which is then evaluated using an adaptive threshold dynamically generated based on historical data and parameter importance. Finally, a list of difference parameters and evaluation results are output, and an AI model is introduced for comprehensive judgment, with continuous optimization possible through a feedback loop. This approach automates, quantifies, and intelligentizes equipment matching evaluation, significantly improving evaluation efficiency, consistency, and accuracy. Specifically, it can be implemented as follows.

[0049] It should be noted that this embodiment first requires acquiring test data from the target test machine and the benchmark test machine group. Specifically, data source access and unified standards are required: accessing multi-source data needed for evaluation, including at least: raw test data (item / value, pass / fail, spec, time), machine information (tool / chamber / recipe), maintenance / calibration event records, probe card / fixture / handler information, environmental data (if any), and map data (die coordinates + values ​​or bins). Measurement result data is used for subsequent distribution, risk, and trend analysis; contextual data is used to define comparability conditions, identify incomparable samples, and support result interpretation and traceability; map data is used to identify spatial structural differences. During data access, cross-system fields are uniformly identified, timestamps are standardized, and naming conventions are unified to ensure the stability of cross-system associations. The predefined comparability conditions are generated based on at least one of the following information: product, test site, test program version, and test recipe. Comparability conditions are determined and family tree constraints are generated: based on information such as product / site / program version / temperature / recipe, family tree conditions (a set of comparability conditions) are generated, classifying samples into the same comparable domain to avoid spurious differences caused by cross-condition comparisons. Then, comparable data is extracted according to the formula "target machine + benchmark machine group + same family tree conditions": comparable data is pulled according to a preset extraction strategy: data is pulled according to "target machine + benchmark machine group + same family tree conditions (product / site / program version / temperature / recipe, etc.)". Extraction follows the principle of "comparison with the same caliber" to ensure that the target machine samples and the benchmark machine group samples enter subsequent alignment and comparison under the same family tree conditions.

[0050] Furthermore, prior to alignment, data preprocessing is required. This includes standardizing fields from different data sources in the test data, unifying test item naming, specification limits, and pass / fail markers, and constructing a unified composite primary key. This composite primary key must include at least the product identifier, site identifier, test program identifier, test recipe identifier, machine identifier, batch identifier, wafer identifier, chip coordinates, and test item identifier. For missing test values ​​in the test data, robust statistics are used to fill in or mark them as missing within a comparable sample set. Records where test values ​​conflict with the original pass / fail markers are unified according to a preset priority strategy. Different units of data for the same test item are converted to a unified unit, and the converted values ​​are normalized based on a robust scale. For each test item, the data distribution range is calculated on the benchmark machine group data, and values ​​in the target machine and benchmark machine that exceed the distribution range are marked as extreme outliers. The distribution range is determined based on the interquartile range or the absolute deviation of the median.

[0051] The specific implementation of the above data processing is as follows: S1: Field normalization and unified primary key construction: The fields in the EvaluationDataset are normalized: unified item naming, unified spec field structure, unified pass / fail flags, and a unified primary key is constructed across systems, including at least: product_id, site_id, program_id, recipe_id, tool_id, lot_id, wafer_id, die(x,y), item_id, and time. Simultaneously, the timestamp standard is unified (e.g., UTC or the factory's local time zone), and time zone information is recorded to ensure consistency in cross-window calculations.

[0052] Missing and outlier handling: The following missing / outlier scenarios are handled in a consistent manner and written to the processing log (process_log): missing value: if the sample size in the same alignment domain (genealogy_key + item + window_id) is sufficient, it is filled with a robust statistic (such as the median); otherwise, it is marked as missing and the sample is removed from the calculation of the item.

[0053] Spec missing or inconsistent: Prioritize backfilling from the spec table corresponding to the program / recipe; if still missing, mark it as spec_missing. This item only participates in the calculation of "features without spec" (distribution / trend / map) and does not participate in near-spec and margin features; pass / fail conflict: When the value and the fail derived from the spec are inconsistent with the original fail, record the conflict mark and select one of the configuration strategies of "spec derivation priority / original mark priority", while retaining the original value for auditing.

[0054] Unit consistency and dimension normalization: When the same item has different units from different sources, the unit is converted and unified according to the unit mapping table; after conversion, the item value is robustly scaled (e.g., divided by IQR or MAD) to support cross-parameter comparability, and the scale and parameters used are written into the audit field.

[0055] Deduplication and consistency verification: Deduplication is performed by (unified primary key + item_id + time); when multiple records appear for the same primary key, one record is retained according to the rule of "latest time / highest data quality priority"; the number of deduplications and the reason statistics are also output.

[0056] Extreme outlier labeling: For each alignment region (genealogy_key + item_id), calculate the quartiles Q1 and Q3 on the baseline sample group to obtain IQR=Q3. Q1. For any sample value x: if x <Q1 k IQR or x>Q3+k For IQR, an extreme outlier is marked as outlier=1; where k is a configurable threshold (e.g., k=3 represents "extreme outlier"). For clearly skewed / heavy-tailed items, the MAD principle can be switched:

[0057] calculate and define

[0058]

[0059] If z > z0 (e.g., z0 = 6), then mark outlier = 1.

[0060] Quality filtering and filtering statistics output: Remove unrepresentative sample sources according to configurable rules, including engineering batches, known abnormal batches, unstable warm-up sections, etc.; and output filtering statistics: sample size before and after filtering, number of filtered samples classified by cause, percentage of filtered windows, and impact on key item coverage, for interpretation and review.

[0061] Furthermore, the specific process of data alignment includes prioritizing exact matching when the target data and candidate benchmark data are not completely consistent in terms of test item set, test recipe, or time window; if exact matching does not meet the conditions, the comprehensive distance between the target data and each candidate benchmark data is calculated, where the comprehensive distance is the weighted sum of the test item set distance, test recipe distance, and time window distance, and the candidate benchmark data with the smallest comprehensive distance is selected as the benchmark dataset aligned with the target dataset, and the matched and aligned target dataset and benchmark dataset are output.

[0062] The above process is specifically implemented, for example, by generating a genealogy_key based on genealogy fields (product / site / program version / temperature / recipe, etc.), and within each genealogy_key, forming: a target machine sample set A; and a baseline machine group candidate set {B1, B2, ..., Bm} (which can be divided by batch / window). When there are minor differences in the recipe or missing fields, it is allowed to enter the candidate set, but the best comparable candidates need to be selected in the next step through similarity matching. In this embodiment, priority matching + similarity calculation is adopted to automatically match the best comparable batch / window. The priority matching strategy is to first perform precise alignment. Precise alignment is performed directly when the following conditions are met: the recipe_id is completely consistent or the recipe_hash is consistent; the consistency of the item set reaches a threshold. (e.g., 0.95); Time window overlap (such as 0.7), if satisfied, output the exact alignment result and assign a high confidence level of conf = 1. Then perform similarity alignment. When the exact alignment is not satisfied, calculate the similarity distance in the candidate set {Bi} and select the optimal candidate B : item set distance (missing item handling): , recipe distance (minor difference handling): Form a vector or hash set from the key configuration items of the recipe and calculate the normalized difference: , time distance / overlap penalty (non - overlapping window handling): , where is the window overlap degree (overlapping duration / total duration) or the normalized center point distance. Comprehensive distance: , select the B with the smallest D as the best comparable batch / window; and output the alignment confidence level: , and at the same time write the "reason for not using exact alignment (missing item / recipe minor difference / window non - overlapping)" into the audit field. Finally, perform fallback alignment. If the optimal candidate still does not meet the minimum comparable requirement (for example, conf < conf_min or item coverage rate is insufficient), then execute the fallback strategy: expand the time window (increase buffer); or relax the recipe conditions but increase the quality filtering intensity; or only perform alignment evaluation on the subset of items with sufficient coverage, and output the "incomparable / sample insufficient" flag for the remaining items. The fallback trigger condition and fallback method are also written into the audit field.

[0063] Further, in this embodiment, the process of generating the alignment confidence level and coverage rate metrics, for example, the output result at least includes: the aligned target sample set and the reference sample set (by lot / wafer / die granularity); item coverage rate: , sample alignment rate (number of successfully aligned samples / number of target samples); alignment confidence level conf.

[0064] Step S20: Determine the multi - dimensional feature vectors of each test item based on the aligned target data set and the reference data set.

[0065] In this embodiment, the multi - dimensional feature vectors include distribution - type features, risk - type features, trend - type features, and wafer space - type features.

[0066] Specifically, the distribution - type features are as follows: Position difference:

[0067] Dispersion difference:

[0068] Tail difference:

[0069] Morphological distance: KS statistic fks, Wasserstein distance fwd, or PSI (select one or more according to configuration).

[0070] Furthermore, the risk-related characteristics are as follows: Let the upper and lower limits of the specification be LSL / USL:

[0071] Failure rate changes: ;

[0072] near-spec ratio change: Δnear (e.g., the ratio difference falling into [LSL,LSL+δ] or [USL-δ,USL]).

[0073] Margin variation: Δmargin (e.g., the change in distance from the median to the specification boundary).

[0074] Furthermore, trend-related features are as follows:

[0075] Perform sliding window statistics and change point / drift calculations on time series data:

[0076] Event jump strength: f_{jump}=median(X_{after})-median(X_{before}) (robust scalar normalization can be performed further);

[0077] Slow drift intensity: sliding window slope f_{slope}, variable point intensity f_{cp}, etc.

[0078] Furthermore, Map-type features include:

[0079] Global similarity: f_{map_sim} is obtained from correlation coefficient / SSIM, etc.; spatial pattern anomaly score f_{map_pat}.

[0080] To ensure comparability across test items and adaptability to skewed / long-tailed distributions, the above features are robustly standardized within the family tree:

[0081] Where f_{j,hist} represents the distribution of historical characteristics during the stable period under the same genealogy_key. To prevent division by zero of extremely small constants.

[0082] In one embodiment, the ring anomaly score, stripe anomaly score, and hotspot anomaly score are quantized and calculated based on the difference map. The ring anomaly score, stripe anomaly score, and hotspot anomaly score are then weighted and summed according to preset weighting coefficients to generate a spatial anomaly evaluation value. This spatial anomaly evaluation value is used as the wafer space class feature in the multidimensional feature vector. Specifically, the quantization details of the map spatial pattern anomaly score are as follows: the wafer map is represented as a two-dimensional grid M(x,y) (missing dies are identified by a mask), and a difference map ΔM=Mt is constructed. Mr (or the difference from the baseline mean / median Map). The ring score is calculated by calculating the radius r with the wafer center as the origin, and then radially aggregating ΔM to obtain the radial profile p(r) (e.g., the mean / median within the same radius ring). Ring anomalies are manifested as significant deviations at certain radii. If necessary, correlation enhancement can be performed with the circular template.

[0083] The calculation process for the stripe fraction involves performing a two-dimensional Fourier transform F(ΔM) on ΔM. The stripe corresponds to the frequency domain energy concentration in a certain direction. The directional bandwidth Omega_{dir} is defined as follows:

[0084]

[0085] The above can take the maximum value in multiple directions to adapt to different stripe directions.

[0086] The hotspot score is calculated by performing a local window scan (fixed window or multi-scale window) on ΔM, and taking the maximum value of the local mean / energy.

[0087]

[0088] Alternatively, spatial autocorrelation (such as Moran's I) can be introduced to emphasize clustering anomalies.

[0089] The final spatial pattern anomaly score is a weighted fusion:

[0090]

[0091] Among them, a, b, and c can be fixed, configured according to parameter type, or automatically learned by supervised learning when there are labels.

[0092] Step S30: The multidimensional feature vector, the alignment confidence and coverage are fused to determine the difference score for each test item, and a corresponding dynamic evaluation threshold is generated for each test item based on the adaptive tolerance mechanism.

[0093] In the specific implementation, the difference score is calculated as follows:

[0094]

[0095] Among them, zj covers distribution / risk / trend / Map-type features. Nonlinear or truncation function To avoid extreme values ​​dominating; Conf is the alignment confidence (derived from alignment similarity / coverage, with automatic weight reduction for insufficient coverage); g(n) is the sample size factor (weight reduction for insufficient samples, e.g., ... .

[0096] The above weights At least three weighting methods are supported: 1) Fixed weights: During the unlabeled cold start phase, default weights for distribution / risk / trend / Map are set based on engineering experience; 2) Weights configured by parameter type: Test items are divided into categories such as center-sensitive, tail-sensitive, failure-sensitive, and spatially sensitive, and different weight templates are used for different categories (e.g., increase the weights of tail and near-spec in leakage / openR); 3) Weights learned through model: When historical treatment labels exist, the fusion model (such as Logistic / GBDT) is trained with the treatment results as the supervision signal to learn the optimal weights or directly learn the mapping from features to DriftScore / risk level; the model version is frozen and included in the audit when going online.

[0097] Furthermore, in this embodiment, an adaptive tolerance mechanism is used to generate a corresponding dynamic evaluation threshold for each test item, including:

[0098] Determine the baseline threshold based on the historical difference score distribution of the test items during the stable period;

[0099] The basic threshold is adjusted based on the historical failure rate index, historical near-specification sensitivity index, and preset parameter importance weights of the test item to obtain the dynamic evaluation threshold. The adjustment formula is as follows:

[0100]

[0101] Among them, T base (item) is the base threshold, Rfail(item) is the historical failure rate index, Rnear(item) is the historical near-size sensitivity index, Imp(item) is the preset parameter importance weight, and λ1, λ2, λ3 are configurable coefficients or obtained by supervised learning in labeled scenarios.

[0102] Step S40: Compare the difference score with the corresponding dynamic evaluation threshold, and output a list of difference parameters and the corresponding evaluation results.

[0103] In practical implementation, after comparison, distribution differences, tail risks, trend drift, and Map anomalies can be integrated to output a parameter-level anomaly score (DriftScore / risk level), while retaining the contributions of each component for interpretation and verification. The parameter-level results are stably sorted and filtered, outputting the Top-N controllable quantity difference parameters (N is configurable or threshold-triggered), and grouped by severity / risk type (e.g., tail risk, failure risk, trend drift, spatial pattern). Further, based on the parameter-level results, a machine-level Machine Health Score is generated, providing assessment recommendations for release, control, or non-release.

[0104] Furthermore, in this embodiment, the AI ​​evaluation results can be integrated with the above evaluation results. Specifically, the multidimensional feature vector is input into the trained AI classification model to obtain the AI ​​evaluation results output by the AI ​​classification model. The AI ​​evaluation results are then integrated with the evaluation results obtained by comparing the AI ​​evaluation results with a threshold to obtain the target evaluation result. The training samples of the AI ​​classification model are based on a single evaluation task, and different feature input subsets or feature weight templates are configured for different types of test items according to the differences in the parameter types of the test items.

[0105] In the aforementioned AI model, the training sample construction and labeling are defined using a "single evaluation task" (i.e., an evaluation of a target machine under a certain genealogy_key and window_id) as a training sample. The input features and labels are constructed as follows: input features include: the driftScore of key test items and its sub-contributions (distribution / risk / trend / map), Top-K statistical features (such as maximum / mean / quantile), key risk features (Δfail, Δnear, Δmargin), and quality features such as alignment confidence (conf) and coverage. Labels represent historical handling conclusions (release / controlled / not released / requires machine adjustment). This sample granularity allows the model to learn the engineering decision-making patterns of "multiple parameters changing together → final handling," rather than just learning the local differences of a single item.

[0106] Feature subsets and input structures for different parameter types (addressing WAT parameter heterogeneity): Given the varying statistical patterns and risk sensitivities of WAT test items, such as "center-sensitive / tail-sensitive / failure-sensitive / spatial-sensitive," the model employs different feature subsets or weight templates for different types during feature engineering. These include at least: Center-sensitive parameters: emphasizing distribution patterns like fshift, fscale, KS / Wass; Tail-sensitive parameters (e.g., leakage / openR): emphasizing features like f_{tail99} / f_{tail01}, near-spec variation, and tail distance; Failure-sensitive parameters: emphasizing risk features like Δfail, Δnear, and Δmargin, using them as strong constraint inputs; Spatial-sensitive parameters: emphasizing MapScore (pattern score, similarity score) and its variations. Implementation methods can include: generating feature sub-vectors for different types and concatenating them as input; or calculating and summarizing statistics for different types to create "grouped feature blocks," enhancing the model's ability to distinguish between different parameter families.

[0107] It should be noted that this AI model is trained using supervised learning (e.g., genealogy stratification + time segmentation + imbalance handling).

[0108] When historical labels are available, train a supervised learning classification model (e.g., LightGBM / XGBoost or equivalent models) and employ the following targeted training strategies: Genealogy-based hierarchical training / validation: Hierarchically split the training and validation sets by genealogy_key, and simultaneously split them by time to avoid data leakage into the validation set from the same event window or batch; Imbalanced sample handling (more normal than abnormal): Use at least one or a combination of the following: Class weight / cost sensitivity: Set higher weights for minority classes such as "not allowed / requires machine adjustment" to more strongly constrain classes with high underreporting costs during training; Hard example resampling: Increase the sampling probability for boundary samples and confirmed abnormal samples; Hierarchical sampling: Ensure minimum coverage of different genealogy_keys and key products during training. Missing items and quality reduction: Use coverage, confidence, and sample size n as input features, or reduce the weight of low-confidence samples in the training loss to avoid overfitting noise caused by "missing items / low coverage". 4) Explainability and engineering consistency constraints, which are optional conditions: Output feature contributions (such as SHAP / importance) form auditable explanations; apply monotonicity constraints to risk features (Δfail, Δnear) (the higher the risk, the lower the probability of not releasing), to enhance explainability and engineering consistency.

[0109] When there are insufficient labels, unsupervised / semi-supervised anomaly detection is adopted. Unsupervised / semi-supervised models such as IsolationForest / One-Class / GMM are used to construct "normal behavior boundaries", output anomaly scores and use them as part of DriftScore fusion or as an auxiliary item for Top-N ranking. The threshold and ranking weights are calibrated by engineers through a small amount of feedback on Top-N, so as to achieve iterative optimization.

[0110] Map input representation (two-dimensional grid and missing structure for chip-compatible Map): Representing the wafer Map as a two-dimensional grid tensor. and introduce at least one mask channel. Identify areas without dies / missing points; add coordinate channels (normalized x and y coordinates) if necessary to preserve spatial priors. Unlike regular images, chip maps typically contain non-rectangular valid regions and missing points, therefore mask channels and missing point masking are essential structures.

[0111] Furthermore, the autoencoder network structure adopts a lightweight CNN and multi-scale spatial pattern extraction. The lightweight convolutional encoder-decoder structure includes the encoder extracting multi-scale features to capture patterns such as stripes, rings, and hot spots; the decoder reconstructs the input map and outputs a reconstruction error map; the statistics of the reconstruction error (mean / quantile / hot spot energy) are used as the spatial anomaly score MapAE_Score. To adapt to the irregularity of the effective area of ​​the map, the output error of the decoder is only counted within the effective area of ​​the mask.

[0112] During training, a masked reconstruction loss is used, and a spatial continuity regularization term is introduced:

[0113] Mask reconstruction loss:

[0114]

[0115] Spatial continuity (smoothness / structure) constraints:

[0116]

[0117] Total loss: L = L_{rec} + \mu L_{tv}L

[0118] This design makes the model more sensitive to structural anomalies such as "continuous stripes / rings" while avoiding interference from missing regions during training; this is a key difference between chip mapping and conventional natural image autoencoders.

[0119] In one embodiment, automatic evidence output and closed-loop continuous learning can also be implemented. For example, evidence maps (CDF / boxline / Trend / Map, etc.) can be automatically generated for Top-N difference parameters, and key elements such as quantile lines, threshold bands, and event points can be marked in a unified template to ensure consistency between "score and evidence". Structured explanation information is output for each anomaly, including the difference type (translation / widening / long tail / multimodal / drift / spatial pattern), key indicators, and threshold comparison results to avoid inconsistencies between explanation and score.

[0120] In one embodiment, user feedback on the output list of difference parameters and evaluation report can also be collected, and the feedback information can be associated with and stored with the corresponding evaluation task metadata. Based on the accumulation of feedback information and system monitoring indicators, at least one update operation can be triggered, such as parameter calibration of the adaptive tolerance mechanism, incremental learning of the AI ​​model, or full retraining, to achieve continuous optimization of the system. Specific implementations include case library accumulation and feedback collection (structured data assets): establishing a case library and retrieval mechanism, solidifying "Top-N difference parameters + evidence graph / key indicators + threshold comparison + final disposal conclusion" for each evaluation task, forming a searchable historical similar pattern library; at the same time, providing a manual feedback entry point, engineers can perform lightweight annotation on each of the Top-N items, including at least: confirming anomalies / acceptable / cause labels / suggested disposal, and binding and storing the annotations with the genealogy_key, window_id, alignment confidence conf, threshold version, model version, and data snapshot identifier of this evaluation to ensure traceability and replayability. Feedback-driven update strategy (online calibration / incremental learning / triggered retraining): When using feedback for updates, the system selects the update path according to the priority from "light" to "heavy": Online calibration (priority, lightweight): When the feedback focuses on "too many false alarms / thresholds too tight or too loose," only the dynamic threshold parameters of the relevant genealogy_key and item are re-estimated (e.g., adjusting quantile p, risk sensitivity coefficient λ, tightening coefficient η), generating a new threshold version without updating the model. Incremental / warm-start learning (medium weight): When the cumulative number of newly labeled samples reaches the threshold N, or covers the key product / key parameter set, incremental retraining (or periodic small-batch training) is performed using the previous version of the model parameters in a warm-start manner to absorb the latest feedback and maintain the continuity of the decision boundary. Triggered retraining (heavy weight): When the monitoring indicators exceed the limits (e.g., increased risk of missed reports, abnormal alarm volume, significant drift in score distribution / genealogy coverage) or feedback shows systematic model mismatch, full retraining and regression validation are triggered to form a new model version. Feedback weighting / sampling and release access control (ensuring controllable and rollback-proof updates): Feedback samples are weighted and sampled to prevent a few events from "hijacking" the model or threshold. Time decay weighting: Recent feedback has higher weights, for example... Importance weighting: Key parameters / key products / high-risk handling (no release, machine adjustment required) have higher weights; Deduplication and balanced sampling: After clustering samples with recurring patterns, downsampling is performed to ensure balanced coverage of different genealogy_keys and different risk types; and oversampling or cost-sensitive weighting is applied to minority classes to suppress false negatives. Updated thresholds / models must be validated on a frozen regression set before the version can be frozen and deployed online using a canary release approach; After deployment, false positives / false negatives, alarm volume, missing data, genealogy coverage, and model drift are continuously monitored. If any anomalies occur, the system is rolled back to the previous stable version, thus forming a continuous improvement loop.

[0121] In this embodiment, target and benchmark test data are acquired. Comparability conditions are generated based on product and site information for data alignment and preprocessing, outputting an aligned dataset and quality indicators. A multi-dimensional feature vector containing distribution, risk, trend, and wafer space characteristics is calculated for each test item based on the aligned data. The feature vector and alignment quality indicators are fused to calculate a difference score, which is then evaluated using an adaptive threshold dynamically generated based on historical data and parameter importance. Finally, a list of difference parameters and evaluation results are output, and an AI model is introduced for comprehensive judgment, with continuous optimization possible through a feedback loop. This approach automates, quantifies, and intelligentizes machine matching evaluation, significantly improving evaluation efficiency, consistency, and accuracy.

[0122] Furthermore, this embodiment of the invention also proposes a storage medium storing a matching evaluation program for a WAT ​​test equipment. When the matching evaluation program for the WAT test equipment is executed by a processor, it implements the steps of the matching evaluation method for the WAT test equipment as described above.

[0123] Reference Figure 2 , Figure 2 This is a structural block diagram of the first embodiment of the matching evaluation system for the WAT testing machine of the present invention.

[0124] like Figure 2 As shown, the matching evaluation system for WAT testing equipment proposed in this embodiment of the invention includes:

[0125] The acquisition module 10 is used to acquire test data of the target test machine and the benchmark test machine group, align the test data based on predefined comparability conditions, output the aligned target dataset and benchmark dataset, and generate alignment confidence and coverage indicators. The predefined comparability conditions are generated based on at least one of the following: product, test site, test program version, and test recipe.

[0126] Extraction module 20 is used to determine a multi-dimensional feature vector for each test item based on the aligned target dataset and the benchmark dataset. The multi-dimensional feature vector includes distribution features, risk features, trend features and wafer space features.

[0127] The fusion module 30 is used to fuse the multidimensional feature vector, the alignment confidence and coverage to determine the difference score of each test item, and generate a corresponding dynamic evaluation threshold for each test item based on the adaptive tolerance mechanism;

[0128] The evaluation module 40 is used to compare the difference score with the corresponding dynamic evaluation threshold, output a list of difference parameters, and generate an evaluation report that includes machine-level health score and release recommendations.

[0129] In this embodiment, target and benchmark test data are acquired. Comparability conditions are generated based on product and site information for data alignment and preprocessing, outputting an aligned dataset and quality indicators. A multi-dimensional feature vector containing distribution, risk, trend, and wafer space characteristics is calculated for each test item based on the aligned data. The feature vector and alignment quality indicators are fused to calculate a difference score, which is then evaluated using an adaptive threshold dynamically generated based on historical data and parameter importance. Finally, a list of difference parameters and evaluation results are output, and an AI model is introduced for comprehensive judgment, with continuous optimization possible through a feedback loop. This approach automates, quantifies, and intelligentizes machine matching evaluation, significantly improving evaluation efficiency, consistency, and accuracy.

[0130] This application embodiment also provides a matching evaluation device for a WAT ​​test machine, including a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other through the communication bus. The memory is used to store the matching evaluation program for the WAT test machine. When the processor executes the program stored in the memory, it implements the matching evaluation method for the WAT test machine described above.

[0131] The communication bus mentioned in the matching evaluation equipment of the aforementioned WAT test machine can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0132] The communication interface is used for communication between the matching evaluation equipment of the aforementioned WAT test machine and other equipment.

[0133] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0134] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0135] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0136] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0138] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0139] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0140] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0141] In addition, for technical details not described in detail in this embodiment, please refer to the matching evaluation method of WAT test equipment provided in any embodiment of the present invention, which will not be repeated here.

[0142] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0143] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0145] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

[0146] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

Claims

1. A matching evaluation method for a WAT ​​testing machine, characterized in that, The matching evaluation method for the WAT testing equipment includes: Acquire test data from the target test bench and the benchmark test bench group, align the test data based on predefined comparability conditions, output the aligned target dataset and benchmark dataset, and generate alignment confidence and coverage indicators. The predefined comparability conditions are generated based on at least one of the following information: product, test site, test program version, and test recipe. Based on the aligned target dataset and benchmark dataset, a multidimensional feature vector is determined for each test item. The multidimensional feature vector includes distribution features, risk features, trend features, and wafer space features. The difference score for each test item is determined by fusing the multidimensional feature vector, the alignment confidence, and the coverage, and a corresponding dynamic evaluation threshold is generated for each test item based on an adaptive tolerance mechanism. The difference score is compared with the corresponding dynamic evaluation threshold, and a list of difference parameters and the corresponding evaluation results are output.

2. The matching evaluation method for WAT testing equipment as described in claim 1, characterized in that, Before aligning the test data based on predefined comparability conditions, the following steps are also included: The fields from different data sources in the test data are standardized, the naming of test items, the structure of specifications and the pass or fail flags are unified, and a unified composite primary key is constructed. The composite primary key includes at least the product identifier, site identifier, test program identifier, test recipe identifier, machine identifier, batch identifier, wafer identifier, chip coordinates and test item identifier. For missing test values ​​in the test data, robust statistics are used to fill in or mark them as missing within a comparable sample set. For records where test values ​​conflict with the original pass or fail markers, they are unified according to a preset priority strategy. Convert data from different units for the same test item to a unified unit, and then normalize the converted values ​​based on a robust scale. For each test item, the test data is used to calculate the data distribution range on the reference machine group data, and the values ​​of the target machine and the reference machine that exceed the distribution range are marked as extreme outliers. The distribution range is determined based on the interquartile range or the absolute deviation of the median.

3. The matching evaluation method for WAT testing equipment as described in claim 1, characterized in that, The alignment process for the test data based on predefined comparability conditions, outputting the aligned target dataset and benchmark dataset, includes: When the target data and candidate benchmark data are not completely consistent in terms of test item set, test recipe, or time window, exact matching is given priority. If exact matching does not meet the conditions, the comprehensive distance between the target data and each candidate benchmark data is calculated. The comprehensive distance is the weighted sum of the test item set distance, test recipe distance, and time window distance. The candidate benchmark data with the smallest comprehensive distance is selected as the benchmark dataset aligned with the target dataset, and the matched and aligned target dataset and benchmark dataset are output.

4. The matching evaluation method for WAT testing equipment as described in claim 1, characterized in that, The process of acquiring the wafer space-type features includes acquiring wafer map data of the target test instrument and the benchmark test instrument group, and calculating the difference map; Based on the difference map, the ring anomaly score, stripe anomaly score, and hotspot anomaly score are calculated by quantization respectively; The ring anomaly score, stripe anomaly score, and hotspot anomaly score are weighted and summed according to preset weight coefficients to generate a spatial anomaly evaluation value, which is then used as the wafer space class feature in the multidimensional feature vector.

5. The matching evaluation method for WAT testing equipment as described in claim 1, characterized in that, The adaptive tolerance mechanism generates a corresponding dynamic evaluation threshold for each test item, including: Determine the baseline threshold based on the historical difference score distribution of the test items during the stable period; The basic threshold is adjusted based on the historical failure rate index, historical near-specification sensitivity index, and preset parameter importance weights of the test item to obtain the dynamic evaluation threshold. The adjustment formula is as follows: Among them, T base (item) is the base threshold, Rfail(item) is the historical failure rate index, Rnear(item) is the historical near-size sensitivity index, Imp(item) is the preset parameter importance weight, and λ1, λ2, λ3 are configurable coefficients or obtained by supervised learning in labeled scenarios.

6. The matching evaluation method for WAT testing equipment as described in claim 1, characterized in that, The method further includes: The multidimensional feature vector is input into the trained AI classification model to obtain the AI ​​evaluation result output by the AI ​​classification model. The AI ​​evaluation result is then combined with the evaluation result obtained by comparing the AI ​​evaluation result with a threshold to obtain the target evaluation result. The training samples of the AI ​​classification model are based on a single evaluation task, and different feature input subsets or feature weight templates are configured for different types of test items according to the differences in the parameter types of the test items.

7. The matching evaluation method for the WAT testing machine as described in any one of claims 1 to 6, characterized in that, The method further includes: Collect user feedback on the output list of difference parameters and evaluation report, and associate and store the feedback information with the corresponding evaluation task metadata; Based on the accumulation of feedback information and system monitoring indicators, at least one update operation is triggered, including parameter calibration of the adaptive tolerance mechanism, incremental learning of the AI ​​model, or full retraining, to achieve continuous optimization of the system.

8. A matching evaluation system for a WAT ​​testing machine, characterized in that, The matching evaluation system for the WAT testing machine is applied to the matching evaluation method for the WAT testing machine as described in any one of claims 1 to 7, and the system comprises: The acquisition module is used to acquire test data from the target test machine and the benchmark test machine group, align the test data based on predefined comparability conditions, output the aligned target dataset and benchmark dataset, and generate alignment confidence and coverage indicators. The predefined comparability conditions are generated based on at least one of the following: product, test site, test program version, and test recipe. The extraction module is used to determine a multi-dimensional feature vector for each test item based on the aligned target dataset and the benchmark dataset. The multi-dimensional feature vector includes distribution features, risk features, trend features, and wafer space features. The fusion module is used to fuse the multidimensional feature vector, the alignment confidence and coverage to determine the difference score of each test item, and generate a corresponding dynamic evaluation threshold for each test item based on the adaptive tolerance mechanism; The evaluation module is used to compare the difference score with the corresponding dynamic evaluation threshold, output a list of difference parameters, and generate an evaluation report that includes machine-level health score and release recommendations.

9. A matching evaluation device for a WAT ​​testing machine, characterized in that, The matching evaluation device for the WAT test bench includes: a memory, a processor, and a matching evaluation program for the WAT test bench stored in the memory and executable on the processor. The matching evaluation program for the WAT test bench is configured to implement the steps of the matching evaluation method for the WAT test bench as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a matching evaluation program for the WAT test equipment, which, when executed by a processor, implements the steps of the matching evaluation method for the WAT test equipment as described in any one of claims 1 to 7.

Images