A mask semi-automatic labeling method and system for a multi-layer stacked chip shielding area

Abstract

This invention relates to a semi-automatic mask annotation method for occluded regions of multi-layer stacked chips. It proposes an iterative training mechanism with multi-paradigm data fusion, integrating a small number of accurate masks and high-quality pseudo-masks generated through prompts into the retraining process. Weak prompts such as endpoints and bounding boxes can be used for candidate generation and selection. A stable multi-candidate mask generation strategy is proposed, generating multiple sets of candidate masks for the same occluded region by applying controlled perturbations to the bounding boxes, endpoints, and segmentation thresholds of the bonding wires. A quality control gate with multi-dimensional index fusion is also proposed, comprehensively judging whether the mask meets the standards based on multiple indicators. This invention can be directly deployed on existing device chains, achieving high-quality mask production and continuous bootstrapping iteration for high-resolution industrial images, reducing manual annotation costs and dataset and model library maintenance costs, and improving the efficiency of target bonding wire mask annotation in ultra-high-resolution industrial images.

Description

Technical Field

[0001] This invention relates to the field of automated chip labeling technology, and in particular to a semi-automatic mask labeling method and system for masked areas of multi-layer stacked chips. Background Technology

[0002] In industrial applications, the bonding wire inspection and quality analysis processes of related chip structures in semiconductor packaging, as well as subsequent bonding wire-related algorithms such as image generation and 3D reconstruction under obscured structures, and the design and experimentation of defect detection in obscured structures, all require the annotation of the mask in the bonding wire occlusion area of ​​multi-layer stacked chips. Figure 1 As shown in (a) and (b), the original images of multilayer stacked chips contain numerous continuous solder lines that are clearly visible on the upper layer and obscure the solder lines on the lower layer. These can be considered as the "target solder lines" for this annotation task. Other continuous solder lines, or solder lines that are themselves obscured, can be considered as "irrelevant solder lines." These solder lines exhibit dense, elongated, intersecting, and multi-layered coverage. Due to factors such as viewing angle, focus, reflection, and obstruction, only continuous solder lines on the upper layer, some bottom solder lines obscured by the upper layer, and some unobscured solder joints can be observed in the two-dimensional images acquired from a top-down perspective. For subsequent tasks such as solder line defect detection, structural analysis, and reliability assessment, it is not only necessary to obtain the contour data of the visible solder lines, but also to make reasonable inferences about the direction and approximate location of the solder lines in the obscured parts, and to use image generation methods to perform defect detection on the obscured parts.

[0003] Existing annotation methods mainly rely on manual pixel-by-pixel drawing or simple semi-automatic tools. For heavily occluded areas, the following prominent difficulties exist: First, considering the specific annotation task, such as... Figure 2 As shown in (a) and (b), multiple weld lines appear against a similar background, and the existing model cannot effectively distinguish between the "target weld lines" that require masking and the "irrelevant weld lines" that do not require masking; secondly, as Figure 3 As shown in (a) and (b), the target weld line and the occluded weld line overlap into a whole. When the existing model is automatically labeled, it cannot clearly identify the blurred and overlapping weld line boundaries, which is prone to incorrect labeling and makes it difficult to obtain high-quality masks, affecting the convergence of subsequent model retraining. Thirdly, manual labeling is time-consuming and involves a large amount of repetitive labor. Moreover, there are significant subjective differences among multiple workers who label sample masks at the same time.

[0004] Therefore, for the annotation process with a large number of samples, there is an urgent need for a method that can effectively utilize a small number of accurately labeled samples to generate a high-quality mask for the "target bonding wire" in the image of a multi-layer stacked chip without changing the structure of the basic segmentation model. At the same time, a unified screening can be achieved through quality scoring before writing to the dataset, reducing the burden of manual annotation and the risk of false labels, and providing controllable constraints on the extrapolation length and extrapolation range of the target bonding wire. Summary of the Invention

[0005] The purpose of this invention is to at least address one of the shortcomings of the prior art by providing a semi-automatic mask annotation method for masked areas of multi-layer stacked chips.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Specifically, a semi-automatic mask annotation method for masked areas of multi-layer stacked chips is proposed, including the following: The input data of the target chip is obtained, including the original image and structured metadata, and the target chip is a multi-layer stacked chip; The input data is preprocessed to obtain preprocessed data; The processed data is then prompted and guided to obtain guided data; The final candidate mask is obtained by generating candidate masks based on the post-guidance data. Calculate the preset evaluation index of the final candidate mask, and perform weighted fusion of the preset evaluation index to obtain the quality score; Determine whether the quality score is higher than a preset threshold. If it is, determine that the final candidate mask is qualified and execute automatic storage. Otherwise, determine that the final candidate mask is unqualified and remove it. The final candidate mask and its input data that are automatically stored are then stored and their consistency is verified.

[0007] Furthermore, specifically, the structured metadata includes, Machine number, batch number, process step, data acquisition timestamp, lens focal length and distortion coefficient, exposure / gain parameters, camera intrinsic and extrinsic parameters.

[0008] Furthermore, specifically, the process of preprocessing the input data to obtain preprocessed data includes: Perform integrity checks and duplicate checks on the input data; Geometric correction and data augmentation are performed on the original image based on calibration parameters; Perform photometric normalization on the original image; If the area of ​​the original image exceeds the area threshold, large-area image slicing processing is performed to obtain the sliced ​​area; if the original image does not exceed the area threshold, it itself is used as its sliced ​​area. Generate a list of ROIs to be processed; The final preprocessed data includes normalized images, a list of ROIs, and structured metadata.

[0009] Furthermore, specifically, the process of providing prompts and guidance to the processed data to obtain guided data includes, Generate a set of hints for each ROI in the ROI list. The cue set includes at least the bounding box and positive and negative points. If a coarse mask exists, it is included in the set. A center positive point cue is added for extremely small targets. Perform intelligent on-time sampling operation, apply distance transformation to the bonding wire targets within the ROI area, extract the skeleton centerline, and sample on-time points at fixed intervals on the centerline as strong cues for target activation; Perform morphological edge negative point sampling operation, morphologically dilate the coarse edge within the bounding box of the bonding wire target in the ROI region to generate a narrow edge band, and sample negative points in the background region outside the narrow band as a strong constraint to suppress irrelevant bonding wires; Finally, the cue set is encoded and standardized, all cue coordinates are mapped to the model input scale, the data type format is unified, and it is encoded into a tensor that the basic segmentation model can directly receive. Together with the original image tensor, it is output to obtain the guided data.

[0010] Furthermore, specifically, the process of generating the final candidate mask based on the post-guidance data includes: Initialize the disturbance generator parameters, set the spatial disturbance range, and set the probability threshold disturbance range; Perform multi-path parallel perturbation inference, apply K independent random perturbations to the guided data prompts, and generate a set containing K candidate masks and their corresponding confidence maps; Perform candidate mask clustering, calculate the cross-union ratio (IoU) between each pair of K candidate masks, merge masks with IoU greater than the cross-union threshold into the same cluster, and select the mask with the highest average confidence within the cluster as the center mask of the cluster. Calculate the cluster stability index. For each cluster, calculate the average IoU value of all masks in the cluster relative to the central mask, which is used as the stability index of the cluster. The comprehensive scoring is used to select the best, and discrete clusters with a stability index lower than 0.85 are removed. Then, the average confidence level and stability index of each cluster are read, and the comprehensive score is calculated by weighting the average confidence level and stability index according to the preset coefficients. The final candidate mask is obtained by selecting the cluster center mask with the highest comprehensive score.

[0011] Furthermore, specifically, a preset evaluation index for the final candidate mask is calculated, and the preset evaluation index is weighted and fused to obtain a quality score, including: The confidence score is calculated as the internal confidence component of the final candidate mask. The mean pixel classification probability of the final candidate mask is extracted as the confidence score. The stability component, or stability score, is obtained by directly reading the stability index of the final candidate mask as the stability score. The boundary fit component, i.e. the boundary fit score, is calculated by performing Canny edge detection on the original image slice region to generate physical edge bands. The proportion of the mask boundary pixels of the final candidate mask falling within the physical edge band is calculated as the boundary fit score. The expansion control component, i.e. the expansion control score, is calculated by expanding the bounding box outward by a preset fixed number of pixels to generate a safety buffer zone. The area percentage of the final candidate mask that exceeds the safety buffer zone is calculated, and the value of 1 - area percentage is used as the expansion control score. Calculate the unified quality score Q by weighting and summing the four components according to their respective preset weights to obtain the normalized quality score Q.

[0012] Furthermore, specifically, the final candidate mask and its input data that are automatically stored in the database are subjected to database storage and consistency verification, including: Establish a data index, create or update the dataset index file for the current round on the storage server, and assign a unique sample ID; Perform a spatial consistency check to examine the IoU between the current final candidate mask and the masks already in the database. If a spatial conflict is found (i.e., IoU > 0.9 but the categories are inconsistent), retain the mask with the higher Q value and discard the mask with the lower Q value. Perform a joint check of IoU and boundary consistency. If there is a conflict with the mask already in the database, i.e. IoU>0.9 but the category is inconsistent or the boundary difference is greater than the threshold, then the mask with the higher Q value is retained and the mask with the lower Q value is removed. If the Q values ​​are the same, the rule decision of user-processed (i.e., fine-tuned) is preferred over automatic rule and the index is updated. Update and save the data, save the qualified mask image to the specified directory, and write the associated metadata, including Q-value information, into the structured database.

[0013] Furthermore, the method also includes, After completing the data entry and consistency verification, iterative training with multi-paradigm fusion will be performed, specifically including: Construct a hybrid training set, extract manually labeled Anchor samples and high-quality pseudo-label samples from the current round of database entry, i.e., the final candidate mask and its input data Expansion, according to a preset ratio, and load the corresponding weak prompting information Guidance; Configure the optimized sampler, set the sampler parameters according to the statistical distribution of the solder wire width, and increase the sampling weight of narrow solder wire samples (i.e., solder wire widths lower than the preset width) to twice that of regular samples. Configure a hybrid loss function, setting it to a weighted sum of regional Dice Loss and boundary Boundary Loss, and setting the weight coefficient of Boundary Loss to 2.0; simultaneously enable point-hint-based consistency constraint loss. Perform fine-tuning training on the model, loading the model weights from the previous round, and conduct incremental training for 50-100 epochs on the mixed dataset. The best model is selected by evaluating the boundary F1 score of the model for each epoch on an independent validation set. The model with the highest score is saved as Best_Model_vN and deployed to the inference server.

[0014] Furthermore, the method also includes, The updated Best_Model_vN is used to process the new batch of unlabeled image data; Statistical observation indicators, calculate the observation acceptance rate of the current batch, and the ratio of qualified samples to total ROIs; Perform online threshold updates. Calculate the difference between the observed acceptance rate and the system's preset target acceptance rate; use this difference and a preset step size to set the preset threshold for the next round, i.e., the admission threshold. Make minor adjustments and updates; Determine the termination condition and check whether the sample coverage of the current dataset has reached the preset target; if not, automatically trigger the next iteration; if it has, terminate the annotation process and generate the final report.

[0015] This invention also proposes a semi-automatic mask labeling device for masking areas of multi-layer stacked chips, comprising the following: The system comprises a sample inspection platform, input / output interfaces, a vision system, an industrial computer, a network interface, and a storage server. The sample inspection platform communicates bidirectionally with each module. The vision system handles station triggering, exposure control, and image acquisition. The industrial computer executes the steps of the method. The network interface facilitates data exchange with the enterprise network, training server, and external systems. The storage server stores datasets and models numbered by round, along with runtime records. The input / output interfaces interact with the upper-level business system, and when centralized training is not available, the training server can be omitted, and retraining can be performed locally on the industrial computer.

[0016] The beneficial effects of this invention are as follows: This invention proposes a semi-automatic mask annotation method and apparatus for masking obscured areas of multi-layer stacked chips. 1. Compatible with existing platforms with minimal deployment changes: The method of this invention is implemented in stages such as candidate generation, quality scoring, and threshold splitting without requiring changes to existing hardware structures such as cameras, industrial computers, and network switches, nor does it require changes to the network structure of the basic segmentation model. It can be directly deployed on existing vision system platforms and servers, reducing the introduction cost and integration complexity.

[0017] 2. Quality control before data entry to reduce the risk of false labels: This invention abandons the traditional crude mode of relying solely on model confidence for screening, and constructs a system that integrates the internal reliability of the model. and Image consistency and the rationality of boundaries This mechanism acts as a gate before data is stored, performing rigorous quantitative audits on each generated mask and allowing only samples with a comprehensive score higher than an adaptive threshold to be included. This design effectively blocks low-quality samples with high confidence but that do not conform to the target bonding mask, ensuring the purity of the incremental dataset and thus ensuring the convergence stability and predictability of subsequent model iterations.

[0018] 3. Specialized Design for Target Bond Lines: Throughout the training phase, this invention employs data augmentation strategies for slender bond line structures, ensuring sufficient attention is given to narrow-width bond lines during training. For target bond lines in occluded areas caused by multi-layer stacking, limited-amplitude mask extrapolation is performed along the main direction of the bond line, and verification is conducted through geometric continuity and edge consistency, eliminating candidates with significant over-extension. Compared to traditional processes relying solely on single-shot segmentation results, this invention improves the consistency of bond line mask boundaries in scenarios with poor focus and local occlusion, and enhances the integrity of the mask in occluded areas without significantly increasing the risk of mislabeling.

[0019] 4. Significantly reduced manual input and cycle time: Manual annotation is only performed on a small number of samples. The main process then adopts an automatic path of writing only when the standard is met and not writing when the standard is not met, avoiding pixel-by-pixel annotation and verification of all samples, thereby reducing the annotation time per sample and the overall iteration cycle.

[0020] 5. The process is traceable and the results are reproducible, meeting the needs of audit management: Data sets and models are numbered and managed by round, and key elements such as thresholds, weights, and pass rates are recorded. Experimental results can be reproduced at any point in time and horizontal comparisons can be carried out, which is conducive to anomaly localization and quality auditing. Attached Figure Description

[0021] The above and other features of this disclosure will become more apparent from the detailed description of the embodiments illustrated in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals denote the same or similar elements. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings: Figure 1 (a) is a schematic diagram of a certain region of a multilayer stacked chip sample. Figure 1 (b) is a schematic diagram of the obstructed portion of the area; Figure 2 (a) is a schematic diagram of incorrect target labeling. Figure 2 (b) is Figure 1 A diagram illustrating the correct target labeling in the middle; Figure 3 (a) is a schematic diagram of error boundary labeling. Figure 2 (b) is Figure 1 A diagram illustrating the correct boundary labeling in the middle; Figure 4 This is a schematic diagram of the structure of a semi-automatic mask labeling device for the masking area of ​​multi-layer stacked chips proposed in this invention; Figure 5 This is a flowchart of a semi-automatic mask annotation method for the masking area of ​​multi-layer stacked chips proposed in this invention; Figure 6 Here is a flowchart for S101; Figure 7 Here is the flowchart for S102; Figure 8 (a) is a schematic diagram of the intelligent sampling strategy of the Zhengdian sampling strategy. Figure 8 (b) is a schematic diagram of the random sampling strategy of the positive sampling strategy; Figure 9 (a) is a schematic diagram of the intelligent sampling strategy for negative sampling strategy. Figure 9 (b) is a schematic diagram of the random sampling strategy for the negative sampling strategy; Figure 10 Here is the flowchart for S103; Figure 11 (a) is a schematic diagram of generating K groups of candidate masks. Figure 11 (b) is a schematic diagram for calculating the stability score and selecting the best option; Figure 12 Here is the flowchart for S104; Figure 13 The flowchart for S107; Figure 14 (a), (b), and (c) are original images of sample slices in one embodiment; Figure 15 (a), (b), and (c) are respectively for Figure 14 (a), (b), and (c) are automatically generated mask images after being processed by the method proposed in this invention; Figure 16 (a), (b), and (c) are respectively Figure 14 Central Plains Map and Figure 15 Image showing the effect of mask fusion. Detailed Implementation

[0022] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The same reference numerals used throughout the accompanying drawings indicate the same or similar parts.

[0023] Considering the problems of high manual costs, large subjective differences, lack of unified criteria for occlusion extrapolation, and uncontrollable quality of pseudo-labels in masking of bonding wire occlusion areas in multi-layer stacked chip scenarios, this invention defines "target bonding wires" and "irrelevant bonding wires" for occlusion areas that are difficult for models to distinguish in multi-layer stacked scenarios: "Target bonding wires" refer to bonding wires visible in the current layer or upper layer structure that actively occlude the lower layer structure. These bonding wires are the annotation targets of this invention, and their continuous parts must be completely annotated; "irrelevant bonding wires" refer to background bonding wires located in the lower layer and occluded by the upper layer structure. This invention aims to automatically suppress the segmentation response to such bonding wires through specific prompting strategies and quality screening, avoiding mislabeling them as targets of the current layer.

[0024] Example 1, referring to Figure 5 This invention proposes a semi-automatic mask annotation method for masked areas of multi-layer stacked chips, comprising the following: S101. Obtain input data of the target chip, the input data including the original image and structured metadata, the target chip being a multi-layer stacked chip; The input data is preprocessed to obtain preprocessed data; S102. Prompt and guide the processed data to obtain guided data; S103. Generate the final candidate mask based on the post-guidance data; S104. Calculate the preset evaluation index of the final candidate mask, and perform weighted fusion on the preset evaluation index to obtain the quality score; S105. Determine whether the quality score is higher than a preset threshold. If yes, determine that the final candidate mask is qualified and perform automatic storage. Otherwise, determine that the final candidate mask is unqualified and remove it. Specifically, step S105 will be further refined by setting a preset threshold. As ordered To filter out mask samples that do not meet the requirements, the system will calculate the... Value and preset threshold Compare. If This indicates that the mask meets the requirements in terms of confidence level, stability, physical fit and geometric rationality. The system judges it as a qualified pseudo-label and performs automatic storage. Otherwise, it is judged as a low-quality sample and is removed. The specific unqualified dimensions are recorded for subsequent statistical analysis. Through this two-way splitting mechanism with a single quality threshold as the core, it can be ensured that the bonding masks written into the dataset have a unified quality benchmark.

[0025] In practice, scale-adaptive threshold splitting includes the following sub-steps: Step 24: Read the corresponding quality admission threshold from the system configuration table. Perform a binary flow splitting decision. Compare the current mask score Q with the threshold. .like The mask is marked as a "qualified pseudo-label" and pushed into the inventory queue; if The sample is marked as "discarded", its Q value and the values ​​of each component are recorded, and then pushed into the log database.

[0026] Step 26: Generate the pass list and the discard list, which will then be entered into the subsequent steps and the statistics queue, respectively.

[0027] S106. Perform storage and consistency verification on the final candidate mask and its input data that are automatically stored in the database.

[0028] To further refine step S101, as a preferred embodiment of the present invention, specifically, the structured metadata includes, Machine number, batch number, process step, data acquisition timestamp, lens focal length and distortion coefficient, exposure / gain parameters, camera intrinsic and extrinsic parameters.

[0029] Specifically, the process of preprocessing the input data to obtain preprocessed data includes: Perform integrity checks and duplicate checks on the input data; Geometric correction and data augmentation are performed on the original image based on calibration parameters; Perform photometric normalization on the original image; If the area of ​​the original image exceeds the area threshold, large-area image slicing processing is performed to obtain the sliced ​​area; if the original image does not exceed the area threshold, it itself is used as its sliced ​​area. Generate a list of ROIs to be processed; The final preprocessed data includes normalized images, a list of ROIs, and structured metadata.

[0030] In this preferred embodiment, the submodule process is as follows: Figure 6 As shown, the system imports industrial images and metadata such as machine, batch, and process step from a multi-layer stacked chip production line, and performs data integrity verification and duplicate detection. The images are geometrically and photometrically normalized, and a fixed-size, fixed-overlap slicing strategy is adopted when necessary. The slice size is determined by the computational performance of the training server. The system aggregates historical detector outputs, previous inference results, and manual selections to form a list of ROIs to be processed, and outputs normalized images, ROIs, and structured metadata.

[0031] In practice, data import and preprocessing, i.e., step S101, includes the following sub-steps: Step 1: The system obtains the original image and structured metadata from the sample testing platform. The metadata includes at least the machine number, batch number, process step, acquisition timestamp, lens focal length and distortion coefficient, exposure / gain parameters, and camera intrinsic and extrinsic parameters.

[0032] Step 2: Perform integrity verification on the input file. Calculate the MD5 hash of the file, check whether the image size, number of channels, bit depth, and encoding format meet the preset standards, search the sample library for duplicates and remove duplicate samples, mark samples with missing or abnormal metadata as invalid and exit the process.

[0033] Step 3: Perform geometric correction and data augmentation on the image based on the calibration parameters. Perform distortion correction, rotation, scaling, and edge cropping to unify the image to a standard resolution and a unified coordinate system, and save the mapping matrix from the acquired coordinates to the standard coordinates for subsequent backprojection.

[0034] Step 4: Perform photometric normalization on the image. Perform white balance and dark current correction, and use limited contrast histogram equalization such as CLAHE and lightweight denoising such as bilateral filtering while preserving edge information to ensure that the illumination differences of the sampled image are not too large and that the gray-level distribution of the input model image is consistent.

[0035] Step 5: Perform large-format image slicing processing. Using 256... A fixed size of 256 pixels and a fixed overlap rate of 15% are used to segment ultra-high resolution images into multiple tiles; a metadata index containing the original image coordinates, row and column indices, and overlap area definitions is generated for each tile.

[0036] Step 6: Integrate multi-source information to generate a list of ROIs to be processed. Integrate candidate boxes from historical detectors, bounding boxes from the previous round of inference, manual point hints, and coarse masks; unify all sources to a standard coordinate system; delete candidates that are out of bounds, have too small an area, or have abnormal aspect ratios.

[0037] Step 7: Output the normalized image, ROI list, and structured metadata, including cross-block indexes and coordinate mappings, and then proceed to the prompt generation stage.

[0038] Step 102 is further refined. As a preferred embodiment of the present invention, the process of prompting and guiding the processed data to obtain guided data includes the following: Generate a set of hints for each ROI in the ROI list. The cue set includes at least the bounding box and positive and negative points. If a coarse mask exists, it is included in the set. A center positive point cue is added for extremely small targets. Perform intelligent on-time sampling operation, apply distance transformation to the bonding wire targets within the ROI area, extract the skeleton centerline, and sample on-time points at fixed intervals on the centerline as strong cues for target activation; Perform morphological edge negative point sampling operation, morphologically dilate the coarse edge within the bounding box of the bonding wire target in the ROI region to generate a narrow edge band, and sample negative points in the background region outside the narrow band as a strong constraint to suppress irrelevant bonding wires; Finally, the cue set is encoded and standardized, all cue coordinates are mapped to the model input scale, the data type format is unified, and it is encoded into a tensor that the basic segmentation model can directly receive. Together with the original image tensor, it is output to obtain the guided data.

[0039] In this preferred embodiment, the submodule process is as follows: Figure 7 As shown, the core problem addressed by this module is how to transform target locations in an image into guiding information that the segmentation model can understand. This cues and guidance mainly include the following two categories: Bounding box hints and guidance strategy: A rectangle generated based on manual selection is used to define the approximate range of the target solder line; Positive / Negative Dot Hints and Guidance Strategy: Intelligent sampling is employed. "Positive dots" generated at the center of the wire bonding skeleton are used to activate the target area, while "negative dots" located in unrelated wire bonding areas are used to suppress non-target areas. In particular, intelligent sampling offers advantages over random sampling in the following ways: Figure 8 As shown in (a) and (b) (where the green dot in (a) is located at the center of the target weld line, and the red dot in (b) is located at the boundary of the target weld line), based on the centrality sampling strategy of distance transformation, the marked positive dots will not appear at the edge of the target weld line but near the center line, thus enhancing the quality of the positive dots; Figure 9 As shown in (a) and (b) (where the green dot in (a) is located in the blue expansion narrow band region of the target bond line, and the red dot in (b) is located in a random position in the background), based on the morphological edge negative point sampling strategy, the edge of the target bond line is expanded to obtain the expansion narrow band region, and the negative point is sampled at a position that is very close to the target bond line but not on the target bond line, in order to suppress the chaotic positioning of the automatically generated mask and exceed the boundary of the target bond line; The system constructs the aforementioned cue set for each ROI and performs a key optimization: sampling positive and negative points near the target boundary with a fixed bandwidth. This significantly improves the model's ability to effectively distinguish between "target weld lines" and "irrelevant weld lines" by establishing clear "foreground-background" anchor points at the center and outer edges of the target region, generating more accurate mask annotations. Finally, the cue set is encoded into an input tensor that the basic segmentation model can directly accept.

[0040] In practical implementation, the prompts and guidance, i.e., step S102, include the following sub-steps: Step 8: Generate a set of hints for each ROI It must contain at least the bounding box and positive and negative points. If a coarse mask exists, it should be incorporated into the set. A center point hint should be added for very small targets.

[0041] Step 9: Perform intelligent spot sampling. Perform a distance transformation (DistanceTransform) on the bonding wire targets within the ROI region to extract the skeleton centerline; sample spot points on the centerline at fixed intervals as strong cues for activated targets.

[0042] Step 10: Perform morphological edge negative point sampling. Morphologically dilate the coarse edges within the target bounding box to generate a narrow edge band; sample negative points in the background region outside the narrow band as a strong constraint to suppress irrelevant solder lines.

[0043] Step 11: Encode and standardize the cue set. Map all cue coordinates to the model input scale, unify the data type format, and encode them into tensors that the basic segmentation model can directly accept, outputting them along with the original image tensors.

[0044] In a preferred embodiment of the present invention, specifically, the process of generating a final candidate mask based on post-guidance data includes: Initialize the disturbance generator parameters, set the spatial disturbance range, and set the probability threshold disturbance range; Perform multi-path parallel perturbation inference, apply K independent random perturbations to the guided data prompts, and generate a set containing K candidate masks and their corresponding confidence maps; Perform candidate mask clustering, calculate the cross-union ratio (IoU) between each pair of K candidate masks, merge masks with IoU greater than the cross-union threshold into the same cluster, and select the mask with the highest average confidence within the cluster as the center mask of the cluster. Calculate the cluster stability index. For each cluster, calculate the average IoU value of all masks in the cluster relative to the central mask, which is used as the stability index of the cluster. The comprehensive scoring is used to select the best, and discrete clusters with a stability index lower than 0.85 are removed. Then, the average confidence level and stability index of each cluster are read, and the comprehensive score is calculated by weighting the average confidence level and stability index according to the preset coefficients. The final candidate mask is obtained by selecting the cluster center mask with the highest comprehensive score.

[0045] In this preferred embodiment, the process of the multi-candidate mask generation module based on small input variations is as follows: Figure 10 As shown, since there are certain positioning errors in the manual annotation process, this invention proposes a method in this module to effectively eliminate low-quality samples caused by errors in the mask generation process. Furthermore, the large-scale segmentation model used in this invention is highly sensitive to cue information; if the cue information contains errors, it will significantly impact the quality of the generated mask. Therefore, the working principle of this module is based on a stability assumption: a high-quality segmentation result should maintain geometric stability even with a certain amount of error, i.e., when the input cue undergoes minor changes; conversely, if a minor change in the input leads to a drastic change in the result, then the result is unreliable.

[0046] Therefore, such as Figure 11 As shown in (a), this system generates K candidate masks by applying controlled perturbations to the inference process without changing the model structure. K is selected from 4 to 8 depending on the server's computing performance. Spatial perturbation: Simulates the positioning error of manual annotation. Centered on the original prompt point, within a 3×3 pixel area, the prompt coordinates (x, y) are randomly shifted in a uniform distribution. Parameter perturbation: Tests the model's sensitivity to classification boundaries. It involves setting the binarization segmentation probability threshold of the model. Perform within the interval [0.45, 0.55] Random fluctuations in amplitude; like Figure 11 As shown in (b), the system then obtains the corresponding confidence scores for the K candidate mask sets and performs the following stability optimization process: Clustering deduplication: Calculate the crossover ratio (IoU) between each pair of candidate mask groups and denot it as . Masks with IoU > 0.90 are considered to belong to the same cluster C.

[0047] Stability quantization: for each cluster Calculate its stability index The index is defined as the average IoU of all masks within the cluster relative to the central mask. A higher index indicates stronger model consistency under perturbations.

[0048] The system calculates a stability score based on stability metrics. , ,in As the balance coefficient, it can be taken as... The system uses this score to eliminate discrete clusters with a stability index below a preset threshold, such as 0.85, retaining only the clusters that meet the criteria. The center mask of the highest-ranking cluster is used as the final candidate for that region, and the stability index is used. Retain entry into step S104.

[0049] In practical implementation, the multi-candidate mask generation and deduplication step S103 includes the following sub-steps: Step 12: Initialize the disturbance generator parameters. Set the spatial disturbance range to 3. Uniform distribution within a 3-pixel range; the probability threshold perturbation range is set to [0.45, 0.55].

[0050] Step 13: Perform multi-path parallel perturbation inference. While keeping the model weights frozen, apply K=8 independent random perturbations to the same set of input prompts, perform model inference in parallel, and generate a set containing 8 candidate masks and their corresponding confidence maps.

[0051] Step 14: Perform candidate mask clustering. Calculate the cross-union ratio (IoU) between each pair of the 8 candidate masks; merge masks with an IoU greater than 0.90 into the same cluster; select the mask with the highest average confidence within the cluster as the center mask of that cluster.

[0052] Step 15: Calculate the cluster stability index. For each cluster, calculate the average IoU value of all masks within the cluster relative to the central mask, which serves as a quantitative indicator of the cluster's stability.

[0053] Step 16: Perform comprehensive scoring and selection. Read the average confidence level and stability index of each cluster, and calculate the comprehensive score according to preset coefficients, such as a confidence coefficient of 0.5 and a stability coefficient of 0.5; remove discrete clusters with a stability index lower than 0.85.

[0054] Step 17: Select the cluster center mask with the highest comprehensive score as the final candidate, and pass the corresponding stability index to the next step.

[0055] Based on the above, this invention proposes a pseudo-mask generation strategy based on controlled perturbation and stable screening. To ensure the quality of the generated pseudo-masks, this invention further proposes a stable multi-candidate mask generation strategy. This strategy aims to address the sensitivity of the segmentation model to minor changes in prompts and simulates the problem of box prompts and point prompts positioning errors in the manual annotation process. It generates multiple sets of candidate masks for the same occluded area by applying controlled perturbations to the prompt information such as the bounding box and endpoints of the bonding wire and the segmentation threshold.

[0056] Step S104 is further refined. As a preferred embodiment of the present invention, specifically, a preset evaluation index for the final candidate mask is calculated, and the preset evaluation index is weighted and fused to obtain a quality score, including: The confidence score is calculated as the internal confidence component of the final candidate mask. The mean pixel classification probability of the final candidate mask is extracted as the confidence score. The stability component, or stability score, is obtained by directly reading the stability index of the final candidate mask as the stability score. The boundary fit component, i.e. the boundary fit score, is calculated by performing Canny edge detection on the original image slice region to generate physical edge bands. The proportion of the mask boundary pixels of the final candidate mask falling within the physical edge band is calculated as the boundary fit score. The expansion control component, i.e. the expansion control score, is calculated by expanding the bounding box outward by a preset fixed number of pixels to generate a safety buffer zone. The area percentage of the final candidate mask that exceeds the safety buffer zone is calculated, and the value of 1 - area percentage is used as the expansion control score. Calculate the unified quality score Q by weighting and summing the four components according to their respective preset weights to obtain the normalized quality score Q.

[0057] In this preferred embodiment, the submodule process is as follows: Figure 12 As shown, this module serves as a crucial link between candidate generation and data storage, and is also the core innovation of this invention. It aims to construct a multi-dimensional quality assessment and screening mechanism. The core purpose of this mechanism is to establish objective quantitative standards to automatically identify and eliminate mislabeled samples that deviate from the target bonding wire mask annotation effect, while reducing the cost of manual annotation and eliminating the need for manual review.

[0058] This module does not rely on a single model output probability, but instead integrates the model's internal reliability, image consistency, and boundary reasonableness to evaluate the optimal candidate mask output by S103: Internal reliability assessment: The system first quantifies the model's inherent certainty regarding the current inference result, specifically including two evaluation dimensions: Confidence Extract the mean classification probability of pixels within the candidate mask region. This metric quantifies the overall confidence level of the model when it classifies pixels within the mask as the target category.

[0059] Disturbance stability : The stability index calculated in the controlled perturbation inference process of the previous step. Based on the stability theory of solving inverse problems, the real and robust target segmentation result should have geometric invariance to small input perturbations, and by using the calculation results of S103, this module can obtain the structural robustness characteristics of the mask at low cost.

[0060] Image consistency verification: To verify the consistency between the segmentation results and the physical features of the image, the system introduces a gradient-based edge conformance evaluation to screen whether the automatically generated mask completely selects the region features of the target bond line. The evaluation principle is that although the bond line structure is complex, the edges of its visible line segments should highly coincide with the gradient regions in the image where the pixel grayscale changes drastically. Therefore, the output mask should completely adhere to the edge of the target bond line rather than be inside it.

[0061] Implementation method: The system performs edge detection, such as the Canny operator, only on the visible regions of the original image to generate physical edge response bands. Subsequently, the intersection-over-union (IoU) ratio between the candidate mask boundary and this physical edge band is calculated. This metric is a physical layer validation of the effectiveness of the S102 cueing strategy—it only has a high response value when the mask boundary precisely matches the real physical contours in the image.

[0062] Boundary rationality constraints: To address the model's inference behavior regarding occluded regions, the system implements strict geometric boundary constraints to suppress excessive mask extrapolation for areas outside the target bond line boundaries, ensuring that the automatically generated mask does not select too much content. Based on the initial cue range, the system constructs a permissible safety buffer zone using morphological dilation operations. The percentage of the mask's area extending beyond this buffer zone is calculated. The geometric rationality of occlusion inference is quantitatively evaluated. Any predicted region that significantly exceeds the prior range will be considered an abnormal expansion, resulting in a lower quality score.

[0063] Based on the above logic, the system adopts the following weighted fusion strategy to integrate the metrics from the four levels into a normalized quality score.

[0064] ; in, The weight coefficients for each component are given and satisfy the following conditions: =1, used to balance the contributions of different evaluation dimensions, can be taken as... The quality score calculated using this formula This is used as input to the next step for threshold splitting.

[0065] In practical implementation, the quality score Q calculation, i.e., step S104, includes the following sub-steps. Step 18: Calculate the internal confidence component. Extract the mean pixel classification probability within the optimal candidate mask region as the confidence score.

[0066] Step 19: Obtain the stability component. Directly read the stability index corresponding to the candidate mask output in the previous step, and use it as the stability score.

[0067] Step 20: Calculate the boundary fit component. Perform Canny edge detection on the visible area of ​​the original image slice to generate physical edge bands; calculate the proportion of mask boundary pixels that fall within these physical edge bands, which is used as the boundary fit score.

[0068] Step 21: Calculate the outward expansion control component. Based on the initial prompt box of S102, expand outward by a fixed number of pixels, such as 15 pixels, to generate a safety buffer band; calculate the area ratio of the mask that exceeds the buffer band, and calculate (1 - the excess ratio) as the outward expansion control score.

[0069] Step 22: Calculate the uniform quality score Q. The four components are weighted and summed according to preset weights such as 0.4, 0.2, 0.2, and 0.2 to obtain the normalized quality score Q.

[0070] Step 23: Output each candidate And four components, used for threshold splitting.

[0071] Based on the above, this invention proposes a multi-dimensional quality control gate that integrates various metrics. To achieve a closed-loop process, this invention innovatively creates a multi-dimensional quality control gate mechanism. This gate is responsible for automatically identifying whether the pseudo-masks generated in the previous step meet the "entry" criteria. Its key innovation lies in the fact that it not only evaluates the model's internal confidence, such as probability and entropy, but also integrates specific target solder line annotation metrics for the solder line occlusion scenario in this task, especially the continuity of solder line boundaries and the degree of mask extrapolation in complex regions.

[0072] As a refinement of step S106, and as a preferred embodiment of the present invention, specifically, the final candidate mask and its input data automatically stored in the database are subjected to database storage and consistency verification, including: Establish a data index, create or update the dataset index file for the current round on the storage server, and assign a unique sample ID; Perform a spatial consistency check to examine the IoU between the current final candidate mask and the masks already in the database. If a spatial conflict is found (i.e., IoU > 0.9 but the categories are inconsistent), retain the mask with the higher Q value and discard the mask with the lower Q value. Perform a joint check of IoU and boundary consistency. If there is a conflict with the mask already in the database, i.e. IoU>0.9 but the category is inconsistent or the boundary difference is greater than the threshold, then the mask with the higher Q value is retained and the mask with the lower Q value is removed. If the Q values ​​are the same, the mask that has been processed by the user (i.e., refined by the user) is preferred over the automatic rule decision and the index is updated. Update and save the data, save the qualified mask image to the specified directory, and write the associated metadata, including Q-value information, into the structured database.

[0073] In this preferred embodiment, the system stores samples and metadata according to a pre-designed data structure, records batch information such as source markers such as automatic entry and refined entry, and registers fields such as candidate serial number and threshold. It performs joint IoU and boundary consistency verification on overlapping areas across blocks to avoid duplicate or conflicting writes to the database and ensure data traceability.

[0074] Specifically, the library consistency verification, i.e., step S106, includes the following sub-steps. Step 27: Create a data index. Create or update the dataset index file for the current round on the storage server, assigning unique sample IDs.

[0075] Step 28: Perform spatial consistency check. Check the IoU between the current mask and the masks already in the database in the cross-slice overlap region; if a spatial conflict is found (IoU > 0.9 but the categories are inconsistent), retain the one with the higher Q value and discard the low-scoring mask.

[0076] Step 29: Perform a joint IoU and boundary consistency check in the cross-block overlapping area; if there is a conflict with the already included mask, that is, IoU>0.9 but the category is inconsistent or the boundary difference>threshold, then follow the "Q high priority, source priority" rule, that is, fine-tuning is better than automatic rule decision and update the index.

[0077] Step 30: Update data and save. Save the qualified mask image to the specified directory and write the associated metadata, including Q-value, generation parameters, and other information, into the structured database.

[0078] In a preferred embodiment of the present invention, the method further includes, S107. After completing the data entry and consistency verification, iterative training with multi-paradigm fusion will be performed, specifically including: Step 31: Construct a hybrid training set, extract manually labeled Anchor samples and high-quality pseudo-label samples added in this round of database according to a preset ratio, namely the final candidate mask and its input data Expansion, and load the corresponding weak prompt information Guidance. Step 32: Configure the optimized sampler. Based on the statistical distribution of the solder wire width, set the sampler parameters and increase the sampling weight of narrow-width solder wire samples (i.e., solder wire widths lower than the preset width) to twice that of regular samples. Step 33: Configure the hybrid loss function, setting it to a weighted sum of the region Dice Loss and the boundary Boundary Loss, and setting the weight coefficient of the Boundary Loss to 2.0; simultaneously enable point-hint-based consistency constraint loss. Step 34: Perform model fine-tuning training, load the model weights from the previous round, and perform incremental training for 50-100 epochs on the mixed dataset. Step 35: Select the best model. Evaluate the boundary F1 score of each Epoch model on the independent validation set, select the model with the highest score, save it as Best_Model_vN, and deploy it to the inference server.

[0079] In this preferred embodiment, the submodule process is as follows: Figure 13 As shown, the core of this module lies in using the high-quality dataset updated in the current round to perform incremental training on the basic segmentation model. Unlike traditional fully supervised training that relies solely on extensive manual annotation, this module constructs a hybrid data-driven iterative training method. Combining data augmentation strategies, it incorporates a small number of precise masks, high-quality pseudo-masks automatically generated by the system, and low-cost weak prompts into the training process, gradually improving the model's ability to segment and complete target weld lines.

[0080] Among them, multi-paradigm hybrid data construction: The system dynamically constructs the training dataset for the current round, which organically integrates data sources from three paradigms to achieve a balance between low-cost bootstrapping and high-quality generation: Precise supervised data (Anchor): This includes a small number of finely annotated bond line masks. This type of data serves as "anchor points" to maintain the model's basic ability to discriminate bond line features, preventing the model from forgetting or shifting concepts during iteration.

[0081] Self-supervised pseudo-label data Expansion: This includes a large number of high-quality pseudo-masks that have been filtered and entered into the database through the quality gate of the preceding steps. This is the main body of the data increment. By introducing massive amounts of unlabeled data, the model's coverage of different machines and different occlusion patterns is expanded, achieving a "bootstrapping" performance improvement.

[0082] Weakly supervised guidance data: At the training input, low-cost weak annotation information such as the endpoints of the bonding wires and bounding boxes is fully utilized. During training, the system randomly simulates these weak cues as the input conditions for the model, enabling the model to reconstruct the complete pixel-level mask even with weak cues of only sparse points or boxes.

[0083] S108. As a preferred embodiment of the present invention, the method further includes, Step 36: Start the batch inference task: Process the new batch of unlabeled image data using the updated Best_Model_vN; Step 37: Statistically analyze the observation indicators and calculate the observation acceptance rate for the current batch: number of qualified samples / total number of ROIs; Step 38: Perform online threshold update. Calculate the difference between the observed acceptance rate and the system's preset target acceptance rate; use this difference and the preset step size to set the preset threshold for the next round, i.e., the admission threshold. Make minor adjustments and updates; Step 39: Determine the termination condition and check whether the sample coverage of the current dataset has reached the preset target; if not, automatically trigger the next iteration; if it has, terminate the annotation process and generate the final report.

[0084] In this preferred embodiment, the system uses the updated model Best_Model_vN to perform batch inference on a new batch of unlabeled data, and can continue to call the above process. To adapt to data distribution drift in the production environment, the system introduces a feedback mechanism: it calculates the current observation acceptance rate (Pass Rate) and compares it with the target acceptance rate. Labelers can then dynamically fine-tune the triage threshold based on this. While ensuring the minimum quality, the annotation system should be kept running efficiently, and the annotation process should be stopped when the quality and quantity of mask samples reach a certain level.

[0085] This invention proposes an iterative training mechanism based on multi-paradigm data fusion. This mechanism integrates a small number of precise masks with high-quality pseudo-masks generated through prompts into the retraining process, and can combine weak prompts such as endpoints and bounding boxes for candidate generation and filtering. The core of this method lies in its reliance on low-cost weak annotations for guidance and its use of pseudo-masks generated by the system itself for iterative training, thereby gradually improving the model's ability to segment and complete target weld lines.

[0086] Example 2: The present invention also proposes a semi-automatic mask labeling device for the masking area of ​​multi-layer stacked chips, comprising the following: like Figure 4As shown, the device includes a sample inspection platform 10, an input / output interface 11, a vision system 12, an industrial computer 13, a network interface 14, and a storage server 15. The sample inspection platform 10 serves as the core of system control and data exchange, communicating bidirectionally with each module. The vision system 12 handles station triggering, exposure control, and image acquisition; the industrial computer 13 executes the method steps of this invention; the network interface 14 enables data exchange with the enterprise network, the training server 16, and external systems; the storage server 15 stores datasets and models numbered by round, as well as running records; the input / output interface 11 interacts with the upper-level business system. When centralized training is not available, the training server 16 can be omitted, and retraining can be performed locally by the industrial computer 13. In the embodiment, as shown... Figure 14 The original sample slices shown in (a), (b), and (c) were used for testing. Two of the weld lines were the target weld lines for the masking process, and masking was performed on them. The remaining weld lines were irrelevant. The result after automatic labeling is as follows: Figure 15 The mask annotation diagrams shown in (a), (b), and (c) are as follows, and... Figure 16 The fusion effect diagrams shown in (a), (b), and (c) are shown in the figure.

[0087] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0088] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0089] Although the description of the invention has been quite detailed and particularly of several described embodiments, it is not intended to limit it to any of these details or embodiments or any particular embodiment, but should be considered as providing a broad possible interpretation of the claims by referring to the appended claims and taking into account the prior art, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

[0090] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention using the same means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A semi-automatic mask annotation method for masked areas of multi-layer stacked chips, characterized in that, Including the following: The input data of the target chip is obtained, including the original image and structured metadata, and the target chip is a multi-layer stacked chip; The input data is preprocessed to obtain preprocessed data; The processed data is then prompted and guided to obtain guided data; The final candidate mask is obtained by generating candidate masks based on the post-guidance data. Calculate the preset evaluation index of the final candidate mask, and perform weighted fusion of the preset evaluation index to obtain the quality score; Determine whether the quality score is higher than a preset threshold. If it is, determine that the final candidate mask is qualified and execute automatic storage. Otherwise, determine that the final candidate mask is unqualified and remove it. The final candidate mask and its input data that are automatically stored are then stored and their consistency is verified.

2. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 1, characterized in that, Specifically, the structured metadata includes, Machine number, batch number, process step, data acquisition timestamp, lens focal length and distortion coefficient, exposure / gain parameters, camera intrinsic and extrinsic parameters.

3. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 2, characterized in that, Specifically, the process of preprocessing the input data to obtain preprocessed data includes: Perform integrity checks and duplicate checks on the input data; Geometric correction and data augmentation are performed on the original image based on calibration parameters; Perform photometric normalization on the original image; If the area of ​​the original image exceeds the area threshold, large-area image slicing processing is performed to obtain the sliced ​​area; if the original image does not exceed the area threshold, it itself is used as its sliced ​​area. Generate a list of ROIs to be processed; The final preprocessed data includes normalized images, a list of ROIs, and structured metadata.

4. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 3, characterized in that, Specifically, the process of providing prompts and guidance to the processed data to obtain guided data includes, Generate a set of hints for each ROI in the ROI list. The cue set includes at least the bounding box and positive and negative points. If a coarse mask exists, it is included in the set. A center positive point cue is added for extremely small targets. Perform intelligent on-time sampling operation, perform distance transformation on the bonding wire targets within the ROI area, extract the skeleton centerline, and sample on-time points at fixed intervals on the centerline as strong cues for activating targets; Perform morphological edge negative point sampling operation, morphologically dilate the coarse edge within the bounding box of the bonding wire target in the ROI region to generate a narrow edge band, and sample negative points in the background region outside the narrow band as a strong constraint to suppress irrelevant bonding wires; Finally, the cue set is encoded and standardized, all cue coordinates are mapped to the model input scale, the data type format is unified, and it is encoded into a tensor that the basic segmentation model can directly receive. Together with the original image tensor, it is output to obtain the guided data.

5. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 1, characterized in that, Specifically, the process of generating the final candidate mask based on the post-guidance data includes: Initialize the disturbance generator parameters, set the spatial disturbance range, and set the probability threshold disturbance range; Perform multi-path parallel perturbation inference, apply K independent random perturbations to the guided data prompts, and generate a set containing K candidate masks and their corresponding confidence maps; Perform candidate mask clustering, calculate the cross-union ratio (IoU) between each pair of K candidate masks, merge masks with IoU greater than the cross-union threshold into the same cluster, and select the mask with the highest average confidence within the cluster as the center mask of the cluster. Calculate the cluster stability index. For each cluster, calculate the average IoU value of all masks in the cluster relative to the central mask, which is used as the stability index of the cluster. The comprehensive scoring is used to select the best, and discrete clusters with a stability index lower than 0.85 are removed. Then, the average confidence level and stability index of each cluster are read, and the comprehensive score is calculated by weighting the average confidence level and stability index according to the preset coefficients. The final candidate mask is obtained by selecting the cluster center mask with the highest comprehensive score.

6. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 5, characterized in that, Specifically, a preset evaluation index for the final candidate mask is calculated, and the preset evaluation index is weighted and fused to obtain a quality score, including: The confidence score is calculated as the internal confidence component of the final candidate mask. The mean pixel classification probability of the final candidate mask is extracted as the confidence score. The stability component, or stability score, is obtained by directly reading the stability index of the final candidate mask as the stability score. The boundary fit component, i.e. the boundary fit score, is calculated by performing Canny edge detection on the original image slice region to generate physical edge bands. The proportion of the mask boundary pixels of the final candidate mask falling within the physical edge band is calculated as the boundary fit score. Calculate the outward expansion control component, i.e., the outward expansion control score, and expand the outer frame outward by a preset fixed number of pixels to generate a safety buffer zone; Calculate the percentage of the final candidate mask that extends beyond the safety buffer zone, and use 1 - the percentage of the area as the outward expansion control score; Calculate the unified quality score Q by weighting and summing the four components according to their respective preset weights to obtain the normalized quality score Q.

7. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 6, characterized in that, Specifically, the final candidate mask and its input data that are automatically stored in the database are subjected to database storage and consistency verification, including: Establish a data index, create or update the dataset index file for the current round on the storage server, and assign a unique sample ID; Perform a spatial consistency check to examine the IoU between the current final candidate mask and the masks already in the database; If a spatial conflict is found, i.e., IoU > 0.9 but the categories are inconsistent, then the mask with the higher Q value is retained and the mask with the lower Q value is discarded. Perform a joint check of IoU and boundary consistency. If there is a conflict with the mask already in the database, i.e. IoU>0.9 but the category is inconsistent or the boundary difference is greater than the threshold, then the mask with the higher Q value is retained and the mask with the lower Q value is removed. If the Q values ​​are the same, the rule decision of user-processed (i.e., fine-tuned) is preferred over automatic rule and the index is updated. Update and save the data, save the qualified mask image to the specified directory, and write the associated metadata, including Q-value information, into the structured database.

8. The semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 1, characterized in that, The method also includes, After completing the data entry and consistency verification, iterative training with multi-paradigm fusion will be performed, specifically including: Construct a hybrid training set, extract manually labeled Anchor samples and high-quality pseudo-label samples from the current round of database entry, i.e., the final candidate mask and its input data Expansion, according to a preset ratio, and load the corresponding weak prompting information Guidance; Configure the optimized sampler, set the sampler parameters according to the statistical distribution of the solder wire width, and increase the sampling weight of narrow solder wire samples (i.e., solder wire widths lower than the preset width) to twice that of regular samples; Configure a hybrid loss function, setting it to a weighted sum of regional Dice Loss and boundary Boundary Loss, and setting the weight coefficient of Boundary Loss to 2.0; simultaneously enable point-hint-based consistency constraint loss. Perform fine-tuning training on the model, loading the model weights from the previous round, and conduct incremental training for 50-100 epochs on the mixed dataset. The best model is selected by evaluating the boundary F1 score of the model for each epoch on an independent validation set. The model with the highest score is saved as Best_Model_vN and deployed to the inference server.

9. A semi-automatic mask annotation method for masked areas of multi-layer stacked chips according to claim 8, characterized in that, The method also includes, The updated Best_Model_vN is used to process the new batch of unlabeled image data; Statistical observation indicators, calculate the observation acceptance rate of the current batch, and the ratio of qualified samples to total ROI; Perform online threshold updates and calculate the difference between the observed acceptance rate and the system's preset target acceptance rate; The difference and the preset step size are used to set the preset threshold for the next round, i.e., the admission threshold. Make minor adjustments and updates; Determine the termination condition and check whether the sample coverage of the current dataset has reached the preset target; If the target is not met, the next iteration will be automatically triggered; if the target is met, the annotation process will be terminated and a final report will be generated.

10. A semi-automatic mask labeling device for masked areas of multi-layer stacked chips, characterized in that, Including the following: The system comprises a sample testing platform, input / output interfaces, a vision system, an industrial computer, a network interface, and a storage server. The sample testing platform communicates bidirectionally with each module. The vision system is used for station triggering, exposure control, and image acquisition. The industrial computer executes the steps of the method described in any one of claims 1-9. The network interface facilitates data exchange with the enterprise network, training server, and external systems. The storage server stores datasets and models numbered by round, as well as running records. The input / output interfaces interact with the upper-level business system, and when centralized training is not available, the training server can be omitted, and retraining can be performed locally on the industrial computer.

Images