A dense micro-chip instance segmentation method of a category mixed mask fusion convolutional neural network

Abstract

This invention discloses a dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network, belonging to the field of chip instance segmentation and detection. First, a chip wafer optical image dataset is constructed, divided into a test set and a training set. Then, an initial model of the category-mixed mask fusion convolutional neural network is constructed. This initial model is trained using the established training set to obtain a final model. Finally, the final model is used to segment the constructed test set, completing the instance segmentation of the chip images. This dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network solves the problem of insufficient detection accuracy of existing detection technologies for industrial-grade dense microchip images. By strengthening the local features of each chip, it significantly improves the model's performance in category classification and fine localization.

Description

Technical Field

[0001] This invention relates to the field of chip instance segmentation and detection, specifically to a dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network. Background Technology

[0002] Microchips themselves contain limited information, and their dense density can interfere with boundary localization. Existing detection algorithms, namely object detection and instance segmentation, do not specifically address the impact of these dense microchip features, thus failing to meet the accuracy requirements for dense microchip detection.

[0003] The presence of dense, small target features presents a challenge for fusing deep and shallow features. Fusing low-level detail information helps the model retain sensitivity to small targets in high-level features. However, in order to extract as many features as possible, both low-level and deep features have a sufficient number of channels, even redundancy. This results in salient features being overemphasized, while weak local details are lost. The loss of weak local details can lead to missed detections and false detections of these targets.

[0004] Mainstream upsampling methods lack sufficient fidelity in restoring key details; their algorithms are simplistic, and for low-frequency pixel regions in images, excessive sampling points and complex sampling methods can easily waste computational resources; for high-frequency pixel regions, sparse sampling frequencies and simple sampling methods lead to image distortion and loss of true details. Dense, minute target features result in numerous high-frequency regions in the image, and the loss of these high-resolution details blurs the boundaries between different instances, leading to inaccurate target localization. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems by providing a dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network. By enhancing the local features of each chip, this method can significantly improve the model's performance in category classification and accurate localization.

[0006] The technical solution adopted in this invention is as follows:

[0007] A method for dense microchip instance segmentation using category-mixed mask fusion convolutional neural networks, comprising the following steps:

[0008] Step 1: Construct a dataset that includes optical images of chip wafers;

[0009] Step 2: Construct an initial model of a category-mixed mask fusion convolutional neural network, and train the initial model using the dataset constructed in Step 1 to obtain the final model of the category-mixed mask fusion convolutional neural network. The initial model includes a multi-scale feature extraction backbone, a category-mixed mask feature fusion network, and a segmentation head.

[0010] Step 3: The final model obtained in Step 2 is used to perform instance-level segmentation of the chip wafer images in the dataset. The multi-scale feature extraction backbone extracts feature maps of each scale of the optical image of the chip wafer. The extracted feature maps of each scale are fused through a class mixing mask feature fusion network. Finally, the fused feature maps are used to complete the instance-level segmentation of the chip image through the segmentation head.

[0011] Further, step 1, constructing the dataset, includes: acquiring optical images of chip wafers, obtaining the preset category to which the chips belong in the acquired optical images of chip wafers, labeling each category of chips in each acquired image according to the preset category to which the chips belong, completing the construction of the dataset, and dividing it into a test set and a training set.

[0012] Furthermore, in step 2, the initial model of the category-mixed mask fusion convolutional neural network is iteratively trained using the training set established in step 1 to obtain the final model of the category-mixed mask fusion convolutional neural network.

[0013] Furthermore, in step 3, the feature maps at various scales of the optical image of the chip wafer extracted from the backbone of scale feature extraction include:

[0014] The dataset images are first subjected to channel upscaling, and then the low-level detail information is extracted to obtain the low-level output feature map F. L Next, deep semantic information is extracted, and finally, channel dimensionality reduction is performed on the extracted deep semantic information to obtain the top-level global information, thus obtaining the top-level output feature map F with global information. H .

[0015] Furthermore, the feature fusion process of the category mixing mask feature fusion network module in step 3 includes:

[0016] Input low-level output feature map F L and top-level output feature map F H To category-mixed mask feature fusion network module;

[0017] Extracting the bottom-level output feature map F L and top-level output feature map F H The different categories of detailed features correspond to the detailed feature maps F divided by category. Lc and F H1 ;

[0018] Feature map FH1 With feature map F Lc Size alignment yields the aligned feature map F. H2 Feature map F H2 The top-level global information mask F for each category is generated by normalization. HC ;

[0019] Detail feature map F Lc Top-level global information mask F of various categories HC The hybrid mask image is obtained by multiplying each channel sequentially.

[0020] By restoring the channel count of the hybrid mask image, the output feature F of the category hybrid mask feature fusion network module is obtained. out .

[0021] Furthermore, extract the underlying output feature map F. L and top-level output feature map F H The different categories of detailed features include:

[0022] For the underlying output feature map F L The sampling layer fusion module extracts features, and then reduces the number of channels to match the number of categories, resulting in detailed feature maps F divided by category. Lc ;

[0023] For the top-level feature map F H Sampling and bottom-level output feature map F L The same method is used to obtain feature maps F of global information for different categories. H1 .

[0024] Furthermore, feature map F H2 The top-level global information mask F for each category is generated by normalization. HC In this process, the normalization method uses Layer Normalization (LN) to normalize the data according to the number of channels.

[0025] Furthermore, the feature map F H1 With feature map F Lc Size alignment is performed using a mixed region resolution alignment module.

[0026] Furthermore, the mixed region resolution alignment module includes:

[0027] Feature map F H1 With feature map F Lc As input to the hybrid region resolution alignment module, feature map F Lc After preprocessing and feature map F H1 Size alignment yields feature map F L1 Feature map F L1 With feature map FH1 F was selected using the key point selection strategy. H1 The key points are then upsampled using a multilayer perceptron (MLP), and the non-key points are upsampled using double-constrained differences to complete the resolution alignment of the mixed region.

[0028] Furthermore, the fused feature maps are used by the segmentation head to complete instance-level segmentation of the chip image, including:

[0029] The category mixing mask feature is fused with the output feature F of the network module. out The segmentation head performs instance segmentation of dense microchips. The segmentation head samples the SOLO Head and predicts the center and size of the target through class branch and mask branch respectively, thus completing instance segmentation of the chip image.

[0030] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0031] 1. In the dense microchip instance segmentation method of the category hybrid mask fusion convolutional neural network of the present invention, a category hybrid mask fusion convolutional neural network model is constructed. The deep features are used as attention and weighted to guide the low-level detailed information of each category. This coordinates the global information of the top layer and the low-level features that provide details and positional information, thus avoiding the target omission and false detection caused by the loss of weak local detailed information.

[0032] 2. In the dense microchip instance segmentation method of the category hybrid mask fusion convolutional neural network of the present invention, the category hybrid mask feature fusion network adopts a hybrid region resolution alignment module. The main function of the hybrid region resolution alignment module is to oversample the high-frequency pixel region, i.e. the key point, and undersample the low-frequency pixel region, i.e. the non-key point. This alleviates the problem of unclear boundary segmentation in the segmentation task and avoids the inaccurate target positioning caused by the distortion of the restored image. Attached Figure Description

[0033] Figure 1 This is a flowchart of a dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network according to the present invention;

[0034] Figure 2 shows chip surface images of the Micro-LED dataset in an embodiment of the present invention. (a) is one chip surface image, (b) is a surface image of some normal chip categories, (c) is a surface image of some defective chip categories, and (d) is a chip image after instance segmentation.

[0035] Figure 3 This is a schematic diagram of the network structure of a dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network according to the present invention.

[0036] Figure 4 This is a schematic diagram of the mixed region resolution alignment used in a dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network according to the present invention. Detailed Implementation

[0037] The present invention will now be described in detail with reference to the accompanying drawings.

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] This invention provides a method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network, such as... Figure 1 As shown, the specific method is implemented according to the following steps:

[0040] Step 1: Construct a dataset. Use an optical image acquisition device to acquire optical images of chip wafers. Use the chip manufacturing process to obtain the preset category to which the chip belongs. Label each category of chip in each image to construct a chip image dataset, and divide it into a test set and a training set.

[0041] In this embodiment, step 1 involves taking photos of the Micro-LED chip wafer using an optical image acquisition device. A total of 2755 optical images of the Micro-LED chip wafer are taken using the optical image acquisition device. The chip is classified into a preset category based on the chip manufacturing process. Each category of chip in each image is labeled, including normal chips, dirty chips, and poorly cut chips. The chip images are shown in Figure 2. A Micro-LED chip image dataset is constructed using the acquired optical images and divided into a test set and a training set. 2204 images are classified for the training set and 551 images are used for the test set. Five-fold cross-validation is used during training, and the image size is standardized to 640×640.

[0042] Step 2: Construct the initial model of the category-mixed mask fusion convolutional neural network. This model includes a multi-scale feature extraction backbone, a category-mixed mask feature fusion network, and a segmentation head. The multi-scale feature extraction backbone extracts features from the chip image to obtain feature maps of different scales. Shallow feature maps are used to capture local details and low-level features, while deep feature maps are used to extract high-level semantic features. These feature maps of different scales are fused in the category-mixed mask feature fusion network. This feature fusion network uses deep features as attention, weighting and guiding the low-level detail information of each category, thereby coordinating the top-level global information and the low-level features that provide detail and location information. Finally, the feature maps are used by the segmentation head to complete instance-level segmentation of the chip.

[0043] In this embodiment, the structural diagram of the dense microchip instance segmentation method using a category-mixed mask fusion convolutional neural network provided in step 2 is as follows: Figure 3 As shown, the chip includes a multi-scale feature extraction backbone, a category-mixed mask feature fusion network, and a segmentation head. Finally, the feature map is segmented by the segmentation head to complete the instance-level segmentation of the chip.

[0044] (1) Multi-scale feature extraction backbone:

[0045] In this embodiment, the multi-scale feature extraction backbone extracts features from the chip image to obtain feature maps of different depths. The shallow feature map is used to capture local details and low-level features of the image, while the deep feature map is used to extract high-level semantic features. The input image is first subjected to channel upscaling by two convolutional modules, and then the low-level detail information is extracted by three consecutive C3 modules to obtain the low-level output feature map F. L Then, deep semantic information is extracted through a combination of convolutional and C3 modules; finally, a convolutional module is used to perform channel dimensionality reduction on the extracted deep information to obtain the top-level global information, thereby obtaining the top-level output feature map F with global information. H As shown in Table 1:

[0046] Table 1. Detailed parameters of the multi-scale feature extraction backbone.

[0047]

[0048]

[0049] (2) Category Mixed Mask Feature Fusion Network:

[0050] In this embodiment, the category-mixed mask feature fusion network fuses feature maps of different scales. The feature fusion network effectively fuses local detail information of the lower layer at the top layer of the network, avoiding the loss of weak local details. By dividing the lower layer features and the top layer features by category, information dimensionality is reduced. Guided by the global information of the category, attention is paid to the detailed features of the corresponding category at the lower layer. This fusion method obtains the importance of the same pixel on different targets, avoiding false detections and false negatives caused by the loss of features of dense and small targets.

[0051] In this embodiment, the category-mixing mask feature fusion module uses deep features as attention, weighting and guiding the low-level detailed information of each category, thereby coordinating the top-level global information and the low-level features that provide detailed and positional information. This module has two inputs: the low-level output feature map F with extracted detailed information. L ∈R 640×640×256 and the top-level output feature map F containing global information H ∈R 80×80×256 .

[0052] Among them, for the bottom feature map F L To extract detailed features from different categories, two 3×3 convolutional CBR modules were used to extract features, and then a 1×1 convolutional kernel was used to reduce the number of channels to 3, where 3 represents the number of chip categories, resulting in detailed feature maps categorized by category. That is, each channel corresponds to a target category; then F Lc It can be derived from the following formula:

[0053] F Lc =PWconv 640×640×256---＞640×640×3 (CBR(CBR(F L ))) (1)

[0054] In the formula, PWconv represents a 1×1 point convolution;

[0055] Secondly, for the top-level feature map F H Utilizing the underlying feature map F L The same method is used to obtain global information for different categories. Right now:

[0056] F H1 =PWconv 80×80×256---＞80×80×3 (CBR(CBR(F H (2)

[0057] Next, the feature map F H1 After three upsampling operations, the feature map F is obtained. Lc Size alignment obtained This upsampling module employs a hybrid region resolution alignment module, F Lc It also participates in the mixed region resolution alignment module, as shown in equation (3):

[0058] F H2 =Upsample 80×80×3--→640×640×3 (F H1 F Lc (3)

[0059] In the formula, Upsample indicates that the region resolution alignment module has been used three times.

[0060] In this embodiment, in order to generate mask information corresponding to each category, feature map F H2It is necessary to highlight key information in each category channel and suppress other information; considering that Batch Normalization (BN) can cause an imbalance in mask feature information between categories, this embodiment uses Least Normalization (LN) for data normalization by channel; LeakyReLU is selected to generate mask information, which, compared with ReLU, retains the information contained in negative elements and achieves suppression through the coefficient α; mask information for each category The calculation is shown in equations (4) and (5):

[0061] F HC =LeakyReLU(LN(F H2 (4)

[0062]

[0063] For each category of detailed features F Lc and various types of top-level global information masks F HC The fusion is performed using a channel-by-channel multiplication method; F Lc and F HC The corresponding elements are multiplied together and then added together by channel to obtain the mixed mask image; the calculation process is shown in the following formula (6):

[0064]

[0065] Finally, two 3×3 convolutions are used to restore the number of channels of the hybrid mask feature map Mask to M, thus obtaining the output features of the category hybrid mask fusion module. Right now:

[0066] F out =Conv 640×640×1---＞640×640×256 (Mask) (7)

[0067] In this embodiment, the mixed region resolution alignment module is specifically described as follows:

[0068] The category-mixed mask feature fusion network employs a mixed region resolution alignment module. Currently, mainstream upsampling methods use the same sampling frequency and sampling method for the entire input image. However, for low-frequency regions in chip images, i.e., the background or the interior of the same object, excessive sampling points and complex sampling methods can easily waste computational resources. For high-frequency regions in chip images, i.e., the boundaries and internal features of objects, sparse sampling frequencies and simple sampling methods can lead to image distortion and loss of real details. To alleviate these issues, a mixed region resolution alignment module is proposed, which is an important step before the fusion of top-level and bottom-level features in the category-mixed mask fusion module. The main function of the mixed region resolution alignment module is to oversample high-frequency pixel regions (MLP method) and undersample low-frequency pixel regions (bilinear interpolation method). This alleviates the problem of unclear boundary segmentation in segmentation tasks, which is particularly important for the prediction of dense microchips.

[0069] In this embodiment, the main operation flow of the hybrid region resolution alignment module is as follows: Figure 4 As shown, the mixed region resolution alignment module has two inputs, namely... and Where F H1 It is the feature map to be upsampled, F Lc This provides detailed information for selecting key points, and the output feature map is... Feature map F Lc Preprocessing is required first, namely, 2×2 average pooling of the log2S layer, to achieve the same result as the feature map F. H1 Size alignment, to obtain F L1 With F H1 F was selected using the key point selection strategy. H1 For key points, MLP is used for upsampling; for non-key points, bilinear interpolation is used for upsampling. The specific calculation process of the mixed region resolution alignment module is as follows:

[0070] First step, select n key points:

[0071] First, it is necessary to select the points to be upsampled, which are called keypoints; keypoints are points in the high-frequency pixel region and have an important impact on the reconstruction of high-resolution feature images. L1 With F H1 Key information was fully considered:

[0072] F L1 Key information in the text:

[0073] F L1Representing the underlying details of an image, regions with large pixel gradients in an 80×80 dimension typically indicate boundaries between different instances of the same category and between different categories; these regions require more attention. Therefore, existing edge detection operators, such as using the Laplacian operator to extract image edges from the feature map, can be directly employed as part of the keypoint selection. Thus, F... L1 A little bit In F L1 Key information C rL The calculation of (x,y,z) is as follows:

[0074]

[0075] In the formula, <,> represents the inner product of two matrices.

[0076] F H1 Key information in the text:

[0077] F H1 Representing global information of the image, starting from dimension C, it signifies the degree of importance each category places on that point. When the difference in importance at a point is large, it means the feature map can clearly assign that point's features to a particular category. When the difference in importance is small and both are low, the point usually belongs to the background. When the difference in importance is small and both are high, it means the point has attracted attention in certain channels, easily leading to incorrect final classification. Therefore, we first consider F... H1 Perform normalization, then calculate F. H1 The weighted mean and variance along dimension C for each point; the point (x, y, z) in F H1 The expression for calculating the mean (x,y,z) in the equation is as follows:

[0078]

[0079] Variance σ 2 The expression for calculating (x, y, z) is as follows:

[0080]

[0081] Therefore, the point (x,y,z) is in F H1 Key information C rH (x,y,z) is calculated as follows:

[0082] C rH (x,y,z)=mean(x,y,z)-σ 2 (x, y, z) (11)

[0083] Therefore, the standard C for selecting key points r The calculation formula is:

[0084] C r =C rL +λ×C rH (12)

[0085] In the formula, λ is a weighting factor used to balance C. rL and C rH In this embodiment, λ = 0.2;

[0086] For F H1 Each channel in the middle, according to C r Sort the points and select the first n points as key points, resulting in a total of n×C key points;

[0087] In this embodiment, n is written as n = W / S × H / S × α, where α is generally less than or equal to 0.1.

[0088] The second step is to perform MLP prediction on the key points:

[0089] Keypoints are considered to be high-frequency pixel regions that need to be upsampled. These points contain potential important features about the object's boundaries and interior that are needed for segmentation. Simple sampling methods will cause information loss in these regions, and interpolation will cause distortion of high-resolution information. Therefore, these keypoints require more complex interpolation methods. MLP can introduce enough training parameters to learn the details of upsampling and reconstruction, and is recommended as the interpolation method for keypoints.

[0090] Upsampling requires inserting new data between a center point and its surrounding points. For each keypoint, three new points need to be generated; therefore, for each feature map, 3n points need to be predicted. An MLP is used to predict the interpolation points corresponding to the keypoints on each feature map, requiring a total of three independent MLP structures; each MLP is implemented using two linear layers.

[0091] The third step is to use bilinear interpolation on the remaining points:

[0092] The remaining non-critical points are considered to be low-frequency pixel regions. Closely related low-frequency pixel regions usually represent the same background information or internal information of the object. The simple upsampling method does not cause information distortion, and at the same time, it can ensure high-resolution features while reducing the overhead of computing resources. Therefore, the remaining non-critical points are upsampled and supplemented using a simple bilinear interpolation method.

[0093] (3) Segmentation Head

[0094] In this embodiment, the output feature F of the category mixing mask fusion module outThe SOLO Head is selected as the segmentation head for the dense microchip segmentation framework. It predicts the center and size of the target through class branch and mask branch respectively, thereby achieving instance-level segmentation. In addition, the loss function corresponding to the SOLO Head is also selected as the loss function of this framework.

[0095] Furthermore, after establishing the initial model of the category-mixing mask fusion convolutional neural network, the model needs to be trained. In this embodiment, the training process is as follows:

[0096] In the PyTorch environment, the Micro-LED chip training set images constructed in step 1 of this embodiment are randomly divided into batches of 64 each, resulting in 34 batches. The last batch contains 28 images. These images are then sequentially input into the initial model of the category-mixed mask fusion convolutional neural network constructed in step 2 of this embodiment for iterative training. After all samples in a batch have been calculated, the loss value is calculated using the loss function, and the weights are updated once using the backpropagation mechanism. After all batches have been updated, the next iteration is performed. The maximum number of iterations for the training samples is 200. After the iterations are completed, the final model of the category-mixed mask fusion convolutional neural network is obtained.

[0097] Step 3: Input the test set images constructed in Step 1 into the final model of the category mixing mask fusion convolutional neural network. After passing through the multi-scale feature extraction backbone, the category mixing mask feature fusion network and the segmentation head, the instance segmentation of the chip image is completed.

[0098] In this embodiment, the Micro-LED chip test set image constructed in step 1 is input into the final model of the category hybrid mask fusion convolutional neural network. After the multi-scale feature extraction backbone, the category hybrid mask feature fusion network and the segmentation head, the instance segmentation of the Micro-LED chip image is completed, and the segmentation results of various types in Table 2 are obtained. The results are shown in Figure 2(d). The accuracy is above 90%, and the effect is obvious.

[0099] Table 2 Segmentation Effects of Mini / Micro LED Chips by Category

[0100] quantity 120684 3713 675 Pixel accuracy 97.8％ 93.6％ 92.4％

[0101] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network, characterized in that, The method includes: Step 1, construct a dataset, which includes optical images of chip wafers; Step 2: Construct an initial model of a category-mixed mask fusion convolutional neural network, and train the model using the dataset constructed in Step 1 to obtain the final model of the category-mixed mask fusion convolutional neural network. The model includes a multi-scale feature extraction backbone, a category-mixed mask feature fusion network, and a segmentation head. Step 3: Using the final model obtained in Step 2, perform instance-level segmentation on the chip wafer images in the dataset. The multi-scale feature extraction backbone extracts feature maps at various scales from the optical images of the chip wafers. The dataset images are first subjected to channel upscaling, and then the low-level detail information is extracted to obtain the low-level output feature map. Next, deep semantic information is extracted, and finally, channel dimensionality reduction is performed on the extracted deep semantic information to obtain the top-level global information, thus obtaining the top-level output feature map with global information. ; The extracted feature maps at various scales are fused using a category-mixing mask feature fusion network, with the input being the bottom layer and the output being the feature map. and top-level output feature map To category-mixed mask feature fusion network module; Extracting the underlying output feature map and top-level output feature map Different categories of detailed features, for the underlying output feature map The sampling layer fusion module extracts features, and then reduces the number of channels to match the number of categories, resulting in detailed feature maps divided by category. For top-level feature maps Using the underlying output feature map Feature maps of global information for different categories are obtained using the same method. ; feature map With feature map Size alignment yields aligned feature maps. Feature map Generate top-level global information masks for each category through normalization. ; detail feature map Top-level global information masks of various categories The hybrid mask image is obtained by multiplying each channel sequentially. ; Mixed mask image The channel count is restored to obtain the output features of the category mixing mask feature fusion network module. ; Finally, the fused feature map is used by the segmentation head to complete instance-level segmentation of the chip image.

2. The method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network as described in claim 1, characterized in that, Step 1, constructing the dataset, includes: acquiring optical images of chip wafers, obtaining the preset category to which the chips belong in the acquired optical images of chip wafers, labeling each category of chips in each acquired image according to the preset category to which the chips belong, completing the construction of the dataset, and dividing it into a test set and a training set.

3. The method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network as described in claim 2, characterized in that, In step 2, the initial model of the category-mixed mask fusion convolutional neural network is iteratively trained using the training set established in step 1 to obtain the final model of the category-mixed mask fusion convolutional neural network.

4. The method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network as described in claim 1, characterized in that, The feature map Generate top-level global information masks for each category through normalization. In this process, the normalization method uses Layer Normalization (LN) to normalize the data according to the number of channels.

5. The method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network as described in claim 1, characterized in that, The feature map With feature map Size alignment is performed using a mixed region resolution alignment module.

6. The method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network as described in claim 5, characterized in that, The hybrid region resolution alignment module includes: Feature map With feature map The feature map serves as input to the hybrid region resolution alignment module. After preprocessing and feature mapping Size alignment yields feature maps. Feature map With feature map Selected through key point selection strategy The key points are then upsampled using a multilayer perceptron (MLP), while non-key points are upsampled using double-constraint interpolation to achieve resolution alignment of the mixed region.

7. The method for dense microchip instance segmentation using a category-mixed mask fusion convolutional neural network as described in claim 1, characterized in that, The step of performing instance-level segmentation of the chip image using the fused feature map through a segmentation head includes: Fuse category mixing mask features with the output features of the network module The segmentation head is used to perform instance segmentation of dense microchips. The segmentation head adopts SOLO Head, which predicts the center and size of the target through class branch and mask branch respectively, thus completing the instance segmentation of the chip image.

Images