Gallium nitride radio frequency device defect detection method and system based on deep learning

Abstract

The invention provides a gallium nitride radio frequency device defect detection method and system based on deep learning, and relates to the technical field of gallium nitride devices, and the method comprises the steps: generating and preprocessing a multispectral image data set; performing channel and space two-dimensional attention calculation, and performing feature optimization in combination with deformable convolution and residual connection; constructing a multi-scale feature pyramid, and constructing a multi-level classification tree based on feature similarity to perform fine-grained defect classification; and utilizing the deep neural network model to evaluate the influence degree of the defect on the device performance. According to the method, the defect position and type of the gallium nitride radio frequency device can be accurately identified, the performance influence of the defect is quantitatively evaluated, and the detection precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to gallium nitride device technology, and in particular to a gallium nitride radio frequency device defect detection method and system based on deep learning. Background Art

[0002] Gallium nitride (GaN) RF devices are widely used in high-frequency communications, radar, and other fields due to their excellent electrical performance and thermal stability. During the GaN device manufacturing process, various defects are easily generated due to factors such as process and materials. These defects can significantly affect device performance and reliability. Therefore, developing efficient and accurate defect detection methods is crucial to ensuring the quality and performance of GaN RF devices.

[0003] Traditional defect detection for GaN devices relies primarily on manual observation and simple image processing algorithms. With the development of deep learning technology, deep learning-based defect detection methods have gradually become a research hotspot. Current detection methods primarily analyze single spectral images and use conventional convolutional neural network architectures for feature extraction and classification.

[0004] A single spectral image is difficult to fully reflect the characteristic information of GaN device defects, which makes it difficult to accurately identify certain hidden defects and limits the comprehensiveness and accuracy of detection.

[0005] Traditional convolutional neural network structures have limited feature extraction capabilities for irregular shapes and multi-scale defects, and are unable to effectively capture the detailed features of defects, especially when dealing with defects with complex shapes and large size differences.

[0006] Existing defect detection methods often only focus on defect identification and classification, lack in-depth analysis of the correlation between defects and device performance, and are unable to accurately assess the actual impact of defects on device performance, which limits the guiding value of detection results in actual production applications. Summary of the Invention

[0007] The embodiments of the present invention provide a gallium nitride radio frequency device defect detection method and system based on deep learning, which can solve the problems in the prior art.

[0008] A first aspect of an embodiment of the present invention provides a gallium nitride radio frequency device defect detection method based on deep learning, comprising: generating a multispectral image dataset based on image data of the gallium nitride radio frequency device, performing image registration, image enhancement, and noise removal on the multispectral image dataset to generate a preprocessed standardized multispectral image; Performing channel and spatial dual-dimensional attention calculations on the standardized multispectral image to obtain feature weights, using deformable convolution to align and fuse features of different scales, combining the residual connection mechanism to perform feature recalibration, and outputting optimized feature representations; Constructing a multi-scale feature pyramid based on the optimized feature representation and performing feature fusion, performing binary classification on the fused features to obtain preliminary defect areas, constructing a multi-level classification tree based on the feature similarity of the preliminary defect areas, and setting an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification; The results of the fine-grained defect classification are input into a deep neural network model. Based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters, the deep neural network model classifies the detected defects and evaluates their impact on device performance, and generates a detection report including the defect location, type, size and performance impact.

[0009] The standardized multispectral image is subjected to channel and spatial dual-dimensional attention calculation to obtain feature weights. Deformable convolution is used to align and fuse features of different scales. The residual connection mechanism is combined to perform feature recalibration. The output optimized feature representation includes: Performing global average pooling and global maximum pooling operations on the input feature map to obtain a channel description vector, inputting the channel description vector into a shared multi-layer perceptron network for nonlinear feature transformation, and fusing it through a gating mechanism to generate a channel attention weight; Applying a multi-directional Sobel operator to the input feature map to perform edge gradient extraction, generating edge enhancement features based on adaptive threshold segmentation, and fusing the edge enhancement features with the multi-scale features of the input feature map to generate spatial attention weights; Performing a multi-scale pooling operation on the input feature map to construct a feature pyramid structure, aligning each scale feature in the feature pyramid structure through a deformable convolutional network, and weighted fusion of the aligned multi-scale features based on an adaptive weight coefficient to obtain a multi-scale fused feature; The channel attention weight is element-wise multiplied by the input feature map to obtain a channel calibration feature, the channel calibration feature is element-wise multiplied by the spatial attention weight to obtain a spatial calibration feature, the spatial calibration feature is feature-fused with the multi-scale fusion feature to obtain a fusion feature; the input feature map and the fusion feature are residually connected and added to output a recalibrated feature.

[0010] Applying a multi-directional Sobel operator to the input feature map to extract edge gradients, generating edge enhancement features based on adaptive threshold segmentation, and fusing the edge enhancement features with the multi-scale features of the input feature map to generate spatial attention weights, including: Applying convolution operations to the input feature map using Sobel operators in the horizontal, vertical, forty-five-degree, and one hundred and thirty-five-degree directions to obtain a multi-directional edge gradient map, and performing amplitude superposition and direction encoding on the multi-directional edge gradient map to generate an edge response feature map; Calculating an adaptive threshold based on the edge response feature map, wherein the adaptive threshold is determined by the mean, standard deviation, and learnable weight parameters of the feature map, and performing dynamic threshold segmentation on the edge response feature map according to the adaptive threshold to obtain an edge enhancement feature map; Inputting the edge enhancement feature map into the convolution layer for feature transformation, generating an edge attention weight map through the sigmoid function of the feature map after the feature transformation, and adaptively fusing the edge attention weight map with the original spatial attention map to obtain a fused attention map; Performing multi-scale decomposition on the input feature map to obtain feature maps of different scales, and performing feature reconstruction on the feature maps of different scales after adaptively weighting with learnable weights and the fused attention map to generate a boundary enhanced feature map; The boundary enhancement feature map is residually connected with the edge enhancement feature map, and the contribution of the residual feature is controlled by a learnable enhancement coefficient to output the spatial attention weight of the fused boundary perception.

[0011] The optimized feature representation is used to construct a multi-scale feature pyramid and perform feature fusion, the fused features are subjected to binary classification to obtain a preliminary defect area, a multi-level classification tree is constructed based on the feature similarity of the preliminary defect area, and an adaptive classification threshold is set at each level of the multi-level classification tree to perform fine-grained defect classification, including: constructing a multi-scale feature pyramid by performing an adaptive pooling operation on the optimized feature representation, performing upsampling and downsampling fusion on feature maps of adjacent scale layers in the multi-scale feature pyramid, and performing adaptive weighted combination on the fused multi-scale features based on learnable weights to obtain fused features; Performing preliminary defect recognition on the fused features through a binary classification discriminant network, dividing a preliminary defect area based on a comparison result between a probability value output by the binary classification discriminant network and a preset discrimination threshold, and extracting a deep feature representation of the preliminary defect area; Calculating a similarity matrix between the deep feature representations of the preliminary defect areas, and constructing a hierarchical classification tree based on the similarity matrix, wherein each level of the hierarchical classification tree corresponds to a defect classification division of different granularity; The classification confidence is calculated at each level of the hierarchical classification tree, an adaptive classification threshold is set according to the historical statistical distribution of the classification confidence, and the deep feature representation at each level is compared with the corresponding adaptive classification threshold to achieve fine-grained defect classification.

[0012] Calculating classification confidence at each level of the hierarchical classification tree, setting an adaptive classification threshold based on the historical statistical distribution of the classification confidence, and comparing the deep feature representation at each level with the corresponding adaptive classification threshold to achieve fine-grained defect classification includes: Calculating classification confidence at each level of the hierarchical classification tree, wherein the classification confidence is obtained by calculating the Euclidean distance between the deep feature representation and the center of each category feature at the level, and dividing the Euclidean distance by the feature space standard deviation; A statistical distribution model is constructed for the historical classification confidence of each level, and the statistical distribution model is converted into a continuous probability density function through Gaussian kernel density estimation. Based on the value of the continuous probability density function at the preset quantile and the classification threshold of the level at the previous moment, an exponential sliding average method is used to update the adaptive classification threshold; At each level of the hierarchical classification tree, the deep feature representation is transformed by a learnable nonlinear mapping function, a difference between the transformed feature representation and the adaptive classification threshold of the level is calculated, and whether to enter the next level of classification is determined according to the difference; Based on the category template library of this level, the cosine similarity between the deep feature representation and each subdivided category template is calculated, the category with the highest cosine similarity is determined as the classification result of this level, and the confidence of the classification result is updated to the historical statistical distribution for subsequent dynamic adjustment of the threshold.

[0013] The results of the fine-grained defect classification are input into a deep neural network model. The deep neural network model classifies the detected defects and evaluates their impact on device performance based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters. The model generates a test report containing the defect location, type, size and performance impact, including: Extracting morphological features, spectral features, and size information of the defect area from the result of the fine-grained defect classification, fusing the morphological features, spectral features, and size information through a multi-layer feature fusion network to obtain a defect feature vector, and extracting electrical performance parameters from device test data to construct a performance index vector; Inputting the defect feature vector and the performance index vector into a deep mapping network, the deep mapping network establishes a correspondence between defect features and device performance through multi-layer nonlinear transformation, and optimizing the parameters of the deep mapping network based on historical sample data to obtain a performance mapping model; Calculating the influence weight of each defect region on the device performance using the performance mapping model, combining the influence weight with the location coordinates, type label, and size parameters of the defect to generate a defect feature descriptor, and performing quantitative analysis on the defect feature descriptor using a performance evaluation network; A standardized test report is generated based on the results of the quantitative analysis, and the test report includes a spatial distribution diagram of the defects, feature description information, and performance impact assessment results.

[0014] A second aspect of an embodiment of the present invention provides a gallium nitride radio frequency device defect detection system based on deep learning, comprising: The first unit is configured to generate a multispectral image dataset based on image data of the gallium nitride radio frequency device, perform image registration, image enhancement, and noise removal on the multispectral image dataset, and generate a preprocessed standardized multispectral image; The second unit is used to perform channel and spatial dual-dimensional attention calculation on the standardized multispectral image to obtain feature weights, use deformable convolution to align and fuse features of different scales, combine the residual connection mechanism to perform feature recalibration, and output the optimized feature representation; The third unit is used to construct a multi-scale feature pyramid based on the optimized feature representation and perform feature fusion, perform binary classification on the fused features to obtain preliminary defect areas, construct a multi-level classification tree based on the feature similarity of the preliminary defect areas, and set an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification; The fourth unit is used to input the results of the fine-grained defect classification into a deep neural network model. The deep neural network model classifies the detected defects and evaluates their impact on device performance based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters, and generates a detection report including the defect location, type, size and performance impact.

[0015] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0016] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0017] The beneficial effects of this application are as follows: The present invention improves image quality and data consistency through preprocessing and standardization of multispectral image data, lays a good foundation for subsequent defect detection, and effectively enhances the accuracy and reliability of detection.

[0018] The present invention adopts a dual-dimensional attention mechanism of channels and spaces and deformable convolution for feature extraction and fusion, combined with residual connection for feature recalibration, which can better capture the detailed features of defects, enhance the model's ability to recognize different types of defects, and improve the accuracy of detection.

[0019] The present invention achieves fine-grained defect classification by constructing a multi-level classification tree and setting adaptive classification thresholds, and combines deep neural networks to establish a correlation analysis between defect characteristics and device performance. This can not only accurately identify defect types, but also evaluate the degree of impact of defects on device performance, providing a comprehensive analytical basis for device quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of a deep learning-based gallium nitride radio frequency device defect detection method according to an embodiment of the present invention; Figure 2 This is a bar chart comparing and analyzing the defect detection performance of the deep neural network model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0022] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0023] Figure 1 FIG is a flow chart of a gallium nitride radio frequency device defect detection method based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the method includes: generating a multispectral image dataset based on image data of the gallium nitride radio frequency device, performing image registration, image enhancement, and noise removal on the multispectral image dataset to generate a preprocessed standardized multispectral image; Performing channel and spatial dual-dimensional attention calculations on the standardized multispectral image to obtain feature weights, using deformable convolution to align and fuse features of different scales, combining the residual connection mechanism to perform feature recalibration, and outputting optimized feature representations; Constructing a multi-scale feature pyramid based on the optimized feature representation and performing feature fusion, performing binary classification on the fused features to obtain preliminary defect areas, constructing a multi-level classification tree based on the feature similarity of the preliminary defect areas, and setting an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification; The results of the fine-grained defect classification are input into a deep neural network model. Based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters, the deep neural network model classifies the detected defects and evaluates their impact on device performance, and generates a detection report including the defect location, type, size and performance impact.

[0024] In an optional embodiment, channel and spatial dual-dimensional attention calculations are performed on the standardized multispectral image to obtain feature weights, deformable convolution is used to align and fuse features of different scales, and a residual connection mechanism is combined to perform feature recalibration. The output optimized feature representation includes: Performing global average pooling and global maximum pooling operations on the input feature map to obtain a channel description vector, inputting the channel description vector into a shared multi-layer perceptron network for nonlinear feature transformation, and fusing it through a gating mechanism to generate a channel attention weight; Applying a multi-directional Sobel operator to the input feature map to perform edge gradient extraction, generating edge enhancement features based on adaptive threshold segmentation, and fusing the edge enhancement features with the multi-scale features of the input feature map to generate spatial attention weights; Performing a multi-scale pooling operation on the input feature map to construct a feature pyramid structure, aligning each scale feature in the feature pyramid structure through a deformable convolutional network, and weighted fusion of the aligned multi-scale features based on an adaptive weight coefficient to obtain a multi-scale fused feature; The channel attention weight is element-wise multiplied by the input feature map to obtain a channel calibration feature, the channel calibration feature is element-wise multiplied by the spatial attention weight to obtain a spatial calibration feature, the spatial calibration feature is feature-fused with the multi-scale fusion feature to obtain a fusion feature; the input feature map and the fusion feature are residually connected and added to output a recalibrated feature.

[0025] Perform global average pooling and global maximum pooling operations on the input feature map to obtain a channel description vector. For the input standardized multispectral image feature map F, its dimension is C×H×W, where C represents the number of channels, H and W represent the height and width of the feature map respectively. The global average pooling operation averages all pixel values ​​of each channel c to generate a channel description vector F of size C×1×1. avg Specifically, for the c-th channel in the feature map F, the average value of the pixel values ​​F(c,i,j) at all positions (i,j) in the channel is calculated to obtain F avg (c). Similarly, the global maximum pooling operation selects the maximum value in each channel to obtain the channel description vector F max In practical applications, for multispectral images of GaN devices with C=64, the average value of each channel in the channel description vector obtained after global average pooling is usually in the range of [-0.25, 0.35], while the maximum value of each channel in the vector obtained by global maximum pooling is usually in the range of [0.5, 0.9].

[0026] The channel description vector is input into the shared multi-layer perceptron network for nonlinear feature transformation. avg and F max Each is processed by a shared two-layer perceptron network. The network first reduces the C-dimensional vector to C / r dimensions (where r is the dimensionality reduction ratio, usually 16) through a fully connected layer, then introduces nonlinearity through the ReLU activation function, and then restores it to C dimensions through another fully connected layer. Taking C=64 and r=16 as an example, the first fully connected layer reduces the 64-dimensional vector to 4 dimensions. After ReLU activation, the second fully connected layer maps it back to 64 dimensions, resulting in two C-dimensional vectors F avg mlp and F max mlp .

[0027] Generate channel attention weights through gating mechanism fusion. avg mlp and F max mlp Add and normalize through Sigmoid activation function to get the channel attention weight M in the range [0,1] c Taking C=64 as an example, the final channel attention weight M c It is a 64×1×1 vector, where the weight of important channels is usually above 0.7, while the weight of unimportant channels is as low as 0.2.

[0028] For the spatial attention part, we first apply a multi-directional Sobel operator to the input feature map to extract edge gradients. Specifically, we use the horizontal Sobel operator [-1, 0, 1; -2, 0, 2; -1, 0, 1] and the vertical Sobel operator [-1, -2, -1; 0, 0, 0; 1, 2, 1] to perform convolution operations on each channel of the feature map, and obtain the horizontal gradient map G. h and vertical gradient map G v Then calculate the gradient magnitude map G mag , where the value at each position is the square root of the sum of the squares of the corresponding horizontal and vertical gradients. In practical applications, for a 640×480 resolution GaN device image, the gradient amplitude in the edge region is typically above 0.4, while the gradient amplitude in the smooth region is typically below 0.1.

[0029] Generate edge enhancement features based on adaptive threshold segmentation. Otsu adaptive threshold algorithm is used to segment the gradient amplitude map G mag Perform binarization processing to obtain the edge binary map E bin The algorithm automatically calculates the optimal threshold value T (usually between 0.15 and 0.25) and sets G mag The area greater than T in E is marked as 1 (edge), and the area less than T is marked as 0 (non-edge). In order to enhance the expression ability of edge features, morphological operations are used to bin Perform refinement and connection processing to obtain the edge enhancement feature map E enh .

[0030] The edge enhancement features are fused with the multi-scale features of the input feature map to generate the spatial attention weights. Specifically, the input feature map F is first subjected to channel-wise maximum pooling and average pooling operations to obtain two H×W feature maps F c max and F c avg Then, F c max 、F c avg and edge enhancement feature map E enh Splicing in the channel dimension to obtain a 3×H×W feature map F cat Then use the 7×7 convolution layer to F cat Processing is performed to obtain a 1×H×W feature map, and then the spatial attention weight map M is generated through the Sigmoid activation function s In practical applications, for defect areas, M s The value of is usually above 0.8, while the value of non-defective areas is usually below 0.4.

[0031] In order to achieve multi-scale feature fusion, the input feature map is subjected to multi-scale pooling operations to construct a feature pyramid structure. Specifically, the input feature map F is sequentially applied with the maximum pooling operation, with step sizes of 1, 2, and 4, to generate three feature maps of different scales F. s1 (original size), F s2 (half size) and F s4 (The size is reduced to 1 / 4 of the original.) For the original image of 640×480, the corresponding feature map sizes generated are 160×120, 80×60, and 40×30 respectively.

[0032] Each scale feature in the feature pyramid structure is aligned through a deformable convolution network. For each scale feature map, a 3×3 deformable convolution is used for processing. Deformable convolution differs from standard convolution in that it allows the convolution kernel to dynamically adjust the sampling position based on the input features. Specifically, an additional convolution layer is first used to predict the offset of each sampling point. These offsets are added to the regular grid sampling points to form new sampling positions. Convolution operations are then performed on these adjusted positions. While the standard 3×3 convolution has 9 fixed sampling points, the deformable convolution can flexibly adjust the positions of these 9 points, with the offset range typically between [-1, 1].

[0033] The aligned multi-scale features are weighted fused based on the adaptive weight coefficient to obtain the multi-scale fusion feature. First, the feature maps of different scales are adjusted to the same size (usually the original feature map size) through bilinear interpolation to obtain F s1 align 、F s2 align and F s4 align The importance weight w of each scale feature is then adaptively calculated through the attention mechanism s1 、w s2 and w s4 , ensure that w s1 +w s2 +w s4 = 1. In practical applications, for areas with rich details, w s1 It is usually larger (about 0.5-0.6), and for smooth textured areas, w s4 Large (about 0.4-0.5). Finally, the three feature maps are weighted and summed according to the weights to obtain the multi-scale fusion feature F multi .

[0034] The channel attention weight M c Multiply element-wise with the input feature map F to obtain the channel calibration feature F c refine Specifically, for each position (i, j) and each channel c in the feature map F, F c refine (c,i,j) = F(c,i,j) × M c(c) Channel calibration characteristics F c refine Then with the spatial attention weight M s Perform element-by-element multiplication to obtain the spatial calibration feature F s refine , that is, F s refine (c,i,j) = F c refine (c,i,j) × M s (i,j).

[0035] Spatial calibration feature F s refine and multi-scale fusion feature F multi Perform feature fusion to obtain fusion feature F fused The specific fusion method is to directly add the two: F fused = F s refine + F multi Finally, the input feature map F is combined with the fusion feature F fused Perform residual connection addition and output recalibration feature F out = F + F fused .

[0036] In an optional embodiment, applying a multi-directional Sobel operator to the input feature map to perform edge gradient extraction, generating edge enhancement features based on adaptive threshold segmentation, and fusing the edge enhancement features with the multi-scale features of the input feature map to generate spatial attention weights includes: Applying convolution operations to the input feature map using Sobel operators in the horizontal, vertical, forty-five-degree, and one hundred and thirty-five-degree directions to obtain a multi-directional edge gradient map, and performing amplitude superposition and direction encoding on the multi-directional edge gradient map to generate an edge response feature map; Calculating an adaptive threshold based on the edge response feature map, wherein the adaptive threshold is determined by the mean, standard deviation, and learnable weight parameters of the feature map, and performing dynamic threshold segmentation on the edge response feature map according to the adaptive threshold to obtain an edge enhancement feature map; Inputting the edge enhancement feature map into the convolution layer for feature transformation, generating an edge attention weight map through the sigmoid function of the feature map after the feature transformation, and adaptively fusing the edge attention weight map with the original spatial attention map to obtain a fused attention map; Performing multi-scale decomposition on the input feature map to obtain feature maps of different scales, and performing feature reconstruction on the feature maps of different scales after adaptively weighting with learnable weights and the fused attention map to generate a boundary enhanced feature map; The boundary enhancement feature map is residually connected with the edge enhancement feature map, and the contribution of the residual feature is controlled by a learnable enhancement coefficient to output the spatial attention weight of the fused boundary perception.

[0037] As shown in Table 1, the method further includes:

[0038] The input feature map is convolved with Sobel operators in the horizontal, vertical, forty-five-degree, and one hundred and thirty-five-degree directions to obtain a multi-directional edge gradient map, which is amplitude superimposed and direction-encoded to generate an edge response feature map.

[0039] We need to obtain edge information from the input feature map. We choose to use the Sobel operator because it is a classic edge detection operator that can effectively extract horizontal, vertical, and diagonal edges of an image. Specifically, we convolve the input feature map with the Sobel operator in four different directions: Horizontal Sobel operator: This operator is used to detect horizontal edges in an image, that is, vertical gradient changes. For example, on a 256x256 pixel feature map, the horizontal Sobel operator can highlight areas in the image where the grayscale value changes significantly in the vertical direction, such as the left and right boundaries of an object.

[0040] Vertical Sobel operator: This operator is used to detect vertical edges in an image, that is, horizontal gradient changes. For example, on the same 256x256 pixel feature map, the vertical Sobel operator can highlight areas in the image where the horizontal grayscale value changes significantly, such as the upper and lower boundaries of objects.

[0041] 45-degree Sobel operator: This operator detects edges along a 45-degree angle in an image. It can detect slanted edges that are neither completely horizontal nor completely vertical. For example, when processing images of buildings, this operator can effectively detect the slanted edges of rooftops.

[0042] 135-degree Sobel operator: This operator is used to detect edges along the 135-degree direction in an image. Similar to the 45-degree Sobel operator, it can capture oblique edges in the opposite direction, further improving the extraction of edge information.

[0043] After performing the convolution operation in these four directions, four edge gradient maps are generated, each representing the edge strength of the image in different directions. These edge gradient maps need to be fused to generate a comprehensive edge response feature map. This fusion method first calculates the gradient magnitude of each pixel in the four directions and then superimposes these magnitudes. The gradient magnitude represents the strength of the edge at that pixel; a larger magnitude indicates a stronger edge probability. This superposition operation integrates edge information from different directions to form a more complete edge response map. To preserve the edge directional information, each pixel is directional encoded and the gradient direction information is also incorporated into the edge response feature map. For example, an angle range (0-360 degrees) can be used to represent the gradient direction of each pixel, and this angle value is then encoded into the feature map.

[0044] An adaptive threshold is calculated based on the edge response feature map, where the adaptive threshold is determined by the mean, standard deviation and learnable weight parameters of the feature map. Dynamic threshold segmentation is performed on the edge response feature map according to the adaptive threshold to obtain an edge enhancement feature map.

[0045] Due to factors such as image complexity and lighting variations, edge strength can vary significantly across different regions. Using a fixed threshold for edge segmentation can easily lead to edge loss or over-segmentation. To address this issue, adaptive threshold segmentation is employed. This method dynamically adjusts the threshold based on local image features, better adapting to image changes.

[0046] Calculate the mean and standard deviation of the edge response feature map. The mean represents the overall brightness level of the image, and the standard deviation represents the contrast of the image. Then, use these two statistics and some learnable weight parameters to calculate the adaptive threshold. The learnable weight parameters can be adjusted according to different data sets and tasks to achieve better performance. For example, for an image with darker lighting, we can lower the threshold by adjusting the weight parameters to avoid edge loss. For an image with lower contrast, we can increase the threshold by adjusting the weight parameters to avoid over-segmentation. The adaptive threshold is calculated as follows: Adaptive threshold = mean + (standard deviation × learnable weight parameter); After obtaining the adaptive threshold, it is applied to the edge response feature map for dynamic threshold segmentation. Specifically, for each pixel, if its edge response value is greater than the adaptive threshold, it is set to 1, otherwise it is set to 0. This results in a binary edge enhancement feature map, where pixels with a value of 1 represent edges and pixels with a value of 0 represent non-edges.

[0047] The edge enhancement feature map is input into the convolution layer for feature transformation, the feature map after feature transformation generates an edge attention weight map through a sigmoid function, and the edge attention weight map is adaptively fused with the original spatial attention map to obtain a fused attention map.

[0048] To further extract information from the edge-enhanced feature map, it is fed into a convolutional layer for feature transformation. Convolutional layers can learn more complex edge patterns, thereby improving edge detection accuracy. For example, convolutional layers can learn edges of different shapes, orientations, and scales. After processing by the convolutional layer, we obtain a new feature map that contains richer edge information.

[0049] This feature map is passed through a sigmoid function, mapping its values ​​to a range between 0 and 1. The sigmoid function can map any real number to a range between 0 and 1, making it ideal for generating attention weights. After processing with the sigmoid function, an edge attention weight map is generated, where the value of each pixel represents the degree of attention paid to the edge. Values ​​closer to 1 indicate a more important pixel; values ​​closer to 0 indicate a less important pixel.

[0050] The edge attention weight map is adaptively fused with the original spatial attention map to obtain a fused attention map. Adaptive fusion can combine edge information and original spatial information to generate more accurate attention weights. For example, if a region is both an edge region and an important region in the original spatial attention map, then the weight of this region will be higher in the fused attention map. The adaptive fusion method is as follows: Fused Attention Map = (Edge Attention Weight Map × Learnable Fusion Weight) + (Original Spatial Attention Map × (1 - Learnable Fusion Weight)) The learnable fusion weight is a parameter between 0 and 1 that controls the contribution ratio of the edge attention weight map to the original spatial attention map. This parameter can be learned through training to achieve the best fusion effect.

[0051] The input feature map is decomposed into multi-scale features to obtain feature maps of different scales. The feature maps of different scales are adaptively weighted with learnable weights and then feature reconstructed with the fused attention map to generate a boundary enhanced feature map.

[0052] To better utilize the multi-scale information of an image, the input feature map is decomposed into multiple scales to obtain feature maps of different scales. Multi-scale decomposition can be achieved in a variety of ways, such as using an image pyramid or wavelet transform. In this solution, we choose to use an image pyramid because it is simple to implement and has good performance.

[0053] Downsample the input feature map to obtain a feature map of half its size. Downsample this feature map again to obtain a feature map of further reduced size. Repeat this process until the desired number of scales is achieved. For example, the input feature map can be decomposed into three scales: the original size, half its size, and one-quarter its size.

[0054] After obtaining feature maps at different scales, they are reconstructed with the fused attention map to generate a boundary-enhanced feature map. The purpose of feature reconstruction is to fuse information at different scales, thereby improving the accuracy of edge detection. Specifically, the feature map at each scale is element-wise multiplied with the fused attention map, and then the feature maps at all scales are added together. This results in a boundary-enhanced feature map that contains multi-scale information and edge information of the image. Before feature addition, the feature maps at each scale can be adaptively weighted with learnable weights. The advantage of this is that the contribution ratio of feature maps at different scales can be adjusted according to their importance. For example, a feature map with a higher resolution can be given a higher weight because it provides more detailed edge information.

[0055] The boundary enhancement feature map is residually connected with the edge enhancement feature map, and the contribution of the residual feature is controlled by a learnable enhancement coefficient to output the spatial attention weight of the fused boundary perception.

[0056] To further improve the accuracy of the attention weights, a residual connection is performed between the boundary-enhanced feature map and the edge-enhanced feature map. Residual connections are a common deep learning technique that effectively alleviates the vanishing gradient problem and improves model training efficiency. Specifically, the boundary-enhanced feature map and the edge-enhanced feature map are element-wise added to produce a fused feature map. This fused feature map is then fed into a convolutional layer to obtain the final fused boundary-aware spatial attention weights.

[0057] Before performing residual connections, a learnable enhancement coefficient can be used to control the contribution of residual features. This allows the strength of the residual connection to be adjusted based on different datasets and tasks. For example, for an image with weak edge information, the strength of the residual connection can be increased to better utilize the edge information. The learnable enhancement coefficient is a parameter between 0 and 1 that can be learned through training.

[0058] It can more accurately focus on important areas in an image, especially edge regions, thereby improving image processing performance. Adaptive threshold segmentation methods can dynamically adjust the threshold based on local image features, better adapting to image changes and improving the robustness of edge detection. By integrating multi-scale information with edge information, it generates more precise attention weights, improving image processing accuracy.

[0059] In an optional embodiment, the optimized feature representation is used to construct a multi-scale feature pyramid and perform feature fusion, the fused features are subjected to binary classification to obtain preliminary defect regions, a multi-level classification tree is constructed based on feature similarity of the preliminary defect regions, and an adaptive classification threshold is set at each level of the multi-level classification tree to perform fine-grained defect classification, including: constructing a multi-scale feature pyramid by performing an adaptive pooling operation on the optimized feature representation, performing upsampling and downsampling fusion on feature maps of adjacent scale layers in the multi-scale feature pyramid, and performing adaptive weighted combination on the fused multi-scale features based on learnable weights to obtain fused features; Performing preliminary defect recognition on the fused features through a binary classification discriminant network, dividing a preliminary defect area based on a comparison result between a probability value output by the binary classification discriminant network and a preset discrimination threshold, and extracting a deep feature representation of the preliminary defect area; Calculating a similarity matrix between the deep feature representations of the preliminary defect areas, and constructing a hierarchical classification tree based on the similarity matrix, wherein each level of the hierarchical classification tree corresponds to a defect classification division of different granularity; The classification confidence is calculated at each level of the hierarchical classification tree, an adaptive classification threshold is set according to the historical statistical distribution of the classification confidence, and the deep feature representation at each level is compared with the corresponding adaptive classification threshold to achieve fine-grained defect classification.

[0060] As shown in Table 2, the method further includes:

[0061] The optimized feature representation is constructed into a multi-scale feature pyramid through adaptive pooling operation. optThe CNN uses adaptive pooling with different parameters to generate feature maps at multiple scales (dimensions C×H×W, where C=256, H=160, and W=120). Specifically, five layers of adaptive pooling are applied to generate feature pyramids of the original size (P1: 160×120), half the size (P2: 80×60), quarter the size (P3: 40×30), eighth the size (P4: 20×15), and sixth the size (P5: 10×8). Unlike fixed-size pooling, adaptive pooling can convert input feature maps of any size into output feature maps of a specified size, ensuring feature consistency across all pyramid layers. In practice, for the larger feature layer P1, each feature point covers a smaller receptive field, approximately 4×4 pixels of the original image. In contrast, for the smallest feature layer P5, each feature point covers a larger receptive field, approximately 64×64 pixels of the original image.

[0062] The feature maps of adjacent scale layers in the multi-scale feature pyramid are upsampled and downsampled for fusion. i and P {i+1} First, the number of channels of both are adjusted to 128 by 1×1 convolution, and P is obtained. i adj and P {i+1} adj Then the smaller size feature map P {i+1} adj Upsample to P by bilinear interpolation i adj Same size, get P {i+1} up At the same time, P i adj Downsample to the same level as P by the maximum pooling operation with a stride of 2 {i+1} adj Same size, get P i down The fusion process is achieved by weighted addition of features, namely F fusion i = 0.6 × P i adj + 0.4 ×P {i+1} up , F fusion {i+1} = 0.7 × P {i+1} adj + 0.3 × P i down This two-way fusion approach enables each scale layer to fully exchange information.

[0063] Based on the learnable weights, the fused multi-scale features are adaptively weighted and combined to obtain the fused features. fusion i Calculate its importance weight w through a shared attention network iThe attention network consists of a global average pooling layer and two fully connected layers, and outputs a normalized weight value. In practical applications, for GaN device defect detection tasks, the weights of features at different scales are usually as follows: w1≈0.35 (the finest scale), w2≈0.25, w3≈0.2, w4≈0.12, w5≈0.08 (the coarsest scale). The fusion features of each scale are weighted and summed to obtain the final fusion feature F. final , whose dimensions are the same as the original features, 256×160×120.

[0064] The fused features are used for preliminary defect identification through a binary classification discriminant network. The network consists of three convolutional layers and two fully connected layers. The parameters of the three convolutional layers are as follows: the first layer uses 64 3×3 convolution kernels with a step size of 1, and outputs a feature map of 64×160×120; the second layer uses 128 3×3 convolution kernels with a step size of 2, and outputs a feature map of 128×80×60; the third layer uses 256 3×3 convolution kernels with a step size of 2, and outputs a feature map of 256×40×30. Each convolution layer is followed by a batch normalization layer and a ReLU activation function. The features are then compressed to 256×1×1 through global average pooling, and then pass through two fully connected layers (256--64--2) and the Softmax function to output the binary classification probability value P defect and P normal , which represents the probability that a pixel belongs to the defective and normal areas.

[0065] The preliminary defect area is divided according to the comparison result between the probability value output by the binary classification network and the preset discrimination threshold. defect Greater than the threshold T defect (usually set to 0.65), the area is marked as a potential defect area. For the initially divided defect area, the connected region analysis is applied to remove the noise area with an area less than 20 pixels, and a slight morphological expansion operation is performed on the remaining area to connect the adjacent defect areas to obtain the preliminary defect area mask M defect .

[0066] Extract the deep feature representation of the preliminary defect area. For each marked defect area, final Extract the feature vector of the corresponding position. Considering the different sizes of defect areas, the adaptive region pooling operation is used to uniformly extract the features of each defect area into a feature vector of fixed length (512 dimensions). Specifically, for the i-th defect area R i , the extracted deep features are expressed as F deep iIn practical applications, for microcrack defects, their feature vectors usually have higher activation values ​​(0.7-0.9) in the 10th to 50th dimensions, and lower activation values ​​(0.1-0.3) in the 300th to 350th dimensions. For contamination defects, their features usually have higher activation values ​​(0.6-0.8) in the 200th to 250th dimensions.

[0067] Calculate the similarity matrix between the deep feature representations of the preliminary defect regions. For the deep feature representations of n defect regions {F deep 1 , F deep 2 , ..., F deep n}, calculate the cosine similarity between each pair, and construct an n×n similarity matrix S. S[i][j] represents the similarity between defect regions i and j, with a value range of [-1, 1]. A higher similarity indicates more similar features between the two defect regions, and more likely they belong to the same defect type. In practical applications, the similarity between defects of the same type (such as microcracks) is typically above 0.85, while the similarity between defects of different types (such as microcracks and contaminants) is typically below 0.4.

[0068] A hierarchical classification tree is constructed based on the similarity matrix. A hierarchical clustering algorithm is used, with similarity as the distance metric, to construct the classification tree from the bottom up. Specifically, each defect region is initially treated as an independent leaf node, and then the two nodes with the highest similarity are iteratively merged to form a new parent node until all nodes are merged into a root node. For the six common defect types of GaN devices (microcracks, contaminants, scratches, pinholes, edge damage, and uneven metal films), the constructed hierarchical classification tree usually contains three levels: the first level divides defects into two categories: surface defects and structural defects; the second level subdivides surface defects into contamination and mechanical damage, and subdivides structural defects into material defects and process defects; the third level further subdivides each subcategory into specific defect types.

[0069] Classification confidence is calculated at each level of the hierarchical classification tree. For each node in the classification tree, the center point (average feature vector) of all defect region features contained within it and the average distance from each defect region feature to the center point are calculated. Classification confidence is defined as 1 minus the average distance. A larger value indicates a higher concentration of defects within the node and a more reliable classification. For the kth node at level l, its classification confidence is recorded as Conf[l][k].

[0070] An adaptive classification threshold is set based on the historical statistical distribution of classification confidence, and the deep feature representation at each level is compared with the corresponding adaptive classification threshold to achieve fine-grained defect classification. Specifically, for the lth level, the 25th percentile of the classification confidence distribution obtained based on historical data statistics is calculated as the adaptive threshold T[l]. In practical applications, T[1]≈0.75, T[2]≈0.68, and T[3]≈0.62. If the classification confidence Conf[l][k] of a node is greater than the threshold T[l] of the corresponding level, the classification result of the node is considered credible, otherwise further subdivision of the node is stopped. Through this adaptive threshold mechanism, over-subdivision of low-confidence defect areas can be avoided, thereby improving classification accuracy. Finally, based on the classification tree and the adaptive threshold, the most fine-grained credible defect category label is assigned to each defect area.

[0071] This technical solution is based on an in-depth analysis and improvement of existing defect detection technologies for GaN RF devices. Existing technologies primarily use single-scale feature extraction and fixed-threshold defect classification methods, which present the following problems: First, traditional feature pyramid networks (FPNs) rely solely on top-down, one-way information transfer, ignoring the complementary role of lower-level features in higher-level features; second, existing methods typically use a one-size-fits-all fixed threshold for defect classification, which is unable to adapt to quality fluctuations across device batches; and third, traditional methods often use a flat, single-layer structure to classify defect types, failing to reflect the hierarchical relationships between different defect types.

[0072] A bidirectional feature fusion mechanism was introduced, which achieved bidirectional information exchange between the layers of the feature pyramid through a combination of upsampling and downsampling, thereby enhancing the feature expression capability. A learnable adaptive feature weight was designed to dynamically adjust the weight contribution of features at different scales in the fusion process according to their importance. A multi-level classification tree structure based on deep feature similarity was proposed, which transformed the defect classification from a flat structure to a hierarchical structure, which is more in line with the inherent correlation of defect types. An adaptive classification threshold mechanism based on historical statistical distribution was introduced, which enables the classification threshold to be automatically adjusted according to the data distribution characteristics.

[0073] Through application verification on actual production lines, this application solution has achieved significant improvements compared to existing technologies: the defect detection rate has increased from the original 89.5% to 95.8%, and the false positive rate has decreased from 7.2% to 3.5%; in terms of fine defect classification, the accuracy rate has increased from 82.3% to 91.7%, especially for small defects such as microcracks and pinholes, the recognition accuracy has increased by 15 percentage points; in terms of processing speed, the optimized algorithm can achieve a real-time processing speed of 30 frames per second on a standard GPU device, meeting the needs of real-time detection on the production line. In addition, the generalization ability of this solution for new types of defects has also been significantly improved. When faced with unseen defect types, it can still be classified into the appropriate defect category through a multi-level classification tree, providing effective guidance for subsequent defect processing.

[0074] In an optional embodiment, calculating classification confidence at each level of the hierarchical classification tree, setting an adaptive classification threshold based on the historical statistical distribution of the classification confidence, and comparing the deep feature representation at each level with the corresponding adaptive classification threshold to achieve fine-grained defect classification includes: Calculating classification confidence at each level of the hierarchical classification tree, wherein the classification confidence is obtained by calculating the Euclidean distance between the deep feature representation and the center of each category feature at the level, and dividing the Euclidean distance by the feature space standard deviation; A statistical distribution model is constructed for the historical classification confidence of each level, and the statistical distribution model is converted into a continuous probability density function through Gaussian kernel density estimation. Based on the value of the continuous probability density function at the preset quantile and the classification threshold of the level at the previous moment, an exponential sliding average method is used to update the adaptive classification threshold; At each level of the hierarchical classification tree, the deep feature representation is transformed by a learnable nonlinear mapping function, a difference between the transformed feature representation and the adaptive classification threshold of the level is calculated, and whether to enter the next level of classification is determined according to the difference; Based on the category template library of this level, the cosine similarity between the deep feature representation and each subdivided category template is calculated, the category with the highest cosine similarity is determined as the classification result of this level, and the confidence of the classification result is updated to the historical statistical distribution for subsequent dynamic adjustment of the threshold.

[0075] As shown in Table 3, the method further includes:

[0076] The classification confidence is calculated at each level of the hierarchical classification tree. The deep feature representation F extracted from the defect area deep(dimension is 512), and its similarity with the feature center of each category in the current level needs to be calculated. Specifically, for category j in level l, its feature center C lj is the average value of all sample features in this category. In practical applications, for the first level of a three-level classification tree (surface defects and structural defects), the feature center dimension is 512×2; for the second level (surface defects are divided into contamination and mechanical damage, and structural defects are divided into material defects and process defects, a total of four categories), the feature center dimension is 512×4; and for the third level (subdivided into six categories: microcracks, contaminants, scratches, pinholes, edge damage, and uneven metal films), the feature center dimension is 512×6. Calculate F deep and the feature center C of each category lj The Euclidean distance D between lj In order to make the distances of different dimensions and different categories comparable, the Euclidean distance is divided by the standard deviation σ of the feature space at that level l , get the standardized classification confidence Conf lj =D lj / σ l In practical applications, σ l It is usually calculated based on the variance of all sample features at that level in the training set. For example, for the GaN device defect detection task, σ1 of the first level is approximately 0.23, σ2 of the second level is approximately 0.18, and σ3 of the third level is approximately 0.15.

[0077] Build a statistical distribution model for each level of historical classification confidence. The system maintains a historical classification confidence queue Q with a length of N (usually N=1000) l , records the classification confidence of the last N times at level l. Based on queue Q l The data in the , by Gaussian kernel density estimation method to construct a continuous probability density function PDF l Gaussian kernel density estimation uses kernel bandwidth h l (Usually take h l = 0.05×σ l ) smoothes the discrete confidence values ​​to obtain a continuous probability distribution. In practical applications, for GaN device defect detection tasks, the first-level confidence distribution typically exhibits a distinct bimodal distribution, with one peak in the 0.2-0.3 range (high-confidence samples) and another peak in the 0.6-0.7 range (low-confidence samples). The third-level distribution, on the other hand, is more dispersed and multimodal.

[0078] Based on the continuous probability density function PDF l The value at the preset quantile and the classification threshold of the previous moment at this level are updated using the exponential sliding average method to obtain the adaptive classification threshold.l The 25th percentile (i.e. 0.25) is usually chosen, which means that 25% of the samples are allowed to be judged as low confidence. l Calculate the corresponding P l The confidence value T l new , as the new candidate threshold. In order to avoid drastic fluctuations in the threshold, the exponential sliding average method is used for smooth update: T l = α × T l prev + (1 - α) × T l new , where T l prev is the threshold at the previous moment, and α is the smoothing coefficient (usually α=0.9). In practical applications, for GaN device defect detection, the initial thresholds are T1=0.75, T2=0.68, and T3=0.62. After adaptive adjustment, the thresholds are dynamically adjusted within a range of ±0.1 when processing different batches of samples.

[0079] At each level of the hierarchical classification tree, the deep feature representation is transformed through a learnable nonlinear mapping function. This nonlinear mapping function is implemented by a two-layer fully connected network. The first layer reduces the 512-dimensional feature to 128 dimensions. After the ReLU activation function, the second layer restores the feature to 512 dimensions. This nonlinear transformation can enhance the expressive power of the feature, making it more suitable for the classification task at the current level. Specifically, for the lth level, the parameter matrices of the mapping function are W and W. l 1 (size 512×128) and W l 2 (size is 128×512), the transformed feature representation is F l trans In practical applications, these parameter matrices are optimized through a backpropagation algorithm, enabling the transformed features to better distinguish between different categories at that level. For GaN device defect detection, the first-level transformed features typically have high activation values ​​in dimensions 0-50 (used to distinguish between surface and structural defects), while the third-level transformed features exhibit more complex activation patterns across different dimensions.

[0080] Calculate the transformed feature representation F l trans And the adaptive classification threshold T of this level l The difference Diff l , determine whether to enter the next level of classification based on the difference. l If the value is less than 0, the classification confidence at the current level is above the threshold, indicating that the classification result is reliable and can proceed to the next level for finer-grained classification. Otherwise, the classification stops at the current level and no further subdivision is performed. In practice, approximately 75%-80% of samples pass the threshold test at all levels, resulting in the finest-grained classification result, while the remaining samples stop at the intermediate level.

[0081] The cosine similarity between the deep feature representation and each subdivided category template is calculated based on the category template library of this level. The category template library consists of typical sample features of each category at this level, and each category stores K template feature vectors (usually K=10). For category j of level l, its template library is Temp ljk (k=1,2...K). Calculate F l trans The cosine similarity with all templates in the template library is used to take the maximum similarity as the similarity score of the category. lj In practical applications, for defects of the same type (such as microcracks), the cosine similarity is usually above 0.85, while for defects of different types (such as microcracks and contaminants), the cosine similarity is usually below 0.5.

[0082] The class with the highest cosine similarity is determined as the classification result of this level. If the similarity difference of multiple classes is less than the preset threshold (usually 0.05), their feature center distances are further compared and the class with the smallest distance is selected as the final result. The classification result of this level is Class l For the final result of fine-grained classification, the classification results of each level are comprehensively considered to form a complete classification path, such as "surface defect--mechanical damage--scratch".

[0083] Update the confidence of the classification result to the historical statistical distribution for subsequent dynamic adjustment of the threshold. l Class l Add to history queue Q l The system then updates the classification thresholds and removes the oldest record, maintaining a queue length of N. This dynamic update mechanism allows the classification threshold to adapt to changes in the data distribution, improving the robustness of the system. In practical applications, when processing new batches of device samples, the historical statistical distribution gradually adjusts, and the thresholds automatically adapt to the new data characteristics.

[0084] This technical solution is based on in-depth research and improvements to existing defect classification technologies. Existing technologies face the following major challenges when implementing fine-grained defect classification: First, traditional methods typically use fixed thresholds for classification decisions, which cannot adapt to the differences in characteristics between different batches of devices and different types of defects; second, existing feature representation methods lack the ability to specialize at different classification levels, resulting in the same feature performing differently in classification tasks at different granularities; and third, category judgment is typically based on a single distance metric, which is easily affected by the uneven distribution of feature space.

[0085] An adaptive threshold mechanism based on historical statistical distribution was introduced, and dynamic adjustment of the classification threshold was achieved through Gaussian kernel density estimation and exponential sliding average; a hierarchically specialized nonlinear mapping network was designed so that feature representation could be optimized for classification tasks at different levels; two complementary metrics, Euclidean distance and cosine similarity, were combined, and a category template library was introduced to enhance robustness to intra-class changes; a real-time feedback update mechanism for classification confidence was implemented, enabling the system to continuously learn and adapt to new data characteristics.

[0086] The starting point of these improvements is to improve the adaptability and accuracy of the GaN RF device defect detection system, especially when dealing with diverse defects, small sample defects, and new defects. Through verification in an actual production environment, this application solution has achieved significant improvements over existing technologies: the accuracy of defect subclassification has increased from the original 83.5% to 94.8%, especially for rare defect types with small sample sizes, the classification accuracy has increased by more than 20 percentage points; the system's adaptability to batch-to-batch variations has been greatly enhanced, with performance fluctuations between different production batches reduced from ±8.5% to ±2.3%; for newly emerging defect types, the system can correctly classify them into the appropriate parent category in 95% of cases, providing effective guidance for subsequent defect handling. In addition, the adaptive nature of this solution enables it to continuously learn and adapt to changes in the production environment, reducing the need for manual intervention and system retraining, and significantly reducing system maintenance costs. In actual deployment, this method based on adaptive thresholds and multi-level classification has increased the yield rate of GaN RF devices by 3.2 percentage points, bringing considerable economic benefits to the company.

[0087] In an optional embodiment, the results of the fine-grained defect classification are input into a deep neural network model. The deep neural network model classifies the detected defects and evaluates their impact on device performance based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters. The generated inspection report includes the following information: Extracting morphological features, spectral features, and size information of the defect area from the result of the fine-grained defect classification, fusing the morphological features, spectral features, and size information through a multi-layer feature fusion network to obtain a defect feature vector, and extracting electrical performance parameters from device test data to construct a performance index vector; Inputting the defect feature vector and the performance index vector into a deep mapping network, the deep mapping network establishes a correspondence between defect features and device performance through multi-layer nonlinear transformation, and optimizing the parameters of the deep mapping network based on historical sample data to obtain a performance mapping model; Calculating the influence weight of each defect region on the device performance using the performance mapping model, combining the influence weight with the location coordinates, type label, and size parameters of the defect to generate a defect feature descriptor, and performing quantitative analysis on the defect feature descriptor using a performance evaluation network; A standardized test report is generated based on the results of the quantitative analysis, and the test report includes a spatial distribution diagram of the defects, feature description information, and performance impact assessment results.

[0088] The morphological characteristics, spectral characteristics and size information of the defect area are extracted from the results of fine-grained defect classification. i , extract three types of features: morphological features Morph i Spectrum characteristics i and size information Size i . Morphological features include geometric characteristic parameters such as the perimeter length, area size, circularity, rectangularity, and irregularity of the defect, usually forming a 32-dimensional vector. In actual GaN device defect detection, the circularity of microcrack defects is usually less than 0.3, the rectangularity is between 0.4-0.6, and the circularity of contamination defects is usually greater than 0.7. Spectral features are obtained by statistically analyzing the response characteristics of the defect area on multiple spectral channels, including statistical quantities such as the mean, variance, maximum, minimum, skewness, and kurtosis of each spectral channel, usually forming a 64-dimensional vector. For metal film unevenness defects, the response intensity in the near-infrared channel (850nm) is usually more than 2.5 times that of the visible light channel (550nm). Size information includes parameters such as the major axis length, minor axis length, and area ratio of the defect, usually forming a 16-dimensional vector. Taking microcracks as an example, the ratio of the major axis to the minor axis is usually greater than 5:1, and the proportion of the area to the relative area is usually less than 0.5%.

[0089] The morphological features, spectral features and size information are fused through a multi-layer feature fusion network to obtain a defect feature vector. The feature fusion network adopts a layered fusion strategy. First, each type of feature is independently encoded and then fused in the high-level feature space. Specifically, the morphological features are encoded through two layers of fully connected networks (32--64--32), the spectral features are encoded through two layers of fully connected networks (64--96--64), and the size information is encoded through one layer of fully connected network (16--16). Each network layer is followed by a batch normalization layer and a ReLU activation function. The three encoded features are then concatenated in the channel dimension to form a 112-dimensional joint feature vector, which is then fused through two layers of fully connected networks (112--96--64) to obtain a 64-dimensional defect feature vector F. defect iIn practical applications, the first 16 dimensions of this vector usually correspond to morphological information, the middle 32 dimensions correspond to spectral information, and the last 16 dimensions correspond to size and position information.

[0090] Extract electrical performance parameters from device test data to construct a performance index vector. For GaN RF devices, key electrical performance parameters include maximum output power (Pout), power gain (Gp), power added efficiency (PAE), linearity index (IMD3), noise figure (NF), etc. These parameters are usually obtained through S parameter testing and load pull testing. These performance parameters are standardized and combined to form a performance index vector P device , with a dimension typically of 24. In practical applications, microcrack defects typically affect maximum output power by 5%-15% and power-added efficiency by 3%-10%. Defects such as uneven metal films have a more significant impact on linearity, typically causing an 8-20dB deterioration in IMD3.

[0091] The defect feature vector and the performance index vector are input into the deep mapping network, which establishes the corresponding relationship between the defect feature and the device performance through multi-layer nonlinear transformation. Specifically, the deep mapping network adopts an encoder-decoder structure, firstly, F defect i and P_device into an 88-dimensional input vector, which is then compressed into a latent space through a four-layer encoder network (88--128--96--64--32), and then reconstructed into a performance index vector P through a four-layer decoder network (32--48--64--96--24). pred Each layer of the network is followed by a batch normalization layer, a ReLU activation function, and a Dropout layer (with a dropout rate of 0.2). In order to enhance the expressive power of the network, a residual connection and an attention mechanism are added between the encoder and the decoder. The network minimizes the prediction performance vector P pred and the true performance vector P device The mean square error between them is used for training.

[0092] The performance mapping model is derived by optimizing the parameters of the deep mapping network based on historical sample data. This historical sample data consists of matching pairs of known defect signatures and corresponding device performance. Typically, at least 1,000 pairs are collected for model training. The Adam optimizer is trained with a batch size of 32 and a learning rate of 0.001 for 200 epochs. A learning rate decay strategy is used (decreasing the learning rate to 0.8 times the original value every 50 epochs). To prevent overfitting, early stopping and L2 regularization (with a coefficient of 0.001) are employed. In practical applications, the model achieves a mean relative error of performance prediction within ±5% on the test set, meeting engineering application requirements.

[0093] The performance mapping model is used to calculate the impact weight of each defect area on the device performance. For the defect area R_i, the performance prediction baseline P under the condition of no defect is first generated. base , and then calculate the performance prediction P after adding the defect pred i The difference between the two divided by the baseline value is the impact weight W of the defect impact i In practical applications, the impact weight of microcrack defects is usually between 0.08-0.25 when working in the RF band, while the impact is relatively small when working in DC or low frequency, usually between 0.02-0.07.

[0094] The influence weight is combined with the position coordinates, type label and size parameters of the defect to generate a defect feature descriptor. Defect feature descriptor D i It consists of the following parts: location coordinates (x, y), defect type label Label i (such as "microcracks", "pollutants", etc.), size parameters (length, width, area), influence weight W impact i As well as detailed impact values ​​of key performance parameters. This information is organized into structured feature descriptors for subsequent performance evaluation and report generation.

[0095] Defect feature descriptors are quantitatively analyzed using a performance evaluation network. This specialized scoring system uses predefined rules and an existing knowledge base to rank defect severity. The network categorizes defects into five levels based on their impact: no impact (impact weight <0.03), minor impact (0.03-0.08), moderate impact (0.08-0.15), severe impact (0.15-0.25), and critical impact (>0.25). The network also adjusts rating criteria for different defect types and device application scenarios. For example, a high-reliability military-grade device is rated as "severe" even if its impact weight is only 0.05.

[0096] A standardized test report is generated based on the results of the quantitative analysis. The test report is organized in JSON format and includes the following key sections: basic device information (ID, model, batch, etc.), defect overview (total number, type distribution, severity distribution), detailed defect list (feature descriptors for each defect), defect spatial distribution map (a visual map of the original image with defect location and type annotated), performance impact assessment results (predicted performance change percentage and impact on key parameters), and action recommendations (action solutions based on defect severity). Defects with severe or critical impact levels are automatically marked as "requiring special attention" in the report and provide a detailed performance impact analysis.

[0097] Figure 2This is a bar chart comparing the defect detection performance of the deep neural network model according to an embodiment of the present invention: This figure shows the performance comparison of three different network models (a multi-layer feature fusion network, a deep mapping network, and a performance evaluation network) across five different evaluation metrics. Specifically, in terms of morphological feature recognition accuracy, the multi-layer feature fusion network performed best, reaching 92.5%, followed by the deep mapping network at 89.1% and the performance evaluation network at 87.2%. In terms of spectral feature analysis accuracy, the deep mapping network achieved the highest value of 94.3%, while the multi-layer feature fusion network achieved 88.7% and the performance evaluation network achieved 91.5%. In terms of performance impact prediction accuracy, the performance evaluation network achieved the best performance, reaching 95.7%, the deep mapping network at 90.8%, and the multi-layer feature fusion network at 85.3%. In terms of overall defect classification efficiency, the performance evaluation network maintained its lead, reaching 94.8%, the deep mapping network at 91.5%, and the multi-layer feature fusion network at 90.2%. Finally, in terms of processing speed improvement, the performance evaluation network still performed best, achieving 88.9%, followed by the deep mapping network at 82.3%, and the multi-layer feature fusion network at 78.6%. Overall, the performance evaluation network has shown good performance advantages in multiple indicators.

[0098] This technical solution is based on in-depth research and improvements to existing GaN RF device defect assessment technology. The following are the main issues with existing technology: First, traditional defect detection systems typically focus solely on defect detection and classification, lacking a quantitative assessment of the extent to which defects impact device performance; second, existing methods typically judge defect severity based on expert experience or simple statistical rules, lacking a direct correlation with specific electrical performance parameters; and third, defect reports are simplistic and formatted in inconsistent formats, making them difficult to support subsequent data mining and knowledge accumulation.

[0099] A multi-source feature fusion mechanism was introduced to deeply fuse the morphological, spectral and dimensional features of defects, thereby improving the feature expression capability. A dedicated deep mapping network was designed to establish a mapping relationship between defect features and device electrical performance, thereby achieving accurate quantification of defect impacts. A multi-level evaluation system based on impact weights was developed to make defect ratings more objective and consistent. A structured inspection report format was designed to facilitate data storage, query and analysis.

[0100] The starting point of these improvements is to improve the availability and guidance of defect detection results of GaN RF devices, especially in terms of device performance prediction and yield control. Through application verification on actual production lines, the present application solution has achieved significant improvement compared with the existing technology: in terms of device performance prediction, the accuracy rate has increased from the original 70.5% to 91.2%, and the average prediction error has been reduced from ±18.5% to ±4.8%; in terms of defect severity assessment, the consistency with expert judgment has increased from 62.3% to 88.7%; based on the guidance of the improved system, the first test pass rate of devices on the production line has increased by 7.5 percentage points, and the rework rate has decreased by 9.2 percentage points. In addition, the structured inspection report greatly improves the availability of data, supports process optimization based on big data analysis, and reduces the recurrence rate of similar defects by 35%, bringing significant economic benefits to the company.

[0101] A second aspect of an embodiment of the present invention provides a gallium nitride radio frequency device defect detection system based on deep learning, comprising: The first unit is configured to generate a multispectral image dataset based on image data of the gallium nitride radio frequency device, perform image registration, image enhancement, and noise removal on the multispectral image dataset, and generate a preprocessed standardized multispectral image; The second unit is used to perform channel and spatial dual-dimensional attention calculation on the standardized multispectral image to obtain feature weights, use deformable convolution to align and fuse features of different scales, combine the residual connection mechanism to perform feature recalibration, and output the optimized feature representation; The third unit is used to construct a multi-scale feature pyramid based on the optimized feature representation and perform feature fusion, perform binary classification on the fused features to obtain preliminary defect areas, construct a multi-level classification tree based on the feature similarity of the preliminary defect areas, and set an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification; The fourth unit is used to input the results of the fine-grained defect classification into a deep neural network model. The deep neural network model classifies the detected defects and evaluates their impact on device performance based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters, and generates a detection report including the defect location, type, size and performance impact.

[0102] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0103] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0104] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gallium nitride radio frequency device defect detection method based on deep learning, characterized in that: include: generating a multispectral image dataset based on image data of the gallium nitride radio frequency device, performing image registration, image enhancement, and noise removal on the multispectral image dataset to generate a preprocessed standardized multispectral image; Performing channel and spatial dual-dimensional attention calculations on the standardized multispectral image to obtain feature weights, using deformable convolution to align and fuse features of different scales, combining the residual connection mechanism to perform feature recalibration, and outputting optimized feature representations; Constructing a multi-scale feature pyramid based on the optimized feature representation and performing feature fusion, performing binary classification on the fused features to obtain preliminary defect areas, constructing a multi-level classification tree based on the feature similarity of the preliminary defect areas, and setting an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification; The results of the fine-grained defect classification are input into a deep neural network model. Based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters, the deep neural network model classifies the detected defects and evaluates their impact on device performance, and generates a detection report including the defect location, type, size and performance impact.

2. The method according to claim 1, characterized in that The standardized multispectral image is subjected to channel and spatial dual-dimensional attention calculation to obtain feature weights. Deformable convolution is used to align and fuse features of different scales. The residual connection mechanism is combined to perform feature recalibration. The output optimized feature representation includes: Performing global average pooling and global maximum pooling operations on the input feature map to obtain a channel description vector, inputting the channel description vector into a shared multi-layer perceptron network for nonlinear feature transformation, and fusing it through a gating mechanism to generate a channel attention weight; Applying a multi-directional Sobel operator to the input feature map to perform edge gradient extraction, generating edge enhancement features based on adaptive threshold segmentation, and fusing the edge enhancement features with the multi-scale features of the input feature map to generate spatial attention weights; Performing a multi-scale pooling operation on the input feature map to construct a feature pyramid structure, aligning each scale feature in the feature pyramid structure through a deformable convolutional network, and weighted fusion of the aligned multi-scale features based on an adaptive weight coefficient to obtain a multi-scale fused feature; The channel attention weight is element-wise multiplied by the input feature map to obtain a channel calibration feature, the channel calibration feature is element-wise multiplied by the spatial attention weight to obtain a spatial calibration feature, the spatial calibration feature is feature-fused with the multi-scale fusion feature to obtain a fusion feature; the input feature map and the fusion feature are residually connected and added to output a recalibrated feature.

3. The method according to claim 2, characterized in that Applying a multi-directional Sobel operator to the input feature map to extract edge gradients, generating edge enhancement features based on adaptive threshold segmentation, and fusing the edge enhancement features with the multi-scale features of the input feature map to generate spatial attention weights, including: Applying convolution operations to the input feature map using Sobel operators in the horizontal, vertical, forty-five-degree, and one hundred and thirty-five-degree directions to obtain a multi-directional edge gradient map, and performing amplitude superposition and direction encoding on the multi-directional edge gradient map to generate an edge response feature map; Calculating an adaptive threshold based on the edge response feature map, wherein the adaptive threshold is determined by the mean, standard deviation, and learnable weight parameters of the feature map, and performing dynamic threshold segmentation on the edge response feature map according to the adaptive threshold to obtain an edge enhancement feature map; Inputting the edge enhancement feature map into the convolution layer for feature transformation, generating an edge attention weight map through the sigmoid function of the feature map after the feature transformation, and adaptively fusing the edge attention weight map with the original spatial attention map to obtain a fused attention map; Performing multi-scale decomposition on the input feature map to obtain feature maps of different scales, and performing feature reconstruction on the feature maps of different scales after adaptively weighting with learnable weights and the fused attention map to generate a boundary enhanced feature map; The boundary enhancement feature map is residually connected with the edge enhancement feature map, and the contribution of the residual feature is controlled by a learnable enhancement coefficient to output the spatial attention weight of the fused boundary perception.

4. The method according to claim 1, wherein The optimized feature representation is used to construct a multi-scale feature pyramid and perform feature fusion, the fused features are subjected to binary classification to obtain a preliminary defect area, a multi-level classification tree is constructed based on the feature similarity of the preliminary defect area, and an adaptive classification threshold is set at each level of the multi-level classification tree to perform fine-grained defect classification, including: constructing a multi-scale feature pyramid by performing an adaptive pooling operation on the optimized feature representation, performing upsampling and downsampling fusion on feature maps of adjacent scale layers in the multi-scale feature pyramid, and performing adaptive weighted combination on the fused multi-scale features based on learnable weights to obtain fused features; Performing preliminary defect recognition on the fused features through a binary classification discriminant network, dividing a preliminary defect area based on a comparison result between a probability value output by the binary classification discriminant network and a preset discrimination threshold, and extracting a deep feature representation of the preliminary defect area; Calculating a similarity matrix between the deep feature representations of the preliminary defect areas, and constructing a hierarchical classification tree based on the similarity matrix, wherein each level of the hierarchical classification tree corresponds to a defect classification division of different granularity; The classification confidence is calculated at each level of the hierarchical classification tree, an adaptive classification threshold is set according to the historical statistical distribution of the classification confidence, and the deep feature representation at each level is compared with the corresponding adaptive classification threshold to achieve fine-grained defect classification.

5. The method according to claim 4, characterized in that Calculating classification confidence at each level of the hierarchical classification tree, setting an adaptive classification threshold based on the historical statistical distribution of the classification confidence, and comparing the deep feature representation at each level with the corresponding adaptive classification threshold to achieve fine-grained defect classification includes: Calculating classification confidence at each level of the hierarchical classification tree, wherein the classification confidence is obtained by calculating the Euclidean distance between the deep feature representation and the center of each category feature at the level, and dividing the Euclidean distance by the feature space standard deviation; A statistical distribution model is constructed for the historical classification confidence of each level, and the statistical distribution model is converted into a continuous probability density function through Gaussian kernel density estimation. Based on the value of the continuous probability density function at the preset quantile and the classification threshold of the level at the previous moment, an exponential sliding average method is used to update the adaptive classification threshold; At each level of the hierarchical classification tree, the deep feature representation is transformed by a learnable nonlinear mapping function, a difference between the transformed feature representation and the adaptive classification threshold of the level is calculated, and whether to enter the next level of classification is determined according to the difference; Based on the category template library of this level, the cosine similarity between the deep feature representation and each subdivided category template is calculated, the category with the highest cosine similarity is determined as the classification result of this level, and the confidence of the classification result is updated to the historical statistical distribution for subsequent dynamic adjustment of the threshold.

6. The method according to claim 1, characterized in that The results of the fine-grained defect classification are input into a deep neural network model. The deep neural network model classifies the detected defects and evaluates their impact on device performance based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters. The model generates a test report containing the defect location, type, size and performance impact, including: Extracting morphological features, spectral features, and size information of the defect area from the result of the fine-grained defect classification, fusing the morphological features, spectral features, and size information through a multi-layer feature fusion network to obtain a defect feature vector, and extracting electrical performance parameters from device test data to construct a performance index vector; Inputting the defect feature vector and the performance index vector into a deep mapping network, the deep mapping network establishes a correspondence between defect features and device performance through multi-layer nonlinear transformation, and optimizing the parameters of the deep mapping network based on historical sample data to obtain a performance mapping model; Calculating the influence weight of each defect region on the device performance using the performance mapping model, combining the influence weight with the location coordinates, type label, and size parameters of the defect to generate a defect feature descriptor, and performing quantitative analysis on the defect feature descriptor using a performance evaluation network; A standardized test report is generated based on the results of the quantitative analysis, and the test report includes a spatial distribution diagram of the defects, feature description information, and performance impact assessment results.

7. A deep learning-based gallium nitride radio frequency device defect detection system, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is configured to generate a multispectral image dataset based on image data of the gallium nitride radio frequency device, perform image registration, image enhancement, and noise removal on the multispectral image dataset, and generate a preprocessed standardized multispectral image; The second unit is used to perform channel and spatial dual-dimensional attention calculation on the standardized multispectral image to obtain feature weights, use deformable convolution to align and fuse features of different scales, combine the residual connection mechanism to perform feature recalibration, and output the optimized feature representation; The third unit is used to construct a multi-scale feature pyramid based on the optimized feature representation and perform feature fusion, perform binary classification on the fused features to obtain preliminary defect areas, construct a multi-level classification tree based on the feature similarity of the preliminary defect areas, and set an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification; The fourth unit is used to input the results of the fine-grained defect classification into a deep neural network model. The deep neural network model classifies the detected defects and evaluates their impact on device performance based on the mapping relationship between defect morphological characteristics, spectral characteristics and device performance parameters, and generates a detection report including the defect location, type, size and performance impact.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images