A method and system for detecting defects in power devices

Abstract

This invention provides a method and system for detecting defects in power devices, relating to the field of semiconductor technology. The method includes: acquiring a set of power device defect images labeled with defects; expanding the power device defect image set through geometric modeling, combining the material properties of the power devices; constructing a decoupled detection model for identifying defect categories and locating defect positions; training the decoupled detection model using the expanded power device defect image set, incorporating a bimodal localization loss function; acquiring real-time images of the power devices; inputting the real-time images of the power devices into the trained decoupled detection model, and outputting the defect category and defect position of the real-time images of the power devices. Through precise model training and data expansion, the accuracy and robustness of detection are improved, effectively solving the problem of missed detections and improving the production quality and efficiency of power devices.

Description

Technical Field

[0001] This invention relates to the field of semiconductor technology, and in particular to a method and system for detecting defects in power devices. Background Technology

[0002] Power devices are semiconductor devices used in power electronic systems to control and convert electrical energy. They typically include diodes, transistors, and three-terminal devices (such as IGBTs). They play a crucial role in regulating voltage, current, and power conversion in power systems. The performance of power devices directly affects the efficiency, stability, and lifespan of power electronic equipment.

[0003] Defect detection in power devices involves identifying and detecting any surface defects that may occur during the manufacturing process, ensuring reliable operation during normal operation. Because power devices withstand high current and high temperature environments, even minor surface defects can lead to serious electrical faults, system failures, or even safety accidents. Therefore, timely and effective surface defect detection is crucial for ensuring the quality and reliability of power devices. This not only reduces equipment failure rates and extends equipment lifespan but also guarantees the safe and efficient operation of power systems.

[0004] With the rapid development of artificial intelligence, defect detection in power devices has gradually replaced manual inspection. However, current training sets for power device defects often cannot be effectively and accurately expanded, resulting in insufficient generalization ability of AI-based power device defect detection. This leads to a large number of missed detections in practical applications, which greatly affects the production quality and efficiency of power devices. Summary of the Invention

[0005] To address the technical problem that existing technologies often fail to effectively and accurately expand the training set for power device defects, resulting in insufficient generalization ability of AI-based power device defect detection and numerous missed detections in practical applications, which greatly affects the production quality and efficiency of power devices, this invention provides a power device defect detection method and system.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect: An embodiment of the present invention provides a method for detecting defects in power devices, comprising: S1: Obtain a set of defect images of power devices with defect tags; S2: Combine the material properties of power devices to expand the defect image set of power devices through geometric modeling; S3: Construct a decoupled detection model for identifying defect categories and locating defect positions; S4: Combine the dual-modal localization loss function and use the expanded power device defect image set to train the decoupling detection model; S5: Acquire real-time images of power devices; S6: Input the real-time image of the power device into the trained decoupled detection model, and output the defect category and defect location of the real-time image of the power device.

[0008] The second aspect: An embodiment of the present invention provides a power device defect detection system, comprising: processor; The memory stores computer-readable instructions, which, when executed by a processor, implement the power device defect detection method as described in the first aspect.

[0009] Third aspect: The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the power device defect detection method as described in the first aspect.

[0010] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, by combining the material properties and geometric modeling of power devices to expand the image set, the problem that existing training sets cannot cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data but also improves the model's generalization ability, reducing the probability of missed detections in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect positions, improving detection accuracy and efficiency. By combining a bimodal localization loss function, the accuracy of defect localization is optimized, further improving the performance of power device defect detection, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detections in the production process, and improving production quality and efficiency. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic flowchart of a power device defect detection method provided in an embodiment of the present invention;

[0013] Figure 2 This is a schematic diagram of the structure of a decoupling detection model provided in an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram of a power device defect detection system provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0016] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0017] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0018] Reference manual attached Figure 1 The diagram shows a flowchart of a power device defect detection method provided by an embodiment of the present invention.

[0019] This invention provides a method for detecting defects in power devices. This method can be implemented using a power device defect detection device, which can be a terminal or a server. The processing flow of the power device defect detection method may include the following steps:

[0020] S1: Obtain a set of power device defect images with defect labels.

[0021] Understandably, the acquired set of defective power device images with defect labels provides the foundational data for subsequent defect detection. These images contain different types of defects, and each image is labeled to provide clear defect category and location references for training the model, thereby ensuring the accuracy and effectiveness of the detection model.

[0022] In one possible implementation, defect labels include crack labels, bubble labels, cold solder joint labels, and pin deformation labels.

[0023] It should be noted that defect labels include crack labels, bubble labels, cold solder joint labels, and lead deformation labels. These labels are used to identify different types of defects in power devices. By labeling each image, the training model can be improved to identify and locate various defects, thereby improving the accuracy of the detection model in classifying and locating different defects and ensuring the accuracy of the detection results.

[0024] S2: Combine the material properties of power devices to expand the defect image set of power devices through geometric modeling.

[0025] It's important to note that the existing defect image set for power devices is expanded by combining material properties and geometric modeling. Specifically, the characteristics of different materials used in power devices (such as silicon and silicon carbide), including their thermal properties and electrical conductivity, are first considered. These material properties affect the formation and development of defects. Based on this, simulated defect images, including cracks and bubbles, are constructed using geometric modeling methods. By calculating the defect patterns that different materials may produce under various stress and temperature conditions, more types of defect images are generated. This ensures that the expanded image set reflects the actual physical properties of the power devices. This expansion not only allows the training set to cover more defect types but also simulates the impact of changes in material properties on defect morphology, improving the model's generalization ability and enabling it to better adapt to different materials and defect scenarios, thereby enhancing the accuracy and robustness of defect detection.

[0026] In one possible implementation, the power device defect image set includes multiple power device defect images. S2 specifically includes:

[0027] S201: Extract grayscale images of power device defect images corresponding to different defect labels in the power device defect image set.

[0028] A grayscale image is a color image converted into a grayscale image, where the value of each pixel represents brightness and does not contain color information. Grayscale images are used to simplify image processing, especially when detecting and analyzing defects, allowing focus on the shape and structure of the image.

[0029] S202: Based on grayscale images, construct differential geometric manifold representations of defect features with curvature and Gaussian geometric curvature tensors to capture line defect primitives and point defect primitives of power devices.

[0030] Among them, curvature captures the line defect primitive, while Gaussian geometric curvature tensor captures the point defect primitive.

[0031] The specific formula for constructing the representation of a differential geometric manifold is as follows: ; in, Representing the differential geometric manifold representation, This represents the determinant of the Hessian matrix describing the Gaussian geometric curvature tensor. The curvature represents the degree of bending at the edge of the defect. Represents the pixel coordinates of a power device defect image. grayscale value at that location This represents the grayscale gradient of a power device defect image. The second derivative of the grayscale value of a power device defect image is the Hessian matrix. The gradient magnitude describes the magnitude of the gradient change. Represents the divergence operator. This represents the determinant operator.

[0032] Differential geometric manifolds are mathematical methods for describing the local geometric structure of points in an image. By analyzing the gradient and second derivative of the image's grayscale values, information about the curvature and Gaussian geometric curvature of the image can be obtained. These features help capture the morphology of defects in the image, especially linear defects (such as cracks) and point defects (such as bubbles). Curvature is used to describe the degree of bending of an image edge or surface at a specific point. Here, curvature is used to capture linear defects, i.e., elongated defects in the image. The Gaussian geometric curvature tensor is used to capture the features of point defects. It evaluates the local curvature of the image by calculating the second derivative of the image (i.e., the Hessian matrix) to identify defects like bubbles or other similar defects.

[0033] Specifically, Indicates the direction of normalized gradient calculation That is, the divergence trend of the unit vector field.

[0034] It should be noted that by extracting the grayscale image and constructing a differential geometric manifold representation incorporating curvature and Gaussian geometric curvature tensors using geometric modeling methods, this representation can effectively capture the geometric features of defects in the image, especially line defects and point defects. By analyzing the grayscale gradient and second derivative of the image, the model can accurately identify the morphology of defects, and thus simulate the diverse manifestations of different defects, especially those types that are difficult to obtain in real-world image sets. By combining material properties with geometric characteristics, more defect samples are generated, ensuring that the training data is more representative. This expansion not only makes the training set more diverse but also enhances the model's adaptability and improves the accuracy and robustness of defect detection.

[0035] S203: Determine the defect propagation direction angle of the power device defect image.

[0036] The formula for the defect propagation direction angle is as follows: ; in, Indicates the direction angle of defect propagation. and These represent the gray-level gradients in the x and y directions of the power device defect image, respectively. Represents the arctangent function. This indicates that the mean value is calculated within the corresponding local window.

[0037] It should be noted that the reason for calculating the defect propagation direction angle is that the crack growth direction is not arbitrary, but constrained by stress distribution and crystal orientation. The principal stress direction often coincides with or approximates the actual crack direction. If the directional constraint is ignored during generation, non-physical crack bending or propagation will occur, affecting the realism of the simulation. Introducing the defect propagation direction angle can effectively control the defect to better reflect the "orientation" distribution in real-world scenarios.

[0038] S204: Based on the defect propagation direction angle, the stochastic transformation field describing the defect propagation law is determined by combining the Navier-Cauchy equation.

[0039] The formula for a randomly transformed field is as follows: ; ; ; in, Represents the image of power device defects in coordinates The random deformation field at that location, Indicates the deformation scaling factor. This indicates the shear modulus of the material corresponding to the detection area of ​​the power device. Indicates the angle of defect propagation direction The relevant rotation tensor, and Let them represent the sine function and the cosine function, respectively. This represents the first Lamé constant, i.e., the volume change resistance, of the material corresponding to the detection region of the power device. Representing coordinates The basic displacement field at the location, The random coefficients of the frequency term k are represented by e, and e represents the natural constant. Indicates the angle related to k along the defect propagation direction. Plane waves in the direction of Representing coordinates that conform to a Gaussian distribution The random perturbation term at the location, This represents the gradient operator. Represents the second derivative operator. This represents the linear elastic equilibrium equation in the Navier-Cauchy equations when there are no body forces.

[0040] The Navier-Cauchy equations describe the elastic equilibrium of materials under no-body-force conditions, describing the relationship between stress and deformation. The Navier-Cauchy equations help simulate the stress and deformation processes during defect propagation by describing the elastic properties of materials (such as shear modulus and volumetric resistance). A random transformation field is used to simulate defect propagation and deformation, taking into account the physical behavior of the defect. Through this transformation field, the defect can be randomly deformed according to a specified propagation direction and stress characteristics, generating new defect images. A rotation tensor is used to rotate the deformation according to the defect propagation direction angle, thus ensuring that the defect propagation direction conforms to actual physical conditions. A basic displacement field describes the displacement or deformation of the defect in the image. A Gaussian-distributed perturbation term simulates noise or irregularities in the image, conforming to a Gaussian distribution, increasing the diversity of generated images and making defect propagation more natural.

[0041] It should be noted that a stochastic transformation field describing the defect propagation law is constructed by combining the defect propagation direction angle and the Navier-Cauchy equation. In this process, the rotation angle is first determined based on the defect propagation direction, and then the stress distribution and defect propagation process of the material are simulated through the transformation field. This method can generate more realistic images of defect propagation and simulate the effects of different stresses and deformations on defects. Through physical modeling and stochastic perturbation, this method can generate diverse defect samples, which not only enhances the diversity of the dataset but also improves the model's adaptability and accuracy to different types of defects, thereby effectively improving the performance and robustness of defect detection.

[0042] S205: Combining the differential geometric manifold representation, under the constraint of the projection consistency constraint of the random transformation field, a power device defect image is generated, thus completing the expansion of the power device defect image set.

[0043] The specific formula for generating defect images of power devices is as follows: ; in, This represents the generated image of power device defects. This represents the generated defect image of the candidate power device. This represents the total variation term that ensures image smoothness. Indicates the weight of the smoothing term. This indicates taking the minimum function value. , This represents the differential geometric manifold representation after a randomized field transformation. This indicates element-wise multiplication. Indicates a mask for the defect area. express Projection image under X-ray imaging, This represents the calculation of the square of the Frobenius norm.

[0044] in, The comparison projection performance ensures consistency between the X-ray image and the deformed structure in industrial inspection imaging systems (XCT), thus simulating the appearance of defects as seen in real-world imaging, rather than merely resembling them. Projection consistency constraints ensure that the generated image and the transformed geometric manifold exhibit consistent performance under X-ray projection, guaranteeing that the structure and physical properties of the generated image accurately reflect the defect morphology. Total variation is a mathematical method for reducing image noise; it aids in image smoothing, preventing excessive smoothing of edges while preserving structural details. A mask is used to specify defect areas in the image, ensuring that smoothing and other processing are performed only within those defect areas.

[0045] Optionally, the smoothing term weight can be set to 0.5.

[0046] It should be noted that by introducing projection consistency constraints when generating defect images of power devices, the generated defect images are ensured to behave consistently with the real defect images under X-ray imaging. The structure of the generated image is optimized by comparing it with its X-ray projection and transformed geometric manifold representation. Total variational terms help smooth the image and avoid noise interference. This method not only generates more realistic defect images but also ensures high physical consistency and detectability in actual industrial inspection, improving the model's ability to identify defects and effectively reducing noise and unnatural artifacts during the generation process.

[0047] Specifically, by combining differential geometric manifold representation and stochastic transformed fields, defect images of power devices are generated using projection consistency constraints. By comparing the generated images with X-ray projections and geometric manifolds, the image structure is optimized to ensure consistent performance of the generated defect images under X-ray imaging, thus improving physical consistency. The total variational term effectively smooths the image, reducing noise interference and making the generated images more realistic in actual inspections. This enhances the model's robustness and defect recognition capabilities, and reduces artifact generation.

[0048] Reference manual attached Figure 2 The diagram shows a structural schematic of a decoupling detection model provided in an embodiment of the present invention.

[0049] S3: Construct a decoupled detection model for identifying defect categories and locating defect positions.

[0050] It should be noted that the decoupled detection model can simultaneously achieve defect category identification and precise defect location. The decoupled design allows the model to independently handle both defect classification and localization tasks, thereby improving detection efficiency and accuracy. In this way, the model can accurately determine the specific location of the defect in the image while identifying the defect type.

[0051] In one possible implementation, the decoupled detection model includes an input layer, a feature extraction module, a defect detection module, and an output layer connected in sequence. The decoupled detection model further includes a defect localization module, which is connected to the feature extraction module and the output layer respectively. The feature extraction module has a CBAM salient feature extraction unit based on dynamic differential feature generation and phase modulation functions.

[0052] The input layer receives image data from the power devices as the initial input to the model. The feature extraction module extracts meaningful features from the input image. This module uses a CBAM (Convolutional Block Attention Module) salient feature extraction unit based on dynamic differential feature generation and phase modulation. The CBAM module adaptively enhances the salient features of important regions while suppressing unimportant regions, helping the model focus on key parts of the image, such as defect areas. Dynamic differential feature generation strengthens the capture of edges and textures in the image, while phase modulation further enhances salient features by adjusting phase information. The defect detection module identifies defect types based on the features provided by the feature extraction module and accurately locates defects using algorithms. The output layer generates the final detection output based on the processing results of the defect detection module.

[0053] It should be noted that by decoupling the defect category identification and localization tasks, the accuracy of each task is effectively improved. The CBAM salient feature extraction unit introduced in the feature extraction module dynamically enhances the saliency of the defect region, enabling the model to more accurately capture defect features in the image. Through this design, the model can focus more on key features, improving the accuracy and efficiency of defect detection, especially when dealing with complex and minute defects.

[0054] S4: Combine the dual-modal localization loss function and use the expanded power device defect image set to train the decoupled detection model.

[0055] It should be noted that by combining a dual-modal localization loss function and training the decoupled detection model using an expanded set of power device defect images, the model can more accurately locate defects and detect defect types, improving its accuracy and robustness in real-world applications. In this way, the model can better handle the diversity of defects, improve localization accuracy, and ensure efficient and accurate detection results.

[0056] In one possible implementation, S4 specifically includes:

[0057] S401: Input the power device defect images from the expanded power device defect image set into the decoupled detection model in sequence to obtain the predicted defect category output by the defect detection module and the predicted defect location output by the defect localization module.

[0058] S402: Calculate the defect category identification accuracy for predicting defect categories.

[0059] S403: Calculate the bimodal localization loss value describing the geometric alignment relationship of the predicted defect location.

[0060] The specific formula for calculating the dual-modal localization loss is as follows: ; ; ; ; in, This represents the bimodal localization loss value. This represents the normalized Wasserstein distance between the predicted defect location and the actual defect location. This represents the area-weighted intersection-over-union ratio (IoU) between the predicted and actual defect locations, where exp represents the natural exponential function. This represents a two-dimensional Gaussian distribution of the prediction box obtained by modeling based on the predicted defect location. The bounding box is a two-dimensional Gaussian distribution obtained by modeling the actual defect location corresponding to the power device defect image, where C represents the sum of the squares of the width of the power device defect image and the actual location of the defect. And power device defect image height squared The relevant normalization constant for avoiding runaway loss values. Represent two two-dimensional Gaussian distributions and The square of the Wasserstein distance between them express The center point position vector, express The center point position vector, This indicates the calculation of the squared Euclidean distance. and They represent and The covariance matrix, This indicates the computation of the trace of a matrix. and These represent the geometric boundaries of the predicted bounding box and the ground truth bounding box, respectively. Represents the geometric boundary of the real bounding box The actual area of ​​the defined frame. This represents the median of the true bounding box area in a power device defect image set. This represents the hyperbolic tangent function.

[0061] The bimodal localization loss combines the Wasserstein distance (NWD) and area-weighted intersection-union ratio (AWIoU) to measure the difference between the predicted and actual defect locations. The goal is to minimize this loss and improve localization accuracy. The Wasserstein distance is a method for measuring the difference between two distributions. AWIoU measures the ratio of the intersection area to the union area of ​​the predicted and actual boxes, adjusted using a hyperbolic tangent function. It considers not only the size of the overlapping region but also the area of ​​the actual box, ensuring that larger defects receive greater attention. The covariance matrix describes the shape and orientation of the distribution, reflecting the shape difference between the predicted and actual boxes. A larger shape difference between the two boxes results in a larger difference in their covariance matrices, leading to a larger Wasserstein distance.

[0062] It should be noted that the geometric alignment between the predicted and ground truth defect locations is evaluated by combining Wasserstein distance and area-weighted intersection-union ratio (AWIoU). Specifically, Wasserstein distance measures the shape and positional differences between the predicted and ground truth bounding boxes, while AWIoU assesses the degree of overlap and weights the area of ​​the ground truth bounding box. This method comprehensively considers both the accuracy and shape differences of the defect location, thereby more precisely adjusting the model and making defect localization more accurate. It provides better localization performance, especially when dealing with irregularly shaped defects, thus improving the accuracy of detection results.

[0063] S404: If the defect category recognition accuracy is lower than the preset defect category recognition accuracy, adjust the hyperparameters of the defect detection module and retrain the defect detection module. If the bimodal localization loss value is lower than the preset bimodal localization loss value, adjust the hyperparameters of the defect localization module and retrain the defect localization module.

[0064] Specifically, by combining Wasserstein distance and area-weighted intersection-union ratio (AWIoU), the geometric alignment between the predicted defect location and the actual defect location is evaluated, thereby optimizing the defect localization accuracy. If the accuracy of the detection results is lower than a preset threshold, the model is retrained by adjusting the hyperparameters of the corresponding modules. The advantage of this method is that it comprehensively considers the differences in the shape and position of the defect, enabling more accurate defect localization, especially when dealing with irregular or complex defects, significantly improving the localization accuracy and the accuracy of the detection results.

[0065] S5: Acquire real-time images of power devices.

[0066] S6: Input the real-time image of the power device into the trained decoupled detection model, and output the defect category and defect location of the real-time image of the power device.

[0067] In one possible implementation, the defect detection module is specifically a random forest module, and the defect localization module is specifically a YOLO module. S6 specifically includes:

[0068] S601: Receives real-time images of power devices through the input layer.

[0069] S602: Extract salient feature maps from real-time images of power devices using the feature extraction module.

[0070] In one possible implementation, the feature extraction module further includes a multi-channel backbone network, wherein the multi-channel backbone network includes ResNeSt, HRNet, and MobileNet, and the output of the multi-channel backbone network is the input of the CBAM salient feature extraction unit.

[0071] In one possible implementation, S602 specifically includes:

[0072] S6021: Extract the original feature map of the real-time image of the power device through the multi-channel backbone network, and input the original feature map into the CBAM salient feature extraction unit.

[0073] S6022: Perform differentiation and Sobel convolution on the original feature map of each channel to construct the differential gradient tensor and differential energy map.

[0074] ; ; in, Represents the original feature map of channel c The gradient intensity at coordinates (x, y), and Let x and y represent the first-order partial derivatives of the original feature map of channel c in the x and y directions, respectively, and Sobel denotes the Sobel operator. Represents the original feature map of channel c The differential energy diagram, where ln represents the natural logarithm. express Activation function Gradient plot The gradient intensity at the pixel at position (i, j) express The Sobel operator response value of the pixel at position (i, j).

[0075] The differential operation is used to calculate the rate of change at a specific location in the image. The Sobel operator is an edge detection operator that extracts edge features from an image through convolution. The gradient intensity is the square root of the sum of the squares of the gradient values ​​at a point in the image along the x and y directions, used to measure the degree of change at that point. The differential energy map obtains the energy information of each pixel in the image by weighted summing of the gradient intensity and the Sobel operator response value, reflecting the local texture features in the image. Softplus is an activation function used to smoothly transform the energy values.

[0076] It should be noted that by performing differentiation and Sobel convolution on the original feature maps of each channel, a differential gradient tensor and a differential energy map are constructed. Differentiation helps calculate the gradient intensity of the image, Sobel convolution extracts edge information, and the differential energy map enhances the image's detailed features by weighting the gradient intensity. This process helps extract texture and edge features from the image, providing rich local information for subsequent defect detection. Especially when dealing with defects with details and edges, it can significantly improve the model's detection capability and accuracy.

[0077] S6023: Perform a Fast Fourier Transform on the original feature map of channel c to obtain the amplitude spectrum and phase spectrum.

[0078] S6024: Modulation of the phase spectrum using differential energy maps to enhance salient features: ; in, This indicates that the original feature map is included. Corresponding frequency domain coordinates Phase spectrum at phase, This represents the modulated phase spectrum. Represents the phase modulation coefficient. Indicates the adjustable sampling area. Let N represent the arctangent function, and N represent the original feature map. The total number of pixels, Represents pi (π). This represents the Fourier kernel function.

[0079] Optionally, the phase modulation coefficient can be in the range of 0.3 to 0.5 to control the modulation intensity.

[0080] In this context, the phase spectrum represents the phase information of an image in the frequency domain, specifically the phase angles of its frequency components. Phase modulation enhances or suppresses specific features of an image by adjusting the phase of its frequency components. By modulating the phase spectrum using a differential energy map, salient features in the image are enhanced. Specifically, the differential energy map provides information about the local texture of the image, while the phase spectrum describes the phase components in the frequency domain. By modulating the phase spectrum, key features of the image are enhanced, particularly when dealing with defects with details or edges. This method helps to improve key features of the image, making defects more prominent, thereby enhancing the accuracy of subsequent detection and localization. The advantage lies in the fact that strengthening salient features through phase modulation can effectively improve the robustness and detection accuracy of the model in complex scenes.

[0081] S6025: The amplitude spectrum, the modulated phase spectrum, the original feature map, and the differential energy map are fused to obtain a salient feature map.

[0082] ; in, Represents a salient feature map. Indicates the inverse Fourier transform. This represents depthwise separable convolution. express Activation function This represents element-wise multiplication. The normalization of the layer is represented by A, which represents the amplitude spectrum.

[0083] Specifically, the original feature maps of real-time images of power devices are extracted using a multi-channel backbone network, and the edge and texture features of the images are enhanced using differential operations and Sobel convolution. Next, a Fast Fourier Transform is performed on the image to obtain the phase spectrum, and the phase spectrum is modulated using a differential energy map to further enhance salient features. Finally, a salient feature map is generated by fusing the amplitude spectrum, the modulated phase spectrum, the original feature map, and the differential energy map. Through differential operations, Sobel convolution, and phase modulation, the local details and features of the image can be effectively improved, especially when dealing with complex or small defects, significantly improving detection accuracy and robustness.

[0084] S603: Input the salient feature map into the random forest module and output the defect category.

[0085] S604: Input the salient feature map into the defect localization module and output the defect location.

[0086] Specifically, this process involves inputting real-time images of the power device into the model. First, a feature extraction module extracts salient feature maps. Then, these salient feature maps are fed into a random forest module for defect category identification, and finally, a YOLO module for precise defect location. Random forest provides stable classification capabilities, while YOLO efficiently locates defect positions, thus enabling efficient and accurate detection and localization of defects in power devices.

[0087] In one possible implementation, after S6, the following is also included: The decoupled detection model is retrained at preset intervals.

[0088] It should be noted that those skilled in the art can set the preset duration according to actual needs, and this invention does not limit this.

[0089] In practical applications, the entire defect detection process begins with acquiring an image set labeled with defects. This image set is then expanded by combining power device material properties and geometric modeling, enhancing the diversity and generalization ability of the training data. Next, a decoupled detection model is constructed and trained using a bimodal localization loss function, improving the accuracy of defect category recognition and location. Finally, real-time images are acquired and input into the trained model to achieve automatic detection and localization of defects in real-time images. Through precise model training and data expansion, the accuracy and robustness of detection are improved, effectively solving the problem of missed detections and improving the production quality and efficiency of power devices.

[0090] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0091] In this embodiment of the invention, by combining the material properties and geometric modeling of power devices to expand the image set, the problem that existing training sets cannot cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data but also improves the model's generalization ability, reducing the probability of missed detections in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect positions, improving detection accuracy and efficiency. By combining a bimodal localization loss function, the accuracy of defect localization is optimized, further improving the performance of power device defect detection, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detections in the production process, and improving production quality and efficiency.

[0092] Reference manual attached Figure 3 The diagram shows a structural schematic of a power device defect detection system provided by the present invention.

[0093] The present invention also provides a power device defect detection system 20, applied to the above-mentioned power device defect detection method, comprising: Processor 201.

[0094] The memory 202 stores computer-readable instructions, which, when executed by the processor 201, implement the power device defect detection method as described in the method embodiment.

[0095] The power device defect detection system 20 provided by the present invention can perform the above-described power device defect detection method and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate further.

[0096] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, by combining the material properties and geometric modeling of power devices to expand the image set, the problem that existing training sets cannot cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data but also improves the model's generalization ability, reducing the probability of missed detections in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect positions, improving detection accuracy and efficiency. By combining a bimodal localization loss function, the accuracy of defect localization is optimized, further improving the performance of power device defect detection, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detections in the production process, and improving production quality and efficiency.

[0097] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0098] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0099] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0100] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0101] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0102] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0105] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0108] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the power device defect detection method as described in the method embodiment.

[0110] The present invention provides a computer-readable storage medium that can implement the steps and effects of the power device defect detection method in the above-described method embodiments. To avoid repetition, the present invention will not repeat the details.

[0111] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, by combining the material properties and geometric modeling of power devices to expand the image set, the problem that existing training sets cannot cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data but also improves the model's generalization ability, reducing the probability of missed detections in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect positions, improving detection accuracy and efficiency. By combining a bimodal localization loss function, the accuracy of defect localization is optimized, further improving the performance of power device defect detection, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detections in the production process, and improving production quality and efficiency.

[0112] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0113] The following points need to be explained:

[0114] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0115] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0116] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0117] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting defects in power devices, characterized in that, include: S1: Obtain a set of defect images of power devices with defect tags; The power device defect image set includes multiple power device defect images; S2: Combining the material properties of power devices, the defect image set of the power devices is expanded through geometric modeling; S3: Construct a decoupled detection model for identifying defect categories and locating defect positions; S4: Combine the dual-modal localization loss function and use the expanded power device defect image set to train the decoupled detection model; S5: Acquire real-time images of power devices; S6: Input the real-time image of the power device into the trained decoupled detection model, and output the defect category and defect location of the real-time image of the power device; S2 specifically includes: S201: Extract grayscale images of power device defect images corresponding to different defect labels in the power device defect image set; S202: Based on the grayscale image, construct a differential geometric manifold representation of the defect features with curvature and Gaussian geometric curvature tensors to capture the line defect primitives and point defect primitives of the power device; The specific formula for constructing the differential geometric manifold representation is as follows: ； in, Representing the differential geometric manifold representation, This represents the determinant of the Hessian matrix describing the Gaussian geometric curvature tensor. The curvature represents the degree of bending at the edge of the defect. Represents the pixel coordinates of a power device defect image. grayscale value at that location This represents the grayscale gradient of a power device defect image. The second derivative of the grayscale value of a power device defect image is the Hessian matrix. The gradient magnitude describes the magnitude of the gradient change. Represents the divergence operator. Represents the determinant operator; S203: Determine the defect propagation direction angle of the defect image of the power device; The formula for the defect propagation direction angle is as follows: ； in, Indicates the direction angle of defect propagation. and These represent the gray-level gradients in the x and y directions of the power device defect image, respectively. Represents the arctangent function. This indicates that the mean value is calculated within the corresponding local window. S204: Based on the defect propagation direction angle, and in conjunction with the Navier-Cauchy equation, determine the stochastic transformation field describing the defect propagation law; The formula for the random transformation field is as follows: ； ； ； in, Represents the image of power device defects in coordinates The random deformation field at that location, Indicates the deformation scaling factor. This indicates the shear modulus of the material corresponding to the detection area of ​​the power device. Indicates the angle of defect propagation direction. The relevant rotation tensor, and Let them represent the sine function and the cosine function, respectively. This represents the first Lamé constant, i.e., the volume change resistance, of the material corresponding to the detection region of the power device. Representing coordinates The basic displacement field at the location, The random coefficients of the frequency term k are represented by e, and e represents the natural constant. Indicates the angle related to k along the defect propagation direction. Plane waves in the direction of Represents coordinates that conform to a Gaussian distribution The random perturbation term at the location, This represents the gradient operator. Represents the second derivative operator. The linear elastic equilibrium equations in the Navier-Cauchy equations for the absence of body forces; S205: Combining the differential geometric manifold representation, under the constraint of the projection consistency constraint of the random transformation field, generate power device defect images to complete the expansion of the power device defect image set; The specific formula for generating the defect image of the power device is as follows: ； in, This represents the generated image of power device defects. This represents the generated defect image of the candidate power device. This represents the total variation term that ensures image smoothness. Indicates the weight of the smoothing term. This indicates taking the minimum function value. , This represents the differential geometric manifold representation after a randomized field transformation. This indicates element-wise multiplication. Indicates a mask for the defect area. express Projection image under X-ray imaging, This represents the calculation of the square of the Frobenius norm; The decoupled detection model includes an input layer, a feature extraction module, a defect detection module, and an output layer connected in sequence. The decoupled detection model also includes a defect localization module, which is connected to the feature extraction module and the output layer. The feature extraction module has a CBAM salient feature extraction unit based on dynamic differential feature generation and phase modulation functions. Specifically, the defect detection module is a random forest module, and the defect localization module is a YOLO module; S6 specifically includes: S601: Receive the real-time image of the power device through the input layer; S602: Extract the salient feature map of the real-time image of the power device through the feature extraction module; S603: Input the salient feature map into the random forest module and output the defect category; S604: Input the salient feature map into the defect localization module and output the defect location; The feature extraction module further includes a multi-channel backbone network, which includes ResNeSt, HRNet, and MobileNet. The output of the multi-channel backbone network is the input of the CBAM salient feature extraction unit. Specifically, S602 includes: S6021: Extract the original feature map of the real-time image of the power device through the multi-channel backbone network, and input the original feature map into the CBAM salient feature extraction unit; S6022: Perform differentiation and Sobel convolution on the original feature map of each channel to construct the differential gradient tensor and differential energy map; S6023: Perform a fast Fourier transform on the original feature map of channel c to obtain the amplitude spectrum and phase spectrum; S6024: Modulate the phase spectrum using the differential energy map to enhance salient features; S6025: The amplitude spectrum, the modulated phase spectrum, the original feature map, and the differential energy map are fused to obtain the salient feature map.

2. The power device defect detection method according to claim 1, characterized in that, The defect labels include crack labels, bubble labels, poor solder joint labels, and pin deformation labels.

3. The power device defect detection method according to claim 1, characterized in that, S4 specifically includes: S401: Input the power device defect images from the expanded power device defect image set into the decoupled detection model in sequence to obtain the predicted defect category output by the defect detection module and the predicted defect location output by the defect localization module. S402: Calculate the defect category identification accuracy of the predicted defect category; S403: Calculate the bimodal localization loss value describing the geometric alignment relationship of the predicted defect location; S404: If the defect category recognition accuracy is less than the preset defect category recognition accuracy, adjust the hyperparameters of the defect detection module and retrain the defect detection module; if the bimodal localization loss value is less than the preset bimodal localization loss value, adjust the hyperparameters of the defect localization module and retrain the defect localization module.

4. The power device defect detection method according to claim 1, characterized in that, Following S6, it also includes: The decoupled detection model is retrained at preset intervals.

5. A power device defect detection system, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the power device defect detection method as described in any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the power device defect detection method as described in any one of claims 1 to 4.

Images