Power device defect detection method and system

Abstract

The invention provides a power device defect detection method and system, and relates to the technical field of semiconductors, and the method comprises the steps: obtaining a power device defect image set with defect tags; expanding a defect image set of the power device through geometric modeling in combination with material attributes of the power device; constructing a decoupling detection model used for identifying defect types and positioning defect positions; in combination with a bimodal positioning loss function, training a decoupling detection model by using the expanded power device defect image set; collecting a real-time image of the power device; and inputting the real-time image of the power device into the trained decoupling detection model, and outputting the defect category and the defect position of the real-time image of the power device. Through accurate model training and data expansion, the accuracy and robustness of detection are improved, the problem of missing detection can be effectively solved, and the production quality and efficiency of power devices are improved.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor technology, and in particular to a power device defect detection method and system. Background Art

[0002] Power devices refer to semiconductor devices used to control and convert electrical energy in power electronic systems, usually including diodes, transistors, three-terminal devices (such as IGBT), etc. They play a key role in regulating voltage, current and power conversion in power systems. The performance of power devices directly affects the efficiency, stability and life of power electronic equipment.

[0003] Defect detection of power devices is to detect and identify appearance defects that may occur during the manufacturing process to ensure that the power devices can work reliably during normal operation. Since power devices are subjected to high current and high temperature environments, any tiny appearance defect may cause serious electrical failures, system failures, or even safety accidents. Therefore, timely and effective appearance defect detection is crucial to ensure the quality and reliability of power devices. This can not only reduce the equipment failure rate and increase the service life of the equipment, but also ensure the safe operation and high efficiency of the power system.

[0004] With the rapid development of artificial intelligence, defect detection of power devices has gradually replaced manual detection. However, the current power device defect training set is often unable to be effectively and accurately expanded, resulting in insufficient generalization ability of power device defect detection based on artificial intelligence, and a large number of missed detection problems in actual application, which greatly affects the production quality and efficiency of power devices. Summary of the invention

[0005] In order to solve the technical problem that the power device defect training set in the prior art is often unable to be expanded effectively and accurately, resulting in insufficient generalization ability of power device defect detection based on artificial intelligence, a large number of missed detection problems occur in actual applications, and the production quality and efficiency of power devices are greatly affected, the present invention provides a power device defect detection method and system.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect: An embodiment of the present invention provides a power device defect detection method, comprising: S1: Obtain a set of power device defect images with defect labels; S2: Combined with the material properties of power devices, the power device defect image set is expanded through geometric modeling; S3: Build a decoupled detection model for identifying defect categories and locating defect locations; S4: Combined with the dual-modal localization loss function, the decoupled detection model is trained using the expanded power device defect image set; S5: Collect real-time images of power devices; S6: Input the real-time image of the power device into the trained decoupling detection model, and output the defect category and defect location of the real-time image of the power device.

[0008] Second aspect: An embodiment of the present invention provides a power device defect detection system, comprising: processor; A memory having computer-readable instructions stored therein, wherein when the computer-readable instructions are executed by the processor, the power device defect detection method according to the first aspect is implemented.

[0009] The third aspect: An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the power device defect detection method according to the first aspect is implemented.

[0010] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: In an embodiment of the present invention, by combining the material properties of power devices with geometric modeling to expand the image set, the problem that the existing training set is difficult to cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data, but also improves the generalization ability of the model, reducing the probability of missed detection in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect locations, improving the accuracy and efficiency of detection. By combining the dual-modal positioning loss function, the accuracy of defect positioning is optimized, and the performance of power device defect detection is further improved, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detection in the production process, and improving production quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of a flow chart of a power device defect detection method provided by an embodiment of the present invention;

[0013] Figure 2 A schematic diagram of the structure of a decoupling detection model provided by an embodiment of the present invention;

[0014] Figure 3 A schematic structural diagram of a power device defect detection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0016] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0017] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0018] Reference Manual Attached Figure 1 , showing a schematic flow chart of a power device defect detection method provided by an embodiment of the present invention.

[0019] An embodiment of the present invention provides a power device defect detection method, which can be implemented by a power device defect detection device, and the power device defect detection device can be a terminal or a server. The processing flow of the power device defect detection method can include the following steps:

[0020] S1: Obtain a set of power device defect images with defect labels.

[0021] It can be understood that the acquired power device defect image set with defect labels provides basic data for subsequent defect detection. These image sets contain different types of defects, and each image is annotated to provide a clear defect category and location reference for the training model, thereby ensuring the accuracy and effectiveness of the detection model.

[0022] In a possible implementation, the defect labels include crack labels, bubble labels, cold solder joint labels, and pin deformation labels.

[0023] It should be noted that defect labels include crack labels, bubble labels, cold solder labels and pin deformation labels, which are used to identify different types of defects in power devices. By labeling each image, it can help the training model identify and locate various types of defects, thereby improving the classification and location accuracy of the detection model for different defects and ensuring the accuracy of the detection results.

[0024] S2: Combined with the material properties of power devices, the power device defect image set is expanded through geometric modeling.

[0025] It should be noted that the original power device defect image set is expanded by combining the material properties and geometric modeling of power devices. Specifically, the properties of different materials used in power devices (such as silicon, silicon carbide, etc.) are first considered, such as their thermal properties, electrical conductivity, etc. These material properties will affect the formation and development of defects. On this basis, the geometric modeling method is used to construct simulated defect images, including defects such as cracks and bubbles. By calculating the defect modes that may be generated by different materials under various stress and temperature conditions, more types of defect images are generated, thereby ensuring that the expanded images conform to the actual physical properties of the power device while obtaining an expanded image set. Such an expansion not only allows the training set to cover more defect types, but also simulates the impact of changes in material properties on defect morphology, improves the generalization ability of the model, enables it to better adapt to different materials and defect scenarios, and improves the accuracy and robustness of defect detection.

[0026] In a possible implementation manner, the power device defect image set includes a plurality of power device defect images. S2 specifically includes:

[0027] S201: Extracting grayscale images of power device defect images corresponding to different defect labels in a power device defect image set.

[0028] Among them, grayscale image is the conversion of color image into grayscale image, in which the value of each pixel represents brightness without color information. Grayscale image is used to simplify image processing, especially when detecting and analyzing defects, it can focus on the shape and structure of the image.

[0029] S202: Based on the grayscale image, a differential geometric manifold representation of the defect feature with curvature and Gaussian geometric curvature tensor is constructed to capture line defect primitives and point defect primitives of the power device.

[0030] Among them, the curvature captures the line defect primitives, and the Gaussian geometric curvature tensor captures the point defect primitives.

[0031] The differential geometry manifold representation construction formula is as follows: ; in, represents the differential geometry manifold representation, represents the Hessian matrix determinant describing the Gaussian geometric curvature tensor, Indicates the curvature that reflects the degree of curvature of the defect edge. Represents the pixel coordinates of the power device defect image The gray value at Represents the grayscale gradient of the power device defect image, The second-order derivative of the grayscale of the power device defect image is the Hessian matrix. Represents the gradient modulus that describes the magnitude of the gradient change, represents the divergence operator, Represents the determinant operator.

[0032] Among them, differential geometry manifold is a mathematical method to describe the local geometric structure of points in an image. By analyzing the gradient and second-order derivative of the grayscale value of the image, information about the curvature and Gaussian geometric curvature in the image can be obtained. These features help capture the defect morphology in the image, especially line defects (such as cracks) and point defects (such as bubbles). Curvature is used to describe the degree of curvature of the edge or surface of the image at a specific point. Here, curvature is used to capture line defects, that is, elongated defects in the image. Gaussian geometric curvature tensor is used to capture the characteristics of point defects. It evaluates the local curvature of the image by calculating the second-order derivative of the image (i.e., the Hessian matrix) to identify defects like bubbles or other similar defects.

[0033] Specifically, Indicates the calculation of the normalized gradient direction That is, the divergence trend of the unit vector field.

[0034] It should be noted that by extracting the grayscale image of the image and using the geometric modeling method to construct a differential geometric manifold representation containing curvature and Gaussian geometric curvature tensor. This representation can effectively capture the geometric features of defects in the image, especially line defects and point defects. By analyzing the grayscale gradient and second-order derivative of the image, the model can accurately identify the morphology of the defect, and then simulate the diverse manifestations of different defects, especially those defect types that are difficult to obtain in the actual image set. By combining material properties with geometric characteristics, more defect samples are generated to ensure that the training data is more representative. Through such expansion, not only the training set is more diverse, but also the adaptability of the model can be enhanced, and the accuracy and robustness of defect detection can be improved.

[0035] S203: Determine the defect extension direction angle of the power device defect image.

[0036] The formula for the defect extension direction angle is as follows: ; in, represents the defect extension direction angle, and They represent the grayscale gradient in the x direction and the grayscale gradient in the y direction of the power device defect image, respectively. represents the inverse tangent function, Indicates averaging within the corresponding local window.

[0037] It should be noted that the reason for calculating the defect extension direction angle is that the growth direction of the crack is not random, but is constrained by stress distribution and crystal orientation. The principal stress direction is often consistent or similar to the actual crack direction. If the direction restriction is ignored during generation, non-physical crack bending or diffusion will occur, affecting the simulation authenticity. Introducing the defect extension direction angle can effectively control the defect to be more in line with the "orientation" distribution in real scenes.

[0038] S204: Based on the defect extension direction angle, a random transformation field describing the defect extension law is determined in combination with the Navier-Cauchy equation.

[0039] The formula form of the random transformation field is specifically: ; ; ; in, Indicates the defect image of the power device at coordinates The random deformation field at represents the deformation scaling factor, Indicates the shear modulus of the material corresponding to the power device detection area, Indicates the angle with the defect extension direction The associated rotation tensor, and represent the sine function and cosine function respectively, Indicates the first Lamé constant of the material corresponding to the power device detection area, i.e., the resistance to volume change. Representing coordinates The basic displacement field at represents the random coefficient of the frequency term k, e represents the natural constant, represents the angle along the defect extension direction related to k Directional plane waves, Represents coordinates that conform to a Gaussian distribution The random disturbance term at represents the gradient operator, represents the second-order derivative operator, It represents the linear elastic equilibrium equation in the absence of body forces in the Navier-Cauchy equation.

[0040] Among them, the Navier-Cauchy equation describes the elastic mechanical equilibrium of the material under force-free conditions and is used to describe the relationship between stress and deformation. The role of the Navier-Cauchy equation is to help simulate the stress and deformation process when the defect expands by describing the elastic properties of the material (such as shear modulus and volume change resistance). The random transformation field is used to simulate the expansion and deformation of the defect, taking into account the physical behavior of the defect. Through this transformation field, the defect can be randomly deformed according to the specified extension direction and stress characteristics to generate a new defect image. The rotation tensor is used to rotate the deformation according to the defect extension direction angle so that the defect extension direction conforms to the actual physical conditions. The basic displacement field is used to describe the displacement or deformation of the defect in the image. The perturbation term of the Gaussian distribution simulates the noise or irregularity in the image, conforms to the Gaussian distribution, increases the diversity of the generated image, and makes the defect expansion more natural.

[0041] It should be noted that by combining the defect extension direction angle and the Navier-Cauchy equation, a random transformation field describing the defect extension law is constructed. In this process, the rotation angle is first determined according to the defect extension direction, and then the stress distribution of the material and the defect extension process are simulated by the transformation field. Through this method, a more realistic defect extension image can be generated to simulate the effects of different stresses and deformations on defects. This method can generate a variety of defect samples through physical modeling and random perturbations, which not only enhances the diversity of the data set, but also improves the adaptability and accuracy of the model to different types of defects, thereby effectively improving the performance and robustness of defect detection.

[0042] S205: In combination with the differential geometry manifold representation, under the constraints of the projection consistency constraints on the random transformation field, a power device defect image is generated to complete the expansion of the power device defect image set.

[0043] The formula for generating the power device defect image is as follows: ; in, represents the generated power device defect image, represents the generated candidate power device defect image, represents the total variation term that ensures image smoothness, represents the weight of the smoothing term, It means taking the minimum function value , represents the differential geometry manifold representation after the random transformation field transformation, represents element-wise multiplication, represents the defect area mask, express Projection diagram under X-ray imaging, Indicates calculation of the square of the Frobenius norm.

[0044] in, The projected performance is compared to the other items. This makes the imaging in the industrial inspection imaging system, namely XCT, consistent with the deformed structure, thereby ensuring that it can simulate the appearance of defects photographed in reality, rather than just looking like it. The projection consistency constraint ensures that the generated image has a consistent performance with the transformed geometric manifold under X-ray projection, ensuring that the structure and physical properties of the generated image truly reflect the morphology of the defect. Total variation is a mathematical method for reducing image noise. It helps smooth the image so that the edges in the image are not affected by over-smoothing while maintaining structural details. Masks are used to specify defect areas in the image to ensure that smoothing and other processing are performed only in the defect area.

[0045] Optionally, the smoothing term weight can be set to 0.5.

[0046] It should be noted that by introducing projection consistency constraints when generating defect images of power devices, it is ensured that the performance of the generated defect images under X-ray imaging is consistent with the real defect images. The structure of the generated image is optimized by comparing the generated image with its X-ray projection and transformed geometric manifold representation. The total variation term helps smooth the image and avoid noise interference. This method can not only generate more realistic defect images, but also ensure that they have high physical consistency and detectability in actual industrial inspections, improve the model's ability to recognize defects, and effectively reduce noise and unnatural artifacts in the generation process.

[0047] Specifically, by combining differential geometry manifold representation and random transformation field, the projection consistency constraint is used to generate power device defect images. By comparing the difference between the generated image and the X-ray projection and geometric manifold, the image structure is optimized to ensure that the generated defect image is consistent under X-ray imaging, thereby improving physical consistency. The total variation term effectively smoothes the image and reduces noise interference, making the generated image more realistic in actual detection, enhancing the robustness and defect recognition ability of the model, and reducing the generation of artifacts.

[0048] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a decoupling detection model provided by an embodiment of the present invention.

[0049] S3: Construct a decoupled detection model for identifying defect categories and locating defect locations.

[0050] It should be noted that the decoupled detection model can simultaneously identify the defect category and accurately locate the defect location. The decoupled design allows the model to independently handle the defect classification task and location task, thereby improving the efficiency and accuracy of detection. In this way, the model can accurately determine the specific location of the defect in the image while identifying the defect type.

[0051] In a possible implementation, the decoupling detection model includes an input layer, a feature extraction module, a defect detection module and an output layer connected in sequence, wherein the decoupling detection model also includes a defect location module, which is respectively connected to the feature extraction module and the output layer, and the feature extraction module has a CBAM significant feature extraction unit based on a dynamic differential feature generation function and a phase modulation function.

[0052] The input layer is used to receive the image data of the power device as the initial input of the model. The task of the feature extraction module is to extract meaningful features from the input image. This module uses the CBAM (Convolutional Block Attention Module) salient feature extraction unit based on dynamic differential feature generation and phase modulation. The CBAM module helps the model focus on key parts of the image, such as defect areas, by adaptively enhancing the salient features of important areas while suppressing unimportant areas. Dynamic differential feature generation can enhance the capture of edges and textures in the image, while phase modulation further enhances the salient features in the image by adjusting the phase information. The defect detection module is responsible for identifying the defect type based on the features provided by the feature extraction module and accurately locating the defect through an algorithm. The output layer generates the final detection output based on the processing results of the defect detection module.

[0053] It should be noted that by decoupling the defect category identification and location tasks, the accuracy of each task is effectively improved. The CBAM salient feature extraction unit introduced in the feature extraction module dynamically enhances the saliency of the defect area, allowing the model to more accurately capture the defect features in the image. Through this design, the model can focus more on key features, improve the accuracy and efficiency of defect detection, especially when dealing with complex and small defects.

[0054] S4: Combined with the dual-modal localization loss function, the decoupled detection model is trained using the expanded power device defect image set.

[0055] It should be noted that by combining the dual-modal positioning loss function and using the expanded power device defect image set to train the decoupled detection model, it is possible to more accurately complete defect positioning and defect type monitoring, thereby improving the accuracy and robustness of the model in real applications. In this way, the model can better handle the diversity of defects, improve positioning accuracy, and ensure efficient and accurate detection results.

[0056] In a possible implementation, S4 specifically includes:

[0057] S401: inputting the power device defect images in the expanded power device defect image set into the decoupling detection model in sequence to obtain the predicted defect category output by the defect detection module and the predicted defect position output by the defect location module.

[0058] S402: Calculate the defect category recognition accuracy of the predicted defect category.

[0059] S403: Calculate a bimodal positioning loss value describing a geometric alignment relationship of the predicted defect position.

[0060] The calculation formula of the dual-modal positioning loss value is as follows: ; ; ; ; in, represents the bimodal positioning loss value, represents the normalized Wasserstein distance between the predicted defect location and the actual defect location. represents the weighted intersection-and-union ratio between the predicted defect location and the actual defect location, exp represents the natural exponential function, represents the two-dimensional Gaussian distribution of the prediction box obtained based on the predicted defect position modeling, represents the real box two-dimensional Gaussian distribution obtained by modeling the real defect position corresponding to the power device defect image, and C represents the square of the width of the power device defect image. and the square of the power device defect image height The associated normalization constant to prevent the loss value from getting out of control, Represents two two-dimensional Gaussian distributions and The square of the Wasserstein distance between express The center point position vector, express The center point position vector, Indicates the calculation of the square of the Euclidean distance, and Respectively and The covariance matrix of represents the calculation matrix trace, and Represent the geometric boundaries of the predicted box and the geometric boundaries of the real box respectively, Represents the geometric boundary of the real box The area of ​​the enclosed real box, It represents the median of the real box area of ​​the power device defect image in the power device defect image set, represents the hyperbolic tangent function.

[0061] Among them, the bimodal positioning loss value combines the Wasserstein distance (NWD) and the area-weighted intersection-over-union (AWIoU) to measure the difference between the predicted defect location and the actual defect location. The goal is to minimize the loss and improve the positioning accuracy. Wasserstein distance is a method to measure the difference between two distributions. AWIoU measures the ratio of the intersection area and the union area of ​​the predicted box and the real box, and adjusts it through the hyperbolic tangent function. It not only considers the size of the overlapping area, but also weights the area of ​​the real box to ensure that defects with larger areas receive higher attention. The covariance matrix describes the shape and direction of the distribution. It reflects the shape difference between the predicted box and the real box. If the shapes of the two boxes are very different, their covariance matrices will be different, so the Wasserstein distance will be larger.

[0062] It should be noted that the geometric alignment between the predicted defect position and the true defect position is evaluated by combining the Wasserstein distance and the area-weighted intersection-over-union ratio. Specifically, the Wasserstein distance measures the shape and position differences between the predicted box and the true box, while AWIoU evaluates the overlap between the two and weights the area of ​​the true box. This method can comprehensively consider the accuracy of the defect position and the shape difference, thereby adjusting the model more accurately and making the defect location more accurate. In particular, when dealing with irregularly shaped defects, it provides better positioning performance and improves the accuracy of the detection results.

[0063] S404: When the defect category recognition accuracy is less than the preset defect category recognition accuracy, adjust the hyperparameters of the defect detection module and retrain the defect detection module. When the dual-modal positioning loss value is less than the preset dual-modal positioning loss value, adjust the hyperparameters of the defect positioning module and retrain the defect positioning module.

[0064] Specifically, by combining the Wasserstein distance and the area-weighted intersection-over-union (AWIoU), the geometric alignment between the predicted defect position and the actual defect position is evaluated to optimize the defect location accuracy. If the accuracy of the detection result is lower than the preset threshold, the model is retrained by adjusting the hyperparameters of the corresponding module. The advantage of this method is that it can locate defects more accurately by comprehensively considering the differences in the shape and position of the defects, especially when dealing with irregular or complex defects, which significantly improves the positioning accuracy and the accuracy of the detection results.

[0065] S5: Collect real-time images of power devices.

[0066] S6: Input the real-time image of the power device into the trained decoupling detection model, and output the defect category and defect location of the real-time image of the power device.

[0067] In a possible implementation, the defect detection module is specifically a random forest module, and the defect location module is specifically a yolo module. S6 specifically includes:

[0068] S601: Receive real-time images of power devices through an input layer.

[0069] S602: Extracting a significant feature map of the real-time image of the power device through a feature extraction module.

[0070] In a possible implementation, the feature extraction module further includes a multi-channel backbone network, wherein the multi-channel backbone network includes ResNeSt, HRNet and MobileNet, and the output of the multi-channel backbone network is the input of the CBAM significant feature extraction unit.

[0071] In a possible implementation manner, S602 specifically includes:

[0072] S6021: Extract the original feature map of the real-time image of the power device through the multi-channel backbone network, and input the original feature map into the CBAM significant feature extraction unit.

[0073] S6022: Perform differentiation operation and Sobel convolution on the original feature map of each channel to construct the differential gradient tensor and differential energy map.

[0074] ; ; in, Represents the original feature map of channel c The gradient strength at coordinate (x, y), and They represent the first-order partial derivatives of the original feature map of channel c in the x and y directions respectively, Sobel represents the Sobel operator, Represents the original feature map of channel c Differential energy diagram of , ln represents the natural logarithm, express Activation function, Represents a gradient map The gradient strength of the pixel at position (i, j), express The Sobel operator response value of the pixel at position (i, j) in .

[0075] Among them, the differential operation is used to calculate the rate of change of a specific position in the image. The Sobel operator is an edge detection operator that extracts the edge features of the image through convolution operations. The gradient strength is the square root of the sum of the squares of the gradient values ​​of a point in the image in the x and y directions, which is used to measure the degree of change of the point. The differential energy map obtains the energy information of each pixel in the image by weighted summation of the gradient strength and the Sobel operator response value, reflecting the local texture features in the image. Softplus is an activation function used to smoothly convert the energy value.

[0076] It should be noted that the differential gradient tensor and differential energy map are constructed by performing differential operations and Sobel convolution on the original feature map of each channel. The differential operation helps calculate the gradient strength of the image, the Sobel convolution extracts the edge information, and the differential energy map strengthens the detailed features of the image by weighting the gradient strength. This process helps to extract the texture and edge features in the image, providing rich local information for subsequent defect detection, especially when dealing with defects with details and edges, which can significantly improve the detection ability and accuracy of the model.

[0077] S6023: Perform fast Fourier transform on the original feature map of channel c to obtain the amplitude spectrum and phase spectrum.

[0078] S6024: Modulation of the phase spectrum by differential energy map to enhance salient features: ; in, Representation includes the original feature map Corresponding frequency domain coordinates The phase spectrum at the phase, represents the phase spectrum after modulation, represents the phase modulation coefficient, represents the adjustable sampling area, represents the inverse tangent function, and N represents the original feature map The total number of pixels, represents pi, represents the Fourier kernel function.

[0079] Optionally, the phase modulation coefficient ranges from 0.3 to 0.5 to control the modulation intensity.

[0080] Among them, the phase spectrum represents the phase information of the image in the frequency domain, that is, the phase angle of the image frequency component. Phase modulation is to enhance or suppress specific features of the image by adjusting the phase of the image frequency component. By modulating the phase spectrum using the differential energy map, the salient features in the image are enhanced. Specifically, the differential energy map provides information about the local texture of the image, while the phase spectrum is used to describe the phase component in the frequency domain. By modulating the phase spectrum, the key features of the image are enhanced, especially when dealing with defects with details or edges. This method helps to enhance the key features of the image, making the defects more prominent, thereby enhancing the accuracy of subsequent detection and positioning. The advantage is that by strengthening the salient features through phase modulation, the robustness and detection accuracy of the model in complex scenes can be effectively improved.

[0081] S6025: Fusing the amplitude spectrum, the modulated phase spectrum, the original feature map and the differential energy map to obtain a significant feature map.

[0082] ; in, represents a salient feature map, represents the inverse Fourier transform, represents depthwise separable convolution, express Activation function, represents element-wise multiplication, represents layer normalization, and A represents the amplitude spectrum.

[0083] Specifically, the original feature map of the real-time image of the power device is extracted through a multi-channel backbone network, and the edge and texture features of the image are enhanced by differential operations and Sobel convolution. Then, the image is fast Fourier transformed to obtain the phase spectrum, and the phase spectrum is modulated using the differential energy map to further enhance the significant features. Finally, the significant feature map is generated by fusing the amplitude spectrum, the modulated phase spectrum, the original feature map and the differential energy map. Through differential operations, Sobel convolution and phase modulation, the local details and features of the image can be effectively enhanced, especially when dealing with complex or small defects, the detection accuracy and robustness can be significantly improved.

[0084] S603: Input the significant feature map into the random forest module and output the defect category.

[0085] S604: Input the significant feature map to the defect location module and output the defect location.

[0086] Specifically, this process inputs the real-time image of the power device into the model, and the feature extraction module first extracts the significant feature map. Then, the significant feature map is input into the random forest module for defect category identification, and then the YOLO module is used to accurately locate the defect position. Random forest provides stable classification capabilities, and YOLO can efficiently locate the defect position, thereby efficiently and accurately detecting and locating the defects of power devices.

[0087] In a possible implementation manner, after S6, the method further includes: Retrain the decoupling detection model at preset intervals.

[0088] It should be noted that those skilled in the art can set the preset duration according to actual needs, and the present invention is not limited thereto.

[0089] In actual application, the entire defect detection process starts with obtaining an image set with defect labels, and then expanding the image set by combining the material properties of power devices and geometric modeling, which improves the diversity and generalization of training data. Next, a decoupled detection model is constructed and trained through a dual-modal positioning loss function to improve the accuracy of defect category recognition and position positioning. Finally, real-time images are collected and input into the trained model to achieve automatic detection and positioning of real-time image defects. Through precise model training and data expansion, the accuracy and robustness of detection are improved, which can effectively solve the problem of missed detection and improve the production quality and efficiency of power devices.

[0090] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0091] In an embodiment of the present invention, by combining the material properties of power devices with geometric modeling to expand the image set, the problem that the existing training set is difficult to cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data, but also improves the generalization ability of the model, reducing the probability of missed detection in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect locations, improving the accuracy and efficiency of detection. By combining the dual-modal positioning loss function, the accuracy of defect positioning is optimized, and the performance of power device defect detection is further improved, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detection in the production process, and improving production quality and efficiency.

[0092] Reference Manual Attached Figure 3 , showing a schematic structural diagram of a power device defect detection system provided by the present invention.

[0093] The present invention further provides a power device defect detection system 20, which is applied to the above-mentioned power device defect detection method, comprising: Processor 201.

[0094] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , the power device defect detection method of the method embodiment is implemented.

[0095] The power device defect detection system 20 provided by the present invention can execute the above-mentioned power device defect detection method and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0096] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: In an embodiment of the present invention, by combining the material properties of power devices with geometric modeling to expand the image set, the problem that the existing training set is difficult to cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data, but also improves the generalization ability of the model, reducing the probability of missed detection in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect locations, improving the accuracy and efficiency of detection. By combining the dual-modal positioning loss function, the accuracy of defect positioning is optimized, and the performance of power device defect detection is further improved, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detection in the production process, and improving production quality and efficiency.

[0097] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0098] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0099] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0100] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0101] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0102] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0103] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0105] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0108] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0109] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a power device defect detection method as described in the method embodiment is implemented.

[0110] A computer-readable storage medium provided by the present invention can implement the steps and effects of the power device defect detection method of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0111] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: In an embodiment of the present invention, by combining the material properties of power devices with geometric modeling to expand the image set, the problem that the existing training set is difficult to cover all defect types is effectively solved. The expanded image set not only enhances the diversity of training data, but also improves the generalization ability of the model, reducing the probability of missed detection in practical applications. The decoupled detection model can simultaneously identify defect categories and accurately locate defect locations, improving the accuracy and efficiency of detection. By combining the dual-modal positioning loss function, the accuracy of defect positioning is optimized, and the performance of power device defect detection is further improved, ensuring high-quality and high-efficiency detection capabilities in practical applications, significantly reducing the problem of missed defect detection in the production process, and improving production quality and efficiency.

[0112] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0113] There are a few points to note:

[0114] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0115] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0116] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0117] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A power device defect detection method, characterized in that: include: S1: Obtain a set of power device defect images with defect labels; S2: expanding the power device defect image set by geometric modeling in combination with the material properties of the power device; S3: Build a decoupled detection model for identifying defect categories and locating defect locations; S4: combining the dual-modal positioning loss function and using the expanded power device defect image set to train the decoupling detection model; S5: Collect real-time images of power devices; S6: Input the real-time image of the power device into the trained decoupling detection model, and output the defect category and defect position of the real-time image of the power device.

2. The power device defect detection method according to claim 1, characterized in that: The defect labels include crack labels, bubble labels, cold soldering labels and pin deformation labels.

3. The power device defect detection method according to claim 1, characterized in that: The power device defect image set includes a plurality of power device defect images; The S2 specifically includes: S201: extracting grayscale images of power device defect images corresponding to different defect labels in the power device defect image set; S202: constructing a differential geometric manifold representation of defect features having curvature and Gaussian geometric curvature tensor based on the grayscale image to capture line defect primitives and point defect primitives of the power device; S203: Determine a defect extension direction angle of the power device defect image; S204: Based on the defect extension direction angle, determine a random transformation field describing the defect extension law in combination with the Navier-Cauchy equation; S205: In combination with the differential geometry manifold representation, under the constraints of the projection consistency constraints on the random transformation field, a power device defect image is generated to complete the expansion of the power device defect image set.

4. The power device defect detection method according to claim 1, characterized in that: The decoupling detection model includes an input layer, a feature extraction module, a defect detection module and an output layer connected in sequence, wherein the decoupling detection model also includes a defect location module, which is respectively connected to the feature extraction module and the output layer, and the feature extraction module has a CBAM significant feature extraction unit based on a dynamic differential feature generation function and a phase modulation function.

5. The power device defect detection method according to claim 1, characterized in that: The S4 specifically includes: S401: inputting the power device defect images in the expanded power device defect image set into the decoupled detection model in sequence, and obtaining the predicted defect category output by the defect detection module and the predicted defect position output by the defect location module; S402: Calculating the defect category recognition accuracy of the predicted defect category; S403: Calculating a bimodal positioning loss value describing a geometric alignment relationship of the predicted defect position; S404: When the defect category recognition accuracy is less than a preset defect category recognition accuracy, adjust the hyperparameters of the defect detection module and retrain the defect detection module; when the dual-modal positioning loss value is less than a preset dual-modal positioning loss value, adjust the hyperparameters of the defect localization module and retrain the defect localization module.

6. The power device defect detection method according to claim 4, characterized in that: The defect detection module is specifically a random forest module, and the defect location module is specifically a yolo module; S6 specifically includes: S601: receiving the real-time image of the power device through the input layer; S602: extracting a significant feature map of the real-time image of the power device by the feature extraction module; S603: Input the significant feature map into the random forest module, and output the defect category; S604: Input the significant feature map to the defect location module, and output the defect location.

7. The power device defect detection method according to claim 6, characterized in that: The feature extraction module further includes a multi-channel backbone network, wherein the multi-channel backbone network includes ResNeSt, HRNet and MobileNet, and the output of the multi-channel backbone network is the input of the CBAM significant feature extraction unit; S602 specifically includes: S6021: extracting an original feature map of the real-time image of the power device through the multi-channel backbone network, and inputting the original feature map into the CBAM significant feature extraction unit; S6022: Perform differentiation operation and Sobel convolution on the original feature map of each channel to construct differential gradient tensor and differential energy map; S6023: Perform fast Fourier transform on the original feature map of channel c to obtain amplitude spectrum and phase spectrum; S6024: modulating the phase spectrum by using the differential energy map to enhance significant features; S6025: Fusing the amplitude spectrum, the modulated phase spectrum, the original feature map and the differential energy map to obtain the significant feature map.

8. The power device defect detection method according to claim 1, characterized in that: After S6, the method further includes: The decoupling detection model is retrained at preset time intervals.

9. A power device defect detection system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the power device defect detection method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the power device defect detection method according to any one of claims 1 to 8 is implemented.

Images