Defect detection method for high-power LED module packaging and related equipment

Abstract

The invention provides a defect detection method for high-power LED module packaging and related equipment. The method comprises the following steps: acquiring a preprocessed image of a high-power LED module corresponding to a multi-angle shot image; determining a feature vector of the preprocessed image according to a deep convolutional neural network model; generating a fused target feature map through the multi-scale fusion feature vector; performing weighting processing on the target feature map to generate a corresponding weighted feature map; and inputting the weighted feature map into a single-step multi-frame detector model, and carrying out defect detection on high-power LED module packaging. Through the implementation of the scheme of the invention, the pre-processed image shot at multiple angles is acquired, the deep convolutional neural network is utilized to extract features, then multi-scale fusion and weighting processing are carried out, and finally, a single-step multi-frame detector model is combined to carry out defect identification, so that the accuracy and reliability of detection are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to a defect detection method and related equipment for high-power LED module packaging. Background Art

[0002] With the rapid development of LED lighting technology, high-power LED modules have been widely used in industrial lighting, automotive lighting and other fields due to their advantages such as high brightness, low energy consumption and long life. In the production process of high-power LED modules, the packaging quality directly affects its performance and reliability, so the detection of packaging defects is particularly important. At present, traditional LED packaging defect detection mainly relies on manual visual inspection or single-angle machine vision inspection methods. Manual inspection has problems such as low efficiency, easy fatigue, and inconsistent standards; and single-angle machine vision inspection is often difficult to fully capture the characteristic information of packaging defects, especially for high-power LED modules with complex three-dimensional structures, some defects may be difficult to detect at specific angles. Although the existing detection methods also use some image processing and machine learning algorithms, they often use simple feature splicing or average fusion methods when processing multi-angle image information. This method cannot effectively utilize the complementary information under different viewing angles, resulting in low detection accuracy and prone to missed detection and false detection. Summary of the invention

[0003] The present application provides a defect detection method and related equipment for high-power LED module packaging, which are used to solve the problem of low detection accuracy when performing packaging defect detection in related technologies.

[0004] The first aspect of the present application provides a defect detection method for a high-power LED module package, the defect detection method for a high-power LED module package comprising: Obtain pre-processed images corresponding to multi-angle photographed images of high-power LED modules; Determining a feature vector of the preprocessed image according to a deep convolutional neural network model; Generate a fused target feature map by fusing the feature vectors at multiple scales; Performing weighted processing on the target feature map to generate a corresponding weighted feature map; By inputting the weighted feature map into a single-step multi-frame detector model, defect detection is performed on the high-power LED module package.

[0005] Optionally, in a first implementation of the first aspect of the present application, the step of determining the feature vector of the preprocessed image according to the deep convolutional neural network model includes: Constructing a deep convolutional neural network model according to the image type of the multi-angle photographed images; wherein the deep convolutional neural network model includes a convolutional layer, a pooling layer and a fully connected layer; Inputting the preprocessed image into a deep convolutional neural network model; Extracting a convolution feature map of the preprocessed image through a convolution operation of the convolution layer; Extracting a pooled feature map of the convolutional feature map through a pooling operation of the pooling layer; The pooled feature map is converted into a one-dimensional vector for output through the integration processing of the fully connected layer; The one-dimensional vector is converted into a corresponding feature vector through normalization processing.

[0006] Optionally, in a second implementation of the first aspect of the present application, the step of generating a fused target feature map by multi-scale fusing the feature vectors includes: Decomposing the feature vector into sub-feature vectors at multiple scales; Construct a feature pyramid based on multi-scale feature representation; Inputting the sub-feature vectors of different scales into different levels of the feature pyramid respectively to generate a multi-level feature map; Sampling and splicing the multi-level feature maps to generate a fused feature map; A target feature map containing defect feature information is determined by performing feature selection on the fused feature map.

[0007] Optionally, in a third implementation of the first aspect of the present application, the step of performing weighted processing on the target feature map to generate a corresponding weighted feature map includes: Determine a pooling feature vector by globally averaging the target feature map; Inputting the pooled feature vector into the first fully connected layer for linear transformation to generate an intermediate feature vector; Performing a nonlinear transformation on the intermediate feature vector by using an activation function to generate an activated feature vector; Inputting the activated feature vector into the second fully connected layer for linear transformation to generate an output feature vector; Generate a weight for each channel according to the output feature vector; wherein the channel is a feature in a feature map; The weighted feature map is determined by multiplying the weights by the fused feature map channel by channel.

[0008] Optionally, in a fourth implementation of the first aspect of the present application, the step of performing defect detection on the high-power LED module package by inputting the weighted feature map into a single-step multi-frame detector model includes: A first default box set of the weighted feature map according to preset parameters of the single-step multi-box detector model; wherein the default box is a multi-scale, multi-aspect ratio candidate bounding box on the weighted feature map; Performing classification prediction on the feature vectors corresponding to each default box in the first default box set, and determining the classification prediction result; Inputting the feature vectors corresponding to the respective default boxes into a regression prediction model to generate a second set of default boxes after regression prediction adjustment; Determine a defect candidate frame set having defects in the second default frame set according to the classification prediction result; The defect detection result of the high-power LED module package is determined by performing non-maximum suppression processing on the defect candidate frame set.

[0009] Optionally, in a fifth implementation of the first aspect of the present application, the defect detection result includes defect location information, and after the step of inputting the weighted feature map into a single-step multi-frame detector model to perform defect detection on the high-power LED module package, the step further includes: Performing semantic segmentation on the preprocessed image according to the defect position information to generate a defect area image; Extracting defect parameter information from the defect area image; Fusing the internal defect information of the packaging material obtained by ultrasonic testing with the defect parameter information to determine fused defect information; The defect type of the high-power LED module package is determined by analyzing the fused defect information.

[0010] Optionally, in a sixth implementation of the first aspect of the present application, the method further includes: Obtaining a spatial distribution diagram of the dielectric constant of the internal material corresponding to the high-power LED module by scanning with a microwave near-field microscopic probe; Determine a local mutation region of dielectric constant according to the dielectric constant spatial distribution map; Constructing a three-dimensional electromagnetic field distortion model according to the local mutation region; Determining a phase offset of microwave reflection according to the three-dimensional electromagnetic field distortion model; When the phase offset is greater than a preset tolerance threshold, marking the topological structure of the deep hidden defect; The topological structure and the multi-angle photographed image are spatially mapped to correct the pseudo-defect features existing in the defect parameter information.

[0011] A second aspect of the present application provides a defect detection device for a high-power LED module package, the defect detection device for a high-power LED module package comprising: An acquisition module is used to acquire pre-processed images corresponding to multi-angle photographed images of the high-power LED module; A determination module, used to determine the feature vector of the preprocessed image according to a deep convolutional neural network model; A first generating module, used for generating a fused target feature map by fusing the feature vectors at multiple scales; A second generating module, used for performing weighted processing on the target feature map to generate a corresponding weighted feature map; The detection module is used to perform defect detection on the high-power LED module package by inputting the weighted feature map into a single-step multi-frame detector model.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the processor is used to execute a computer program stored in the memory, and when the processor executes the computer program, it implements each step of the defect detection method for high-power LED module packaging provided in the first aspect of the embodiment of the present application.

[0013] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each step of the defect detection method for high-power LED module packaging provided in the first aspect of the embodiments of the present application is implemented.

[0014] In summary, according to the defect detection method and related equipment for high-power LED module packaging provided by the present application, a pre-processed image corresponding to the multi-angle shot image of the high-power LED module is obtained; the feature vector of the pre-processed image is determined according to the deep convolutional neural network model; the fused target feature map is generated by multi-scale fusion of the feature vector; the target feature map is weighted to generate a corresponding weighted feature map; and the high-power LED module package is defect-detected by inputting the weighted feature map into a single-step multi-frame detector model. Through the implementation of the present application, pre-processed images shot at multiple angles are obtained, and features are extracted using a deep convolutional neural network, and then multi-scale fusion and weighted processing are performed, and finally, the single-step multi-frame detector model is combined to perform defect identification, which significantly improves the accuracy and reliability of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a process flow of a defect detection method for a high-power LED module package provided in an embodiment of the present application; Figure 2 A schematic diagram of a program module of a defect detection device for high-power LED module packaging provided in an embodiment of the present application; Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0017] In order to solve the problem of low detection accuracy in packaging defect detection in the related art, the present application embodiment provides a defect detection method for high-power LED module packaging, such as Figure 1 The flowchart of the defect detection method of the high-power LED module package provided in this embodiment is as follows. The defect detection method of the high-power LED module package includes the following steps: Step 110: Obtain pre-processed images corresponding to multi-angle photographed images of the high-power LED module.

[0018] Specifically, in this embodiment, in the high-power LED module package defect detection, image acquisition is the basis of the entire detection process. Through multi-angle shooting, all parts of the LED module can be fully captured, including solder joints, chip surfaces and packaging materials. The purpose of multi-angle shooting is to ensure the integrity and diversity of image data, thereby improving the accuracy of defect detection. Image preprocessing is to process the collected original image to make it more suitable for subsequent feature extraction and analysis. The preprocessing steps usually include operations such as grayscale, noise removal and contrast enhancement, which can improve the quality of the image, reduce interference information, and highlight defect features.

[0019] Step 120: Determine a feature vector of the preprocessed image according to the deep convolutional neural network model.

[0020] Specifically, in this embodiment, the deep convolutional neural network (DCNN) is an important tool in the field of modern image processing and computer vision. Through DCNN, high-level features can be extracted from the preprocessed image, and these features can effectively represent the defect information in the image. DCNN is usually composed of convolution layers, pooling layers, and fully connected layers, and each layer is responsible for different feature extraction tasks. The convolution layer extracts local features through convolution kernels, the pooling layer reduces the amount of data through downsampling, and the fully connected layer synthesizes and abstracts the features. By training DCNN, the feature representation of different defects can be learned, thereby providing a basis for subsequent defect detection.

[0021] In an optional implementation of the present embodiment, the step of determining the feature vector of the preprocessed image according to the deep convolutional neural network model includes: constructing a deep convolutional neural network model according to the image type of the multi-angle shot image; wherein the deep convolutional neural network model includes a convolutional layer, a pooling layer and a fully connected layer; inputting the preprocessed image into the deep convolutional neural network model; extracting the convolution feature map of the preprocessed image through the convolution operation of the convolutional layer; extracting the pooled feature map of the convolution feature map through the pooling operation of the pooling layer; converting the pooled feature map into a one-dimensional vector for output through the integration processing of the fully connected layer; and converting the one-dimensional vector into the corresponding feature vector through normalization processing.

[0022] Specifically, in the present embodiment, in the high-power LED module package defect detection, building a deep convolutional neural network model (DCNN) is a crucial step. First, a DCNN model is constructed according to the image type of the image taken from multiple angles. For example, if the image is a color image, the number of channels of the input layer should be set to 3, corresponding to the three RGB channels; if it is a grayscale image, the number of channels is 1. Next, the structure of the DCNN is constructed, which usually includes multiple convolutional layers, pooling layers, and fully connected layers. The parameters of these layers (such as convolution kernel size, number, stride, etc.) need to be adjusted according to the resolution of the image and the complexity of the defect features. For example, for high-resolution images, more convolutional layers and larger convolution kernels may be required to extract richer features. The convolution layer extracts local features of the image, such as edges, textures, etc., through convolution operations. For example, if the input image size is 224x224x3, the convolution kernel size of the first convolution layer is 3x3, the stride is 1, and the padding is 1, then the size of the feature map after convolution is still 224x224, but the number of channels will increase according to the number of convolution kernels. For example, if 64 convolution kernels are used, the number of channels of the output feature map is 64. The pooling layer reduces the spatial size of the feature map through pooling operations, reduces the computational complexity, and retains the main features. For example, in a 2x2 maximum pooling operation, the feature map is divided into 2x2 regions, and the maximum value of each region is taken as the output, so that the height and width of the feature map are halved. If the feature map size output by the convolution layer is 224x224x64, after 2x2 maximum pooling, the feature map size becomes 112x112x64. The fully connected layer is responsible for converting the pooled feature map into a one-dimensional vector for subsequent classification or regression tasks. This process usually involves flattening the feature map and calculating the weights of the fully connected layer. For example, a 112x112x64 feature map is flattened into a one-dimensional vector of length 11211264, and then processed through a fully connected layer to output a vector of fixed length, such as 4096 dimensions. This vector contains high-level feature representations of the image and can be used for defect classification and location. Normalization is the process of converting a one-dimensional vector into a corresponding feature vector so that it has a fixed length and range for subsequent processing and analysis. For example, a 4096-dimensional vector is normalized into a probability distribution through a softmax function, and the value of each dimension represents the probability that the feature belongs to a certain category. In this way, the feature vector not only contains the feature information of the image, but also has the attributes of classification, which provides strong support for defect detection. Through the above steps, the deep convolutional neural network model can effectively extract defect features from images taken from multiple angles and provide accurate feature vectors for subsequent defect detection.

[0023] Step 130: Generate a fused target feature map by fusing the multi-scale feature vectors.

[0024] Specifically, in this embodiment, multi-scale feature fusion is to fuse feature maps of different levels to retain feature information of different scales. High-level feature maps have rich semantic information but low resolution; low-level feature maps have high resolution but less semantic information. Through multi-scale feature fusion, features of different levels can be combined to generate a more comprehensive target feature map. The fusion method adopts element-by-element addition, splicing and other methods. In addition, the introduction of the attention mechanism can further highlight important feature information and suppress unimportant features, thereby improving the accuracy of defect detection.

[0025] In an optional implementation of the present embodiment, the step of generating a fused target feature map by fusing feature vectors at multiple scales includes: decomposing feature vectors into sub-feature vectors at multiple scales; constructing a feature pyramid based on multi-scale feature representation; inputting sub-feature vectors at different scales into different levels of the feature pyramid to generate a multi-level feature map; sampling and splicing the multi-level feature maps to generate a fused feature map; and determining a target feature map containing defect feature information by performing feature selection on the fused feature map.

[0026] Specifically, in this embodiment, feature extraction first starts with a deep convolutional neural network (CNN), in which different convolutional layers extract features with different semantic granularity. The shallow network focuses on high-frequency details and captures features with higher spatial resolution such as solder joint edges and small particle contamination, while the deep network captures low-frequency features and provides global semantic information, such as cracks in packaging materials or solder joint areas. In order to combine these features of different scales, dilated convolution (DConv) is introduced. This convolution increases the expansion rate of the receptive field, thereby retaining a larger range of contextual information when extracting local features. Mathematically, the dilated convolution operation can be expressed as: , in, is the original feature map, is the dilation rate of the dilated convolution, is the kth subscale feature map. Dilation rate The value range is {6, 12, 18}. These three different expansion rates help the model capture contextual information of different scales from multi-scale feature maps, such as the virtual solder joints and the bubble morphology of the packaging material. The dilated convolution operation enables the model to utilize information from a large range of contexts when detecting small defects, thereby improving the detection accuracy.

[0027] In the process of extracting multi-scale features, a feature pyramid structure is used to fuse feature maps from different scales. There are significant differences in the resolution and semantic information of these feature maps. Low-level feature maps usually have higher spatial resolution and can accurately locate the boundaries of defects, while high-level feature maps contain more semantic information and help distinguish different defect types. In order to effectively fuse these features, bilinear interpolation is used for upsampling, and low-level features are combined with high-level features by element-by-element addition. The generation of multi-level feature maps can be expressed by the following formula: , in, is the original feature map of layer l, is a 1×1 convolution operation to match the number of channels, It is an upsampling operation that performs bilinear interpolation on high-level feature maps. is the fused l-th layer pyramid feature map. In this way, the high-resolution feature map of the lower layer can be combined with the semantic features of the higher layer to provide richer information, helping the model to accurately locate and identify various defects. In order to further enhance the expressive power of the feature map, deformable convolution is used for adaptive spatial sampling. Deformable convolution learns the position offset of the convolution kernel so that the convolution operation is no longer limited to a regular grid, so that defect areas with irregular shapes or deviations from conventional forms can be captured more accurately. For example, for irregularly shaped defects such as cracks, the key areas in the feature map may not be completely aligned with the regular grid. The deformable convolution can dynamically adjust the sampling position of the convolution kernel so that the model can cover the crack area more accurately. The operation of the deformable convolution can be expressed by the following formula: , in, is the value of position p in the output feature map, is the weight of the convolution kernel, is the offset of the standard convolution kernel, is the learned deformable offset, is the position in the input feature map Through this dynamic sampling method, the feature map can adapt to different defect forms more flexibly.

[0028] In the fused feature map, the feature selection is further optimized by introducing the gated attention mechanism. The gated attention mechanism is divided into channel attention and spatial attention. The former weights the feature contributions of different channels through global average pooling (GAP), while the latter weights the important areas in the feature map through a combination of maximum pooling and average pooling. Specifically, the channel attention weights and spatial attention weights Generated by the following formula: , , in, is the weight matrix of the first fully connected layer, is the weight matrix of the second fully connected layer, is the ReLU activation function, is the Sigmoi activation function, To fuse feature maps, GAP is a global average pooling operation. and They are average pooling and maximum pooling operations respectively. After the channel and spatial attention weights are multiplied element by element with the fusion feature map, the target feature map is finally obtained. : , Among them, the symbol It means element-by-element multiplication, that is, the values ​​corresponding to each position in the feature map are multiplied one by one. Through such element-by-element multiplication, the model can strengthen the area related to the target position in the feature map, while suppressing the irrelevant area. The target feature map highlights the feature area related to the defect and suppresses the irrelevant area, thereby improving the accuracy of detection.

[0029] Step 140: Perform weighted processing on the target feature map to generate a corresponding weighted feature map.

[0030] Specifically, in this embodiment, weighted processing is to weight the fused feature map through the attention mechanism to highlight the area related to the defect. The attention mechanism performs weighted processing on the feature map by calculating the weight of each channel, thereby enhancing important feature information and suppressing unimportant feature information. Weighted processing can further improve the expressiveness of the feature map and make defect detection more accurate.

[0031] In an optional implementation of the present embodiment, the step of performing weighted processing on the target feature map to generate a corresponding weighted feature map includes: determining a pooled feature vector by globally averaging the pooled target feature map; inputting the pooled feature vector into the first fully connected layer for linear transformation to generate an intermediate feature vector; performing nonlinear transformation on the intermediate feature vector through an activation function to generate an activated feature vector; inputting the activated feature vector into the second fully connected layer for linear transformation to generate an output feature vector; generating weights for each channel according to the output feature vector; wherein the channel is a feature in the feature map; and determining a weighted feature map by multiplying the weights by the fused feature map channel by channel.

[0032] Specifically, in this embodiment, by extracting features from the target feature map through global average pooling, global information can be effectively obtained, and a basis for subsequent feature weighting and optimization is provided. The core idea of ​​the global average pooling operation is to perform average pooling on the feature map in the spatial dimension, so that the output of each feature map channel represents the average response of the channel in the entire spatial range. Assume that the input target feature map The size of is H×W×C, where H and W are the height and width of the feature map, respectively, and C is the number of channels. When performing global average pooling, the average operation is performed on the H×W position of each channel, which can be expressed as: , in, is the pooled eigenvalue of channel c, is the value of the cth channel at position (i, j) in the target feature map, and H and W are the height and width of the feature map, respectively. Through global average pooling, the spatial information of the feature map is compressed into a pooling vector representing the global state, and the shape is transformed from H×W×C to 1×1×C, in preparation for the subsequent fully connected layer operation. Next, the pooled feature vector is input into the first fully connected layer for linear transformation. The function of the fully connected layer is to perform matrix multiplication of the input feature vector with the weight matrix, and add the bias term to generate a new feature vector. This process can be expressed as follows: , in, is the output intermediate feature vector of the first fully connected layer, is the bias term. The weight matrix The dimension is , where C is the dimension of the input feature vector (i.e. the number of channels), is the dimension of the output feature of the first fully connected layer. Afterwards, the ReLU activation function is used to activate the intermediate feature vector Perform nonlinear transformation to generate an activated feature vector. Input the activated feature vector to the second fully connected layer for another linear transformation. Similar to the first fully connected layer, the function of the second fully connected layer is to perform matrix multiplication of the input activated feature vector with the new weight matrix to generate an output feature vector. After obtaining the output feature vector of the second fully connected layer, the weights of each channel can be generated by processing the output vector. These weights are used for subsequent weighted operations to help the model highlight the channel features related to defects. The weight generation process can be done through the output vector of the fully connected layer Combined with an activation function (such as Sigmoid) to achieve: , in, is the weight of the cth channel, generated The value of is between [0,1], indicating the importance of the channel. Using the generated weight Fusion feature map Perform a weighted operation. The weighted operation multiplies the features of each channel by the corresponding weight to obtain a weighted feature map. The formula is expressed as: , in, is the weighted feature map, is the feature of the cth channel in the fusion feature map. The feature of each channel The corresponding weight The weighting is performed to highlight the important features related to the defects and suppress the irrelevant features. This operation can improve the accuracy and robustness of the model in detecting high-power LED module packaging defects.

[0033] Step 150 : Perform defect detection on the high-power LED module package by inputting the weighted feature map into a single-step multi-frame detector model.

[0034] Specifically, in this embodiment, the Single Shot MultiBox Detector (SSD) is an efficient target detection model that can simultaneously detect multiple targets of different sizes and shapes. By inputting the weighted feature map into the SSD model, default boxes of different positions and scales can be generated, and each default box is classified and regressed to obtain the category and location information of the defect. The SSD model generates anchor boxes of different sizes through multi-scale feature blocks to adapt to defects of different sizes and shapes. By improving the SSD model, more default box sizes and aspect ratios can be added to further improve the accuracy of defect detection.

[0035] In an optional implementation of the present embodiment, the step of performing defect detection on a high-power LED module package by inputting a weighted feature map into a single-step multi-frame detector model includes: weighting a first default frame set of the feature map according to preset parameters of the single-step multi-frame detector model; performing classification prediction on feature vectors corresponding to each default frame in the first default frame set, and determining a classification prediction result; inputting the feature vector corresponding to each default frame into a regression prediction model to generate a second default frame set after regression prediction adjustment; determining a defect candidate frame set containing defects in the second default frame set according to the classification prediction result; and determining a defect detection result of the high-power LED module package by performing non-maximum suppression processing on the defect candidate frame set.

[0036] Specifically, in the present embodiment, the first set of default boxes of the feature map is weighted according to the preset parameters of the single-step multi-box detector model, and the preset parameters include but are not limited to the scale, aspect ratio and anchor position of the box. The default box is a candidate bounding box based on multiple scales and different aspect ratios of the weighted feature map, which can cover defect areas of different sizes and shapes to ensure the detection of all potential defects. By generating these candidate boxes at each position of the weighted feature map, the model can effectively detect at different spatial scales and different ratios. Next, the model will perform classification prediction on each default box to determine whether the box contains a defect or whether it is a background area. The classification prediction is based on the feature vectors corresponding to the default box. These feature vectors are extracted from the weighted feature map through a convolutional neural network, and each feature vector represents the defect information that the default box may contain. The classification prediction is performed through a fully connected layer, and a probability value is output to indicate whether the box contains a defect. Assume that the output of the classification prediction is ,in is the defect probability of the kth default box, if Greater than the classification threshold , then the box is considered to contain a defect, otherwise, it is a background area. After completing the classification prediction, the model will input the feature vectors of these default boxes into the regression prediction model, and adjust the position and size of each box through regression prediction. The purpose of regression prediction is to correct the position and size of each default box to a more accurate defect location and boundary. The regression model usually consists of multiple fully connected layers, and the input feature vector Features from the same default box used in classification prediction are processed through a regression network to generate offsets in position and size. These offsets describe the positional difference between the default box and the true defect area. It can be expressed as: , in, is the adjusted second default box set, are the center coordinates of the original default box, is the default box width and height, is the offset generated by regression prediction. With the default box set adjusted by regression prediction, the model then selects those boxes containing defects based on the classification prediction results. Only those boxes predicted as defects in the classification will be retained, and these boxes constitute the defect candidate box set. This process screens the boxes predicted by regression prediction to determine all candidate areas that may contain defects. In order to avoid repeated detection of the same defect area, the defect candidate box set needs to be processed by non-maximum suppression. Non-maximum suppression determines which boxes are redundant by calculating the overlap between candidate boxes. After non-maximum suppression processing, the remaining boxes constitute the final defect detection results. These boxes represent all possible defect locations in the weighted feature map, which effectively improves the accuracy of high-power LED module package defect detection.

[0037] In an optional implementation of the present embodiment, after the step of performing defect detection on the high-power LED module package by inputting the weighted feature map into the single-step multi-frame detector model, the method further includes: performing semantic segmentation on the preprocessed image according to the defect position information to generate a defect area image; extracting defect parameter information from the defect area image; fusing the internal defect information of the packaging material obtained by ultrasonic detection with the defect parameter information to determine the fused defect information; and determining the defect type of the high-power LED module package by analyzing the fused defect information.

[0038] Specifically, in this embodiment, by mapping these defect location information to the original preprocessed image, the model will determine whether each area belongs to the defect area based on these location information. Through semantic segmentation, the model can not only identify the location of the defect in the image, but also clarify the boundary of the area and distinguish the difference between the defect and the normal area. Semantic segmentation technology usually uses deep convolutional neural networks (such as U-Net or DeepLab) to perform pixel-level classification to divide the image into different areas. At each pixel of the image, the network will determine whether the pixel belongs to the defect area, thereby generating an independent defect area image for each defect. After obtaining the defect area image, it is necessary to extract defect parameter information, including but not limited to the area, perimeter, shape, boundary irregularity, etc. of the defect, and these parameters can further analyze the characteristics of the defect. For example, for solder joint defects, the model may calculate the changes in its area and shape to judge the severity of the defect; for crack defects, perimeter and edge irregularity may be important analysis indicators. Through these defect parameters, the model can understand the nature of the defect in more detail and provide important information for subsequent judgment and repair decisions. Parameter extraction usually uses image processing techniques, such as edge detection, contour extraction and other methods. These methods can extract the geometric features of defects and effectively distinguish different types of defects.

[0039] In addition to image information, ultrasonic testing technology provides additional information about internal defects in packaging materials. Ultrasonic testing can effectively identify potential problems such as bubbles and cracks inside packaging materials, which cannot be accurately detected by optical imaging technology. Ultrasonic sensors can identify defects inside materials by measuring the propagation time and reflection intensity of sound waves. Therefore, during the detection process, the internal defect information of packaging materials obtained by ultrasonic testing can be combined with the surface defect information obtained by image processing, thereby providing a more comprehensive perspective for overall defect assessment. Data fusion technology is used to combine these two different sources of information. The fusion of defect information can be carried out in a variety of ways, such as feature-level fusion and decision-level fusion. Specifically, the defect information obtained by ultrasonic testing is first matched with the defect parameter information extracted from the image to establish a comprehensive defect information model. When performing data fusion, the two types of data must first be preprocessed to ensure that they can be effectively combined. For example, normalization can be used to make data from different sources have the same scale, or interpolation methods can be used to map ultrasonic data to the spatial coordinates of the image. After these processes, the ultrasonic detection information and the defect parameters in the image can be combined to form a fused defect information map. This fusion information map can better reflect the defect areas in the high-power LED module package, including not only surface defects but also defects inside the material, thus providing a more comprehensive defect analysis. Finally, by analyzing the fused defect information, the model can determine the type of defects in the high-power LED module package. Since different types of defects have different characteristics and manifestations, the model can identify multiple defect types such as solder joints, cracks, bubbles, and contamination by comprehensively considering the shape, size, location, internal structure and other information of the defects. When judging the defect type, classification models are usually used. After training, these models can predict which category the defect belongs to based on the characteristics of the fused defect information. For example, for an irregularly shaped defect, combined with the internal information of ultrasonic testing, the model may determine that this is a crack; while for a defect with a regular shape and clear edges, the model may determine that it is a solder joint.

[0040] In an optional implementation of the present embodiment, a dynamic correction matrix is ​​constructed based on the correlation mapping relationship between defect types and packaging process parameters; the fused defect information is input into a multi-physical field coupling simulation model for simulation verification, and the simulation verification results are obtained; based on the dynamic correction matrix and the simulation verification results, the packaging equipment control parameters are iteratively adjusted through a gradient descent algorithm to generate an optimized packaging parameter set.

[0041] Specifically, in this embodiment, in the defect correction process of high-power LED module packaging, it is first necessary to construct a dynamic correction matrix based on the correlation mapping relationship between the defect type and the packaging process parameters. The construction of this matrix is ​​based on historical data and engineering experience. By modeling the relationship between different defect types (such as cold solder joints, cracks, contamination, etc.) and key parameters in the packaging process (such as welding temperature, pressure, time, etc.), a mathematical model that reflects the defect type and the corresponding process adjustment in real time can be formed. Specifically, each element of this matrix represents the degree of influence between a certain defect type and a certain process parameter, which is usually obtained by regression analysis, statistical analysis, etc. For example, in the case of solder joint defects, it may be found that when the welding temperature exceeds a certain threshold, the probability of cold solder joints will increase, and vice versa; and for crack defects, it may be found that the temperature change rate and cooling rate of the material have a significant effect on the formation of cracks. Secondly, the simulation model used for simulation verification combines the coupling effects of multiple physical fields such as thermal, mechanical, and electrical fields, and can simulate the generation mechanism of various defects in the packaging process and their impact on the performance of the final product. In the simulation process, the fused defect information will be used as input, and physical simulation will be used to calculate how the changes in packaging process parameters affect the performance of the product under different defect conditions. Assuming that in the simulation model, the packaging process parameters p=[p1,p2,...,pn] are the parameter set to be optimized, and the defect information d=[d1,d2,...,dm] is the defect type and related characteristics obtained through detection, the simulation model can be expressed as a multi-physical field coupling system. By solving these equations, the change results of product performance y=f(p,d)) can be obtained, where y is the performance indicator of the product (such as reliability, life, quality, etc.), and f(p,d) is the performance response calculated based on the defect information and packaging process parameters. Through the simulation model, the impact of different defects on the product during the packaging process can be accurately predicted, and the performance under various process parameter conditions can be obtained. Based on the simulation verification results and the dynamic correction matrix, the packaging equipment control parameters can be iteratively adjusted through the gradient descent algorithm to further optimize the packaging process. The update formula of gradient descent can be written as: , in, is the set of process parameters in the kth iteration, is the learning rate, is the gradient of the loss function with respect to the process parameters, indicating the direction in which the loss function changes the fastest. Through multiple iterations, the process parameters will gradually adjust to a value that minimizes the loss function, thereby optimizing the packaging process. In practical applications, the gradient descent algorithm will iteratively adjust each process parameter based on the relationship between the defect type and process parameters provided by the dynamic correction matrix and the prediction results of the simulation model, and finally generate an optimized set of packaging parameters.

[0042] In an optional implementation of the present embodiment, a spatial distribution map of the dielectric constant of the internal material corresponding to the high-power LED module is obtained by scanning with a microwave near-field microprobe; the local mutation area of ​​the dielectric constant is determined according to the spatial distribution map of the dielectric constant; a three-dimensional electromagnetic field distortion model is constructed according to the local mutation area; the phase offset of the microwave reflection is determined according to the three-dimensional electromagnetic field distortion model; when the phase offset is greater than a preset tolerance threshold, the topological structure of the deep hidden defect is marked; the topological structure is spatially mapped with the multi-angle shot image, and the pseudo-defect features existing in the defect parameter information are corrected.

[0043] Specifically, in this embodiment, when implementing defect detection based on scanning the packaged LED module based on a microwave near-field microprobe and combining the dielectric constant spatial distribution map, it is first necessary to obtain the dielectric constant spatial distribution map of the packaged internal material through microwave technology. The microwave near-field microprobe can perform high-resolution scanning on the sample and obtain the spatial distribution information of the dielectric constant inside the material. The dielectric constant is an important electromagnetic property of the material, which is closely related to the electric field response, polarization characteristics and propagation speed of the electromagnetic wave of the material. Through scanning, the dielectric constant distribution of the packaging material at different positions can be obtained. After obtaining the spatial distribution map of the dielectric constant, a three-dimensional electromagnetic field distortion model is constructed based on the spatial distribution data of the dielectric constant, with particular attention paid to the local mutation area of ​​the dielectric constant caused by defects such as bubbles or cracks. The change in the dielectric constant will cause the reflection, refraction and propagation characteristics of the electromagnetic wave to change, thereby causing electromagnetic field distortion. For the defect area caused by bubbles or cracks, this distortion is manifested as a shift in the microwave reflection phase. Therefore, the potential defect area can be identified and located by calculating the microwave reflection phase shift caused by the defect inside the material. Considering the relationship between the phase shift of microwave reflection and the change of dielectric constant of the material, the reflection phase shift It can be calculated by the following formula: , in, and represent the impedance outside and inside the defect area respectively, and represent the dielectric constants outside and inside the defect region, respectively, is the phase shift, which reflects the phase difference of electromagnetic waves caused by the change of dielectric constant. It should be noted that the mutation caused by bubbles or cracks will significantly affect the microwave reflection phase, and then affect the distribution of the entire electromagnetic field. Therefore, the calculated phase shift can be used to determine whether there is a defect. Specifically, when the phase shift exceeds the preset tolerance, it can be considered that there is a deep hidden defect in the area.

[0044] After identifying the potential deep hidden defects, the specific form of the defects should be further located by marking their topological structures. The topological structure describes the shape and position of the defects in three-dimensional space and is usually represented by geometric modeling methods. For example, inside the LED module, if bubbles or cracks exist in the packaging material, they usually present a certain regular or irregular shape. The dielectric constant mutation area obtained by scanning can be represented by a three-dimensional geometric model to mark the morphological characteristics of the defects. Combining optical images for defect location and correction, especially when performing surface crack detection, often faces the problem of false alarms. Defect detection of optical images taken from multiple angles often produces pseudo defects due to material inhomogeneity or other factors. In order to eliminate these pseudo defects, it is necessary to spatially map the deep defect topological structure obtained by microwave near-field microprobe detection with the optical image. Through this mapping, the area marked as cracks in the optical image can be compared with the actual deep defect area, thereby eliminating the internal material inhomogeneity. Considering the different spatial resolutions of optical images and microwave detection, geometric alignment is required, and algorithms such as iterative closest point (ICP) are usually used to optimize the alignment process. The ICP algorithm achieves precise alignment of image data by minimizing the Euclidean distance error between the surface crack detection feature point set and the deep defect feature point set, ensuring the spatial consistency of optical images and microwave data, thereby reducing false positives and negatives. The combination of phase offset calculation based on dielectric constant mutation, topological structure marking and spatial mapping of multi-angle optical images not only makes the identification of deep defects more accurate, but also effectively improves the accuracy of surface crack detection.

[0045] According to a defect detection method for high-power LED module packaging provided by the present application, a pre-processed image corresponding to the multi-angle shot image of the high-power LED module is obtained; the feature vector of the pre-processed image is determined according to the deep convolutional neural network model; the fused target feature map is generated by multi-scale fusion feature vector; the target feature map is weighted to generate the corresponding weighted feature map; and the weighted feature map is input into the single-step multi-frame detector model to perform defect detection on the high-power LED module packaging. Through the implementation of the present application, the pre-processed image shot at multiple angles is obtained, and the features are extracted using the deep convolutional neural network, and then through multi-scale fusion and weighted processing, finally combined with the single-step multi-frame detector model for defect recognition, which significantly improves the accuracy and reliability of detection.

[0046] Figure 2 A defect detection device for a high-power LED module package provided in an embodiment of the present application can be used to implement the defect detection method for a high-power LED module package in the above-mentioned embodiment. Figure 2 As shown, the defect detection device for high-power LED module packaging mainly includes: An acquisition module 10 is used to acquire pre-processed images corresponding to multi-angle photographed images of the high-power LED module; A determination module 20, configured to determine a feature vector of a preprocessed image according to a deep convolutional neural network model; A first generating module 30, configured to generate a fused target feature map by using a multi-scale fused feature vector; A second generating module 40 is used to perform weighted processing on the target feature map to generate a corresponding weighted feature map; The detection module 50 is used to perform defect detection on the high-power LED module package by inputting the weighted feature map into the single-step multi-frame detector model.

[0047] In an optional implementation of the present embodiment, the determination module is specifically used to: construct a deep convolutional neural network model according to the image type of images taken from multiple angles; wherein the deep convolutional neural network model includes a convolutional layer, a pooling layer and a fully connected layer; input the preprocessed image into the deep convolutional neural network model; extract the convolution feature map of the preprocessed image through the convolution operation of the convolution layer; extract the pooled feature map of the convolution feature map through the pooling operation of the pooling layer; convert the pooled feature map into a one-dimensional vector for output through the integration processing of the fully connected layer; and convert the one-dimensional vector into the corresponding feature vector through normalization processing.

[0048] In an optional implementation of the present embodiment, the first generation module is specifically used to: decompose the feature vector into multi-scale sub-feature vectors; construct a feature pyramid based on the multi-scale feature representation; input the sub-feature vectors of different scales into different levels of the feature pyramid to generate a multi-level feature map; sample and splice the multi-level feature map to generate a fused feature map; and determine the target feature map containing defect feature information by performing feature selection on the fused feature map.

[0049] In an optional implementation of the present embodiment, the second generation module is specifically used to: determine the pooling feature vector by globally averaging the pooling target feature map; input the pooling feature vector to the first fully connected layer for linear transformation to generate an intermediate feature vector; perform nonlinear transformation on the intermediate feature vector through an activation function to generate an activated feature vector; input the activated feature vector to the second fully connected layer for linear transformation to generate an output feature vector; generate weights for each channel according to the output feature vector; wherein the channel is a feature in the feature map; and determine a weighted feature map by multiplying the weights by the fused feature map channel by channel.

[0050] In an optional implementation of the present embodiment, the detection module is specifically used to: weight a first default box set of the feature map according to preset parameters of the single-step multi-box detector model; wherein the default box is a multi-scale, multi-aspect ratio candidate boundary box on the weighted feature map; classify and predict the feature vectors corresponding to each default box in the first default box set, and determine the classification prediction result; input the feature vector corresponding to each default box into the regression prediction model to generate a second default box set after regression prediction adjustment; determine a defect candidate box set with defects in the second default box set according to the classification prediction result; and determine the defect detection result of the high-power LED module package by performing non-maximum suppression processing on the defect candidate box set.

[0051] In an optional implementation of the present embodiment, the detection module is also used to: perform semantic segmentation on the preprocessed image according to the defect location information to generate a defect area image; extract defect parameter information from the defect area image; fuse the internal defect information of the packaging material obtained by ultrasonic detection with the defect parameter information to determine the fused defect information; and determine the defect type of the high-power LED module package by analyzing the fused defect information.

[0052] In an optional implementation of the present embodiment, the defect detection device further includes: a correction module. The correction module is used to: obtain a spatial distribution map of the dielectric constant of the internal material corresponding to the high-power LED module by scanning with a microwave near-field microprobe; determine the local mutation area of ​​the dielectric constant according to the spatial distribution map of the dielectric constant; construct a three-dimensional electromagnetic field distortion model according to the local mutation area; determine the phase offset of the microwave reflection according to the three-dimensional electromagnetic field distortion model; when the phase offset is greater than a preset tolerance threshold, mark the topological structure of the deep hidden defect; perform spatial mapping between the topological structure and the multi-angle shot image, and correct the pseudo-defect features existing in the defect parameter information.

[0053] According to a defect detection device for high-power LED module packaging provided by the present application, a pre-processed image corresponding to the multi-angle shot image of the high-power LED module is obtained; the feature vector of the pre-processed image is determined according to the deep convolutional neural network model; the fused target feature map is generated by multi-scale fusion feature vectors; the target feature map is weighted to generate the corresponding weighted feature map; and the weighted feature map is input into the single-step multi-frame detector model to perform defect detection on the high-power LED module packaging. Through the implementation of the present application, pre-processed images shot at multiple angles are obtained, and features are extracted using a deep convolutional neural network, and then multi-scale fusion and weighted processing are performed, and finally, the single-step multi-frame detector model is combined to perform defect identification, which significantly improves the accuracy and reliability of detection.

[0054] According to the application plan provided Figure 3An electronic device provided in an embodiment of the present application. The electronic device can be used to implement the defect detection method of the high-power LED module package in the above-mentioned embodiment, mainly comprising: The memory 301, the processor 302 and the computer program 303 stored in the memory 301 and executable on the processor 302 are connected by communication. When the processor 302 executes the computer program 303, the defect detection method for the high-power LED module package in the above-mentioned embodiment is implemented. The number of processors can be one or more.

[0055] The memory 301 may be a high-speed random access memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as a disk memory. The memory 301 is used to store executable program codes, and the processor 302 is coupled to the memory 301 .

[0056] Furthermore, the present application also provides a computer-readable storage medium, which may be provided in the electronic device in the above embodiments. Figure 3 Memory in the illustrated embodiment.

[0057] The computer readable storage medium stores a computer program, and when the program is executed by the processor, the defect detection method for high-power LED module packaging in the above-mentioned embodiment is implemented. Furthermore, the computer storable medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, etc., which can store program codes.

[0058] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0059] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0060] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A defect detection method for high-power LED module packaging, characterized in that: include: Obtain pre-processed images corresponding to multi-angle photographed images of high-power LED modules; Determining a feature vector of the preprocessed image according to a deep convolutional neural network model; Generate a fused target feature map by fusing the feature vectors at multiple scales; Performing weighted processing on the target feature map to generate a corresponding weighted feature map; By inputting the weighted feature map into a single-step multi-frame detector model, defect detection is performed on the high-power LED module package.

2. The defect detection method for high-power LED module packaging according to claim 1, characterized in that: The step of determining the feature vector of the preprocessed image according to the deep convolutional neural network model comprises: Constructing a deep convolutional neural network model according to the image type of the multi-angle photographed images; wherein the deep convolutional neural network model includes a convolutional layer, a pooling layer and a fully connected layer; Inputting the preprocessed image into a deep convolutional neural network model; Extracting a convolution feature map of the preprocessed image through a convolution operation of the convolution layer; Extracting a pooled feature map of the convolutional feature map through a pooling operation of the pooling layer; The pooled feature map is converted into a one-dimensional vector for output through the integration processing of the fully connected layer; The one-dimensional vector is converted into a corresponding feature vector through normalization processing.

3. The defect detection method for high-power LED module package according to claim 1, characterized in that: The step of generating a fused target feature map by fusing the feature vectors at multiple scales comprises: Decomposing the feature vector into sub-feature vectors at multiple scales; Construct a feature pyramid based on multi-scale feature representation; Inputting the sub-feature vectors of different scales into different levels of the feature pyramid respectively to generate a multi-level feature map; Sampling and splicing the multi-level feature maps to generate a fused feature map; A target feature map containing defect feature information is determined by performing feature selection on the fused feature map.

4. The defect detection method for high-power LED module package according to claim 3, characterized in that: The step of performing weighted processing on the target feature map to generate a corresponding weighted feature map comprises: Determine a pooling feature vector by globally averaging the target feature map; Inputting the pooled feature vector into the first fully connected layer for linear transformation to generate an intermediate feature vector; Performing a nonlinear transformation on the intermediate feature vector by using an activation function to generate an activated feature vector; Inputting the activated feature vector into the second fully connected layer for linear transformation to generate an output feature vector; Generate a weight for each channel according to the output feature vector; wherein the channel is a feature in a feature map; The weighted feature map is determined by multiplying the weights by the fused feature map channel by channel.

5. The defect detection method for high-power LED module package according to claim 1, characterized in that: The step of performing defect detection on the high-power LED module package by inputting the weighted feature map into a single-step multi-frame detector model comprises: A first default box set of the weighted feature map according to preset parameters of the single-step multi-box detector model; wherein the default box is a multi-scale, multi-aspect ratio candidate bounding box on the weighted feature map; Performing classification prediction on the feature vectors corresponding to each default box in the first default box set, and determining the classification prediction result; Inputting the feature vectors corresponding to the respective default boxes into a regression prediction model to generate a second set of default boxes after regression prediction adjustment; Determine a defect candidate frame set having defects in the second default frame set according to the classification prediction result; The defect detection result of the high-power LED module package is determined by performing non-maximum suppression processing on the defect candidate frame set.

6. The defect detection method for high-power LED module package according to claim 5, characterized in that: The defect detection result includes defect location information. After the step of performing defect detection on the high-power LED module package by inputting the weighted feature map into a single-step multi-frame detector model, the method further includes: Performing semantic segmentation on the preprocessed image according to the defect position information to generate a defect area image; Extracting defect parameter information from the defect area image; Fusing the internal defect information of the packaging material obtained by ultrasonic testing with the defect parameter information to determine fused defect information; The defect type of the high-power LED module package is determined by analyzing the fused defect information.

7. The defect detection method for high-power LED module package according to claim 6, characterized in that: The method further comprises: Obtaining a spatial distribution diagram of the dielectric constant of the internal material corresponding to the high-power LED module by scanning with a microwave near-field microscopic probe; Determine a local mutation region of dielectric constant according to the dielectric constant spatial distribution map; Constructing a three-dimensional electromagnetic field distortion model according to the local mutation region; Determining a phase offset of microwave reflection according to the three-dimensional electromagnetic field distortion model; When the phase offset is greater than a preset tolerance threshold, marking the topological structure of the deep hidden defect; The topological structure and the multi-angle photographed image are spatially mapped to correct the pseudo-defect features existing in the defect parameter information.

8. A defect detection device for high-power LED module packaging, characterized in that: The defect detection device for high-power LED module packaging comprises: An acquisition module is used to acquire pre-processed images corresponding to multi-angle photographed images of the high-power LED module; A determination module, used to determine the feature vector of the preprocessed image according to a deep convolutional neural network model; A first generating module, used for generating a fused target feature map by fusing the feature vectors at multiple scales; A second generating module, used for performing weighted processing on the target feature map to generate a corresponding weighted feature map; The detection module is used to perform defect detection on the high-power LED module package by inputting the weighted feature map into a single-step multi-frame detector model.

9. An electronic device, characterized in that: The device comprises a memory and a processor, wherein: The processor is used to execute the computer program stored in the memory; When the processor executes the computer program, the steps in the defect detection method for high-power LED module packaging described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the defect detection method for high-power LED module packaging described in any one of claims 1 to 7 are implemented.

Images