Chip pin coplanarity detection method and system based on multi-focus image fusion

Abstract

The invention relates to a chip pin coplanarity detection method and system based on multi-focus image fusion, and belongs to the technical field of chip detection.The chip pin coplanarity detection method comprises the steps that a data set is constructed according to side face images of chip pins, the images in the data set are preprocessed, a training set is constructed, and cut image blocks are obtained; and then, constructing a network model based on multi-focus image fusion according to the cut image blocks, setting training parameters, taking a training set as the input of the multi-focus image fusion network model, training the multi-focus image fusion network model, and finally, carrying out multi-focus chip image fusion. And carrying out chip pin and main body segmentation on the fused image, calculating a quantized value of pin coplanarity, and carrying out comparative judgment on the quantized value and a threshold value. According to the invention, the feature representation capability of the network is enhanced, and the cross attention feature fusion module based on semantic prior segmentation reduces error features and strengthens related features, so that feature fusion is more sufficient.

Description

Technical Field

[0001] The present invention relates to the field of chip detection technology, and in particular to a chip pin coplanarity detection method and system based on multi-focus image fusion. Background Art

[0002] At present, chip pin forming mainly relies on two methods: equipment forming and manual forming. During the forming process, due to the rebound phenomenon of the pin material, the pin may not be coplanar or not attached to the bottom. This micro-rebound phenomenon cannot be quantified and interpreted visually. In addition, if the coplanarity does not meet the requirements, it will directly lead to unqualified chip products and affect the operation process of the next link. The coplanarity of chip pins is one of the key indicators to measure chip quality. In the field of electronic assembly technology, the coplanarity of chip pins directly affects the installation quality of chips and the stability and reliability of subsequent electronic equipment. The current mainstream chip pin coplanarity detection method mainly relies on manual visual inspection or single focal length image acquisition. Manual visual inspection is a traditional chip pin coplanarity detection method that relies on the operator's visual observation and experience judgment. In this method, the inspector usually uses a magnifying glass or microscope to observe the chip pins to evaluate whether their coplanarity meets the specified standards. The operator will judge whether there is a height difference based on the visual alignment of the pins to determine whether the chip meets the quality requirements. Ordinary algorithms for detecting chip pin coplanarity usually rely on image processing technology and three-dimensional reconstruction methods. These methods extract height information by analyzing the image of the chip pins, and then calculate the coplanarity. For example, a single-image integrated circuit pin coplanarity detection method first establishes a relationship model between the camera, light source and the surface to be tested, and then determines the relevant parameters of the material of the IC pin to be tested through experiments. Next, based on the established relationship model between light intensity and image grayscale, the height information is solved, and finally the surface three-dimensional morphology of the IC pin and its solder joint is restored based on the height information. The experimental results show that the height measurement error of this method is less than ±0.08mm, and the relative error is -2.6%, which verifies the effectiveness of the method. In addition, a study has proposed a chip pin coplanarity visual inspection system based on high-precision contour line laser. The system uses digital signal processing technology, a blue laser optical system and a DSP chip to process and analyze the collected digital signals, and realizes high-precision detection of small electronic components such as SOP and QFP chip pins. With the development and application of deep learning, many chip coplanarity detections have also begun to use deep learning methods for detection. For example, YOLOv8 combines spatial attention mechanism to detect tiny defects in chip pins. By constructing a chip surface defect dataset, including both real and synthetic sets, this method achieves significant improvements in detecting tiny ink marks and crack defects.

[0003] Single-focus image detection has significant limitations when dealing with chip pin coplanarity analysis. Since this method can only acquire images on a single focal plane, it cannot provide clear image information for pins of different heights at the same time. This results in that when detecting pins with different heights or thicknesses, some pins may not be accurately evaluated due to defocus, thus affecting the measurement accuracy of coplanarity. In addition, single-focus image detection is highly sensitive to environmental changes and chip placement, which further limits its application in automated and high-throughput detection systems. Therefore, this detection method may not meet the detection requirements for high precision and high efficiency when facing complex and changing industrial application scenarios. Summary of the invention

[0004] The present invention overcomes the shortcomings of the prior art and provides a chip pin coplanarity detection method and system based on multi-focus image fusion.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] The first aspect of the present invention provides a chip pin coplanarity detection method based on multi-focus image fusion, which is characterized by specifically comprising:

[0007] Acquire an incoming signal of a chip, and collect a side image of a chip pin according to the incoming signal of the chip;

[0008] Constructing a data set according to the chip pin side image and preprocessing the images in the data set to construct a training set and obtain cropped image blocks;

[0009] Constructing a network model based on multi-focus image fusion according to the cropped image blocks, setting training parameters and using the training set as input of the multi-focus image fusion network model to train the multi-focus image fusion network model;

[0010] Multi-focus chip image fusion is performed, and the chip pins and body are segmented on the fused image. The quantitative value of the pin coplanarity is calculated and compared with the threshold for judgment.

[0011] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, a data set is constructed according to the chip pin side image and the images in the data set are preprocessed to construct a training set and obtain the cropped image blocks, specifically:

[0012] Adjust the imaging device to maximize the depth of field, and then set the long focus point and the short focus point respectively, so that the camera automatically switches between the two focus points and collects images to obtain the side image of the chip pins;

[0013] Constructing a data set according to the side image of the chip pin, and preprocessing each image data in the data set to obtain a preprocessed data set, and using the preprocessed data set as a training set;

[0014] 80% of the image samples in the training set are used for training, and the remaining 20% ​​are used for testing. The corresponding image data of the images used for training are cropped into a number of non-overlapping image blocks of predetermined pixels.

[0015] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, a network model based on multi-focus image fusion is constructed according to the cropped image blocks, and training parameters are set and the training set is used as the input of the multi-focus image fusion network model to train the multi-focus image fusion network model, specifically:

[0016] Constructing a network model based on multi-focus image fusion, and inputting the cropped image block into a dense convolution layer and a transformation layer of the network model based on multi-focus image fusion;

[0017] The inputs of dense convolutional layers and transformation layers are input into the enhancement module through weighted concatenation, and the low-frequency and high-frequency features are enhanced through the spatial attention mechanism of the enhancement module.

[0018] The enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network respectively to obtain enhanced high-frequency features and enhanced low-frequency features, and the enhanced high-frequency features and enhanced low-frequency features are input into the cross-attention module based on DeepLabv3 prior;

[0019] By calculating the loss function and optimizing the weight parameters, when training the network model based on multi-focus image fusion, the error between the predicted quality score and the target quality score output by the network model based on multi-focus image fusion is continuously reduced, and the weight parameters of the network model based on multi-focus image fusion are continuously optimized.

[0020] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, the enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network to obtain enhanced high-frequency features and enhanced low-frequency features, specifically including:

[0021] The enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network respectively to obtain enhanced high-frequency features and enhanced low-frequency features, where the following relationship is satisfied:

[0022] F EH =E(F H +F L )×F H , F EL =E(F H+F L )×F L ,

[0023] where F EH represents the enhanced high-frequency features, F EL represents the enhanced low-frequency feature, F H represents the high-frequency features before enhancement, F L represents the low-frequency features before enhancement, and E represents the spatial attention enhancement operation.

[0024] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, the enhanced high-frequency features and enhanced low-frequency features are input into the cross-attention module based on DeepLabv3 prior, specifically:

[0025] The enhanced high-frequency features and enhanced low-frequency features of the input are fused and input into the cross-attention module based on DeepLabv3 prior, where the attention calculation formula satisfies the following relationship:

[0026]

[0027] Where Q, K, and V represent query, key, and value, respectively, which are calculated by multiplying the input by the trainable matrix. k is the dimension of K, B is the relatively learnable positional encoding, and softmax is the normalization function;

[0028] The attention function is performed h times and the result is connected to the multi-head self-attention, where h is set to 4. Next, the feature tokens generated by the multi-head attention layer are refined by two multi-layer perceptron layers with GELU activation layers;

[0029] Given two local window features X1 and X2 from different domains, the weight matrix W shared across different windows is Q , W K , W V , project the weight matrix W Q , W K , W V into Q, K, and V, fully integrating the complementary information of different focal lengths, and mapping the fused depth features back to the image space.

[0030] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, the loss function is calculated, the weight parameters are optimized, and when the network model based on multi-focus image fusion is trained, the error between the predicted quality score and the target quality score output by the network model based on multi-focus image fusion is continuously reduced, and the weight parameters of the network model based on multi-focus image fusion are continuously optimized. Specifically, Apoca:

[0031] When training the network model based on multi-focus image fusion, the error between the predicted quality score and the target quality score output by the network model based on multi-focus image fusion is continuously reduced, and the weight parameters of the network model based on multi-focus image fusion are continuously optimized. The weights of the model are optimized through the ReLU activation function and the optimization algorithm that uses Adam to automatically update the learning rate.

[0032] Among them, the calculation formula of the loss function is:

[0033] L=γL aux +μL texture ,

[0034] Among them, L aux Indicates auxiliary strength loss, L texture Represents the gradient loss; γ and μ are weight factors used to control the contribution of each term; the gradient loss is designed to suppress the fusion result to keep important details from the input image, and its formula is as follows:

[0035]

[0036] where H and W represent the height and width of the source image, respectively; ‖·‖ represents the I1 norm, max(·) refers to the element-by-element maximum selection, and I f is the fusion result image, I1 and I2 are the source images in two focusing states respectively;

[0037] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, it also includes:

[0038] The auxiliary intensity loss is used to guide the model to capture intensity information, where the following relationship is satisfied:

[0039]

[0040] L aux represents the auxiliary intensity loss, H and W represent the height and width of the source image, I1 and I2 are the source images in two focusing states, and ‖·‖ represents the I1 norm.

[0041] Furthermore, in the chip pin coplanarity detection method based on multi-focus image fusion, multi-focus chip image fusion is performed, and the chip pin and the main body are segmented on the fused image, the quantitative value of the pin coplanarity is calculated, and compared with the threshold value for judgment, specifically:

[0042] Use the trained model to fuse the side views of the chip pins at different focal lengths, segment the chip pins from the background and the subject using the yolov8 segmentation algorithm, and then calculate the quantitative value of the coplanarity using the chip pin coplanarity algorithm, and use the threshold to determine whether it is within the qualified range;

[0043] The fitting straight line method is used to obtain the quantitative value of coplanarity. First, the upper rightmost point of each pin is used as the reference point, and the reference point is fitted into a reference line. The distance X from each reference point to the reference line is calculated. i , filter out the maximum value X max With the minimum value X min , where the quantitative value of the pin coplanarity satisfies the following relationship:

[0044] L=X max -X min ,

[0045] Where L is the quantitative value of the pin coplanarity, X max is the maximum distance from each reference point to the reference line, x min The minimum distance from each reference point to the reference line.

[0046] A second aspect of the present invention provides a chip pin coplanarity detection system based on multi-focus image fusion, the system comprising a memory and a processor, the memory comprising a chip pin coplanarity detection method program based on multi-focus image fusion, and when the chip pin coplanarity detection based on multi-focus image fusion is executed by the processor, any step of the chip pin coplanarity detection method based on multi-focus image fusion is implemented.

[0047] The third aspect of the present invention provides a computer-readable storage medium, including a chip pin coplanarity detection method program based on multi-focus image fusion. When the chip pin coplanarity detection based on multi-focus image fusion is executed by a processor, any step of the chip pin coplanarity detection method based on multi-focus image fusion is implemented.

[0048] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0049] The chip pin coplanarity detection method and system based on multi-focus image fusion provided by the present invention firstly uses a feature extraction module based on a dense convolution layer and a transformation layer in parallel, which not only enhances the feature representation capability of the network, but also makes the increase in memory complexity negligible; then, the cross-attention feature fusion module based on semantic prior segmentation respectively reduces erroneous features and strengthens related features, thereby making feature fusion more sufficient. Finally, the information extraction algorithm provided by the present invention can detect the coplanarity quantization values ​​of various types of chips, and the research results can provide a theoretical reference for the visual chip coplanarity detection system to improve positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, drawings of other embodiments can be obtained based on these drawings without paying creative work.

[0051] Figure 1 It is a flow chart of a chip pin coplanarity detection method and system based on multi-focus image fusion provided by an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of an imaging system provided by an embodiment of the present invention.

[0053] Among them, Figure 2 middle:

[0054] 1. Optical camera, 2. Lens, 3. 45 degree coaxial, 4. Light source, 5. Beam splitter, 6. Increasing lens, 7. IC chip, 8. Detection platform.

[0055] Figure 3 Schematic diagram of the network structure of the multi-focus image fusion model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0058] Example 1

[0059] like Figure 1 As shown, the first aspect of the present invention provides a chip pin coplanarity detection method based on multi-focus image fusion, which is characterized by specifically comprising:

[0060] S102: Obtaining an incoming signal of a chip, and collecting a side image of a chip pin according to the incoming signal of the chip;

[0061] S104: constructing a data set according to the chip pin side image and preprocessing the images in the data set to construct a training set and obtain cropped image blocks;

[0062] S106: constructing a network model based on multi-focus image fusion according to the cropped image blocks, setting training parameters and using the training set as input of the multi-focus image fusion network model to train the multi-focus image fusion network model;

[0063] S108: Perform multi-focus chip image fusion, segment the chip pins and the main body of the fused image, calculate the quantitative value of the pin coplanarity, and compare it with the threshold for judgment.

[0064] It should be noted that the pre-processing process includes denoising, filtering, etc.

[0065] The step S104 specifically includes:

[0066] like Figure 2 As shown, a 45° coaxial light source imaging method is used to obtain a side view of the chip pin as a data set, and the data set is preprocessed; wherein the training data set includes 2 groups, the imaging device is adjusted to maximize the depth of field, and then the focal length is adjusted to make the thickest chip and the thinnest chip respectively in the best focus position, and the images collected twice are used as the multi-focus input of the data set;

[0067] For the images in the above dataset, they are randomly divided into two parts, where 80% of the image samples are used for training and the remaining 20% ​​are used for testing. This ensures that the model can be effectively verified on both training and test data;

[0068] For the images used for training, the images are cropped into multiple non-overlapping 64×64 pixel blocks. This helps the network learn more detailed features and improves the accuracy of image fusion.

[0069] Among them, in step S106, specifically including: Figure 3 As shown in the figure, the cropped image blocks are input into the Enhance Dense Block (EDB) in the multi-focus image fusion model. Each EDB has a dense convolutional neural network (DC), a transform layer (TR), and an enhancement module (EM). In order to prevent the extraction of high-frequency features and low-frequency features from interfering with each other, the DC layer and the TR layer are connected in parallel. The inputs of the DC layer and the TR layer are input into the EM layer by weighted splicing. The low-frequency and high-frequency features are enhanced respectively by the spatial attention mechanism of the EM layer. After four EDB layers, a 48×16-dimensional feature map is obtained.

[0070] The enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network respectively, and the calculation is shown as follows:

[0071] F EH =E(F H +F L )×F H ,

[0072] F EL =E(F H +F L )×F L ,

[0073] Among them, F EH represents the enhanced high-frequency features, F EL represents the enhanced low-frequency feature, F H represents the high-frequency features before enhancement, F L Represents the low-frequency features before enhancement. E represents the spatial attention enhancement operation.

[0074] The enhanced high-frequency features F of the fusion input EH and strengthen the low frequency characteristics F EL , and input it into the Cross Attention module (CA) based on DeepLabv3 prior. The attention calculation formula is as follows:

[0075]

[0076] Where Q, K, and V represent query, key, and value, respectively, and are calculated by multiplying the input by the trainable matrix. k is the dimension of K, B is the relatively learnable positional encoding, and softmax is the normalization function.

[0077] The attention function is performed h times and the result is connected to the multi-head self-attention, where h is set to 4. Next, the feature tokens generated by the Multi Head Attention (MHA) layer are refined by two Multilayer Perceptron (MLP) layers with GELU activation layers. The normalization layer (LN) is performed after MHA and before MLP. The complete process can be expressed as:

[0078]

[0079] Given two local window features X1 and X2 from different domains, the weight matrix W shared across different windows is Q , W K , WV They are projected into Q, K, and V in the above way. The whole process of cross-attention based fusion is formalized as:

[0080]

[0081] For Q1 from focal length 1, it merges cross-domain information by performing weighted attention on K2 and V2 from focal length 2, while preserving the information in focal length 1 through residual connections, and vice versa.

[0082] After fully fusing the complementary information of different focal lengths, a CNN-based image reconstruction unit is designed to map the fused depth features back to the image space. Deployment of CNN-based image reconstruction unit H CR

[0083] To reduce the number of channels and generate a fused image I f , which is expressed as:

[0084] I f =H CR (F D ),

[0085] Calculate the loss function and optimize the weight parameters. When training the network model, continuously reduce the error between the predicted quality score and the target quality score output by the network model, and continuously optimize the weight parameters of the network model. The activation function uses the ReLU activation function, and the Adam (Adaptive moment estimation) optimization algorithm automatically updates the learning rate. The batch-size is set to 64, the learning rate is fixed to 0.001, and the loss function calculation formula is shown in the formula:

[0086] L=γL aux +μL texture ,

[0087] Among them, L aux Indicates auxiliary strength loss, L texture Denotes the gradient loss. γ and μ are weight factors used to control the contribution of each term. The gradient loss is designed to suppress the fusion result to keep important details from the input image, and its formula is as follows:

[0088]

[0089] Where H and W represent the height and width of the source image, respectively. ‖·‖ represents the l1 norm, max(·) refers to the element-wise maximum value selection, and I f is the fusion result image, I1 and I2 are the source images in two focusing states respectively.

[0090] For auxiliary intensity loss, an excellent image fusion algorithm expects to generate a fused image with appropriate intensity based on the global apparent intensity information of the source images. To this end, the following auxiliary intensity loss is designed to guide our fusion model to capture appropriate intensity information:

[0091]

[0092] Among them, in step S108,

[0093]

[0094] Step 3: Use the trained model to fuse the side views of the chip pins at different focal lengths, segment the chip pins from the background and the subject using the yolov8 segmentation algorithm, and then calculate the quantitative value of the coplanarity using the designed chip pin coplanarity algorithm. Use the threshold to determine whether it is within the qualified range. Use the fitted straight line method to obtain the quantitative value of the coplanarity. First, use the upper rightmost point of each pin as the reference point, fit the reference point into a reference line, and calculate the distance X from each reference point to the reference line. i , filter out the maximum value X max With the minimum value X min . Then the coplanarity quantification value L is as follows:

[0095] L=X max -X min ,

[0096] Among them, the threshold is used to determine whether it is within the qualified range. For example, when the calculated coplanarity quantization value L is greater than the preset threshold, it is unqualified / qualified. When the calculated coplanarity quantization value L is not greater than the preset threshold, it is unqualified / qualified. Relevant technical personnel can freely set it according to the actual situation.

[0097] Example 2

[0098] A second aspect of the present invention provides a chip pin coplanarity detection system based on multi-focus image fusion, the system comprising a memory and a processor, the memory comprising a chip pin coplanarity detection method program based on multi-focus image fusion, and when the chip pin coplanarity detection based on multi-focus image fusion is executed by the processor, the technical solution of Example 1 is implemented.

[0099] Example 3

[0100] The third aspect of the present invention provides a computer-readable storage medium, including a chip pin coplanarity detection method program based on multi-focus image fusion. When the chip pin coplanarity detection based on multi-focus image fusion is executed by a processor, the technical solution of Example 1 is implemented.

[0101] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0102] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0104] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and other media that can store program codes.

[0105] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0106] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A chip pin coplanarity detection method based on multi-focus image fusion, characterized in that: Specifically include: Acquire an incoming signal of a chip, and collect a side image of a chip pin according to the incoming signal of the chip; Constructing a data set according to the chip pin side image and preprocessing the images in the data set to construct a training set and obtain cropped image blocks; Constructing a network model based on multi-focus image fusion according to the cropped image blocks, setting training parameters and using the training set as input of the multi-focus image fusion network model to train the multi-focus image fusion network model; Multi-focus chip image fusion is performed, and the chip pins and body are segmented on the fused image. The quantitative value of the pin coplanarity is calculated and compared with the threshold for judgment.

2. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 1 is characterized in that: A data set is constructed based on the chip pin side image and the images in the data set are preprocessed to construct a training set and obtain cropped image blocks, specifically: Adjust the imaging device to maximize the depth of field, and then set the long focus point and the short focus point respectively, so that the camera automatically switches between the two focus points and collects images to obtain the side image of the chip pins; Constructing a data set according to the side image of the chip pin, and preprocessing each image data in the data set to obtain a preprocessed data set, and using the preprocessed data set as a training set; 80% of the image samples in the training set are used for training, and the remaining 20% ​​are used for testing. The corresponding image data of the images used for training are cropped into a number of non-overlapping image blocks of predetermined pixels.

3. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 1, characterized in that: A network model based on multi-focus image fusion is constructed according to the cropped image blocks, and training parameters are set and the training set is used as the input of the multi-focus image fusion network model to train the multi-focus image fusion network model, specifically: Constructing a network model based on multi-focus image fusion, and inputting the cropped image block into a dense convolution layer and a transformation layer of the network model based on multi-focus image fusion; The inputs of dense convolutional layers and transformation layers are input into the enhancement module through weighted concatenation, and the low-frequency and high-frequency features are enhanced through the spatial attention mechanism of the enhancement module. The enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network respectively to obtain enhanced high-frequency features and enhanced low-frequency features, and the enhanced high-frequency features and enhanced low-frequency features are input into the cross-attention module based on DeepLabv3 prior; By calculating the loss function and optimizing the weight parameters, when training the network model based on multi-focus image fusion, the error between the predicted quality score and the target quality score output by the network model based on multi-focus image fusion is continuously reduced, and the weight parameters of the network model based on multi-focus image fusion are continuously optimized.

4. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 3 is characterized in that: The enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network to obtain enhanced high-frequency features and enhanced low-frequency features, including: The enhanced low-frequency and high-frequency feature maps are mixed and input into the fusion network respectively to obtain enhanced high-frequency features and enhanced low-frequency features, where the following relationship is satisfied: F EH =E(F H +F L )×F H ,F EL =E(F H +F L )×F L , Among them, F EH represents the enhanced high-frequency features, F EL represents the enhanced low-frequency feature, F H represents the high-frequency features before enhancement, F L represents the low-frequency features before enhancement, and E represents the spatial attention enhancement operation.

5. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 3 is characterized in that: The enhanced high-frequency features and enhanced low-frequency features are input into the cross-attention module based on DeepLabv3 prior, specifically: The enhanced high-frequency features and enhanced low-frequency features of the input are fused and input into the cross-attention module based on DeepLabv3 prior, where the attention calculation formula satisfies the following relationship: Where Q, K, and V represent query, key, and value, respectively, which are calculated by multiplying the input by the trainable matrix. k is the dimension of K, B is the relatively learnable positional encoding, and softmax is the normalization function; The attention function is performed h times and the result is connected to the multi-head self-attention, where h is set to 4. Next, the feature tokens generated by the multi-head attention layer are refined by two multi-layer perceptron layers with GELU activation layers; Given two local window features X1 and X2 from different domains, the weight matrix W shared across different windows is Q , W K , W V , project the weight matrix W Q , W K , W V into Q, K, and V, fully integrating the complementary information of different focal lengths, and mapping the fused depth features back to the image space.

6. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 3 is characterized in that: Calculate the loss function, optimize the weight parameters, and continuously reduce the error between the predicted quality score and the target quality score output by the network model based on multi-focus image fusion when training the network model based on multi-focus image fusion, and continuously optimize the weight parameters of the network model based on multi-focus image fusion. Specifically, Apoca: When training the network model based on multi-focus image fusion, the error between the predicted quality score and the target quality score output by the network model based on multi-focus image fusion is continuously reduced, and the weight parameters of the network model based on multi-focus image fusion are continuously optimized. The weights of the model are optimized through the ReLU activation function and the optimization algorithm that uses Adam to automatically update the learning rate. Among them, the calculation formula of the loss function is: L=γL aux +μL texture , Among them, L aux Indicates auxiliary strength loss, L texture Represents the gradient loss; γ and μ are weight factors used to control the contribution of each term; the gradient loss is designed to suppress the fusion result to keep important details from the input image, and its formula is as follows: where H and W represent the height and width of the source image, respectively; ‖·‖ represents the I1 norm, max(·) refers to the element-by-element maximum selection, and I f is the fusion result image, I1 and I2 are the source images in two focusing states respectively.

7. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 6, characterized in that: Also includes: The auxiliary intensity loss is used to guide the model to capture intensity information, where the following relationship is satisfied: L aux represents the auxiliary intensity loss, H and W represent the height and width of the source image, I1 and I2 are the source images in two focusing states, and ‖·‖ represents the I1 norm.

8. The chip pin coplanarity detection method based on multi-focus image fusion according to claim 1, characterized in that: Perform multi-focus chip image fusion, segment the chip pins and the main body of the fused image, calculate the quantitative value of the pin coplanarity, and compare it with the threshold for judgment, specifically: Use the trained model to fuse the side views of the chip pins at different focal lengths, segment the chip pins from the background and the subject using the yolov8 segmentation algorithm, and then calculate the quantitative value of the coplanarity using the chip pin coplanarity algorithm, and use the threshold to determine whether it is within the qualified range; The fitting straight line method is used to obtain the quantitative value of coplanarity. First, the upper rightmost point of each pin is used as the reference point, and the reference point is fitted into a reference line. The distance X from each reference point to the reference line is calculated. i , filter out the maximum value X max With the minimum value X min , where the quantitative value of the pin coplanarity satisfies the following relationship: L=X max -X min , Where L is the quantitative value of the pin coplanarity, X max is the maximum distance from each reference point to the reference line, X min The minimum distance from each reference point to the reference line.

9. Chip pin coplanarity detection system based on multi-focus image fusion, characterized in that: The system includes a memory and a processor, wherein the memory includes a chip pin coplanarity detection method program based on multi-focus image fusion. When the chip pin coplanarity detection based on multi-focus image fusion is executed by the processor, the steps of the chip pin coplanarity detection method based on multi-focus image fusion as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: It includes a chip pin coplanarity detection method program based on multi-focus image fusion. When the chip pin coplanarity detection based on multi-focus image fusion is executed by a processor, the steps of the chip pin coplanarity detection method based on multi-focus image fusion as described in any one of claims 1 to 8 are implemented.

Images