Quality detection method, device and equipment for flip chip assembly after FC flip and medium

Abstract

The application provides a quality detection method, device, equipment and medium for FC flip-chip assembly after welding, comprising the following steps: synchronously collecting multi-modal data of the FC flip-chip assembly after welding in real time, generating to-be-fused multi-modal data after pre-processing the multi-modal data; extracting high-precision features from the to-be-fused multi-modal data, normalizing feature vectors of different modes, and splicing to generate to-be-recognized image data according to a dynamic weight distribution rule; inputting the to-be-recognized image data into a pre-constructed defect recognition model based on deep learning to perform defect recognition, and outputting a defect recognition result based on a defect type, a defect position coordinate and a defect severity; inputting the defect recognition result into a constructed welding process parameter optimization strategy network to output a real-time adjustment strategy result, which is used for adaptive adjustment of welding process parameters; and the application effectively improves the production quality and reliability of the chip assembly.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit manufacturing technology, and in particular to a method, apparatus, equipment and medium for quality inspection of FC flip-soldering chip components. Background Technology

[0002] In today's rapidly developing electronic technology landscape, electronic devices are continuously moving towards miniaturization, thinner designs, and higher performance. This trend places extremely stringent demands on integrated circuit packaging technology. Among these advancements, FC (Flip Chip) technology, with its unique advantages, is playing an increasingly important role in the integrated circuit packaging field. FC technology enables high-density interconnection between the chip and the substrate. Compared to traditional wire bonding and other packaging technologies, it significantly reduces the package size of chip components and effectively improves electrical performance, such as reducing signal transmission delay and enhancing signal integrity. This greatly satisfies the urgent need of modern electronic devices for high-performance, miniaturized chip components.

[0003] The flip-soldering process for chips is incredibly complex and sophisticated, involving several delicate and critical steps. During the flip-soldering process between the chip and the substrate, precise alignment is paramount; even the slightest misalignment can severely impact the subsequent soldering quality. The subsequent soldering process is equally crucial, requiring precise control of key parameters such as soldering temperature, pressure, and time. Even slight deviations can easily lead to various defects. Common defect types include cold solder joints (where the solder joint appears connected but lacks proper contact, resulting in unstable electrical connections); short circuits (causing unexpected continuity between different circuits, leading to circuit failure); open circuits (causing circuit interruptions and preventing normal signal transmission); and chip misalignment (causing the chip to deviate from its intended position on the substrate, affecting overall performance). These defects not only significantly reduce the performance and reliability of chip components but can also, in severe cases, cause the entire product to fail completely, resulting in substantial losses and challenges for electronic product manufacturing.

[0004] Currently, the industry mainly uses methods such as visual inspection, X-ray inspection, and ultrasonic inspection for quality inspection of FC flip-soldering chip components. Visual inspection is usually achieved through manual observation or machine vision systems. Manual observation relies on the rich experience of inspectors to judge whether the shape of the solder joints on the chip component surface is regular, whether the color is normal, and whether the chip position is accurate. However, this method is inefficient and easily affected by human factors, making it difficult to guarantee the accuracy and consistency of the inspection. Although machine vision systems can improve inspection efficiency to some extent by analyzing the surface features of the chip component through image recognition technology, they can only detect visible defects on the surface and cannot analyze hidden soldering defects inside the chip (such as cold solder joints, micro-cracks, etc.).

[0005] X-ray inspection technology works by emitting X-rays that penetrate chip components. By utilizing the differences in X-ray absorption by different materials, it creates an image of the internal structure, allowing for the inspection of solder joints within the chip. However, X-ray inspection has insufficient sensitivity for detecting minute defects, such as microcracks and extremely fine solder joints, which can easily lead to missed detections. Furthermore, X-ray inspection equipment is expensive, has high maintenance costs, and requires specific environmental conditions, which limits its widespread application.

[0006] Ultrasonic testing technology utilizes the reflection and refraction properties of ultrasonic waves as they propagate through different media to detect the structural integrity of chips. When ultrasonic waves encounter defects, they generate abnormal reflection signals. By analyzing these signals, the presence, location, and size of defects can be determined. However, ultrasonic testing requires a high level of professional knowledge and operational experience from the testing personnel. Different personnel may obtain different test results for the same object, and the accuracy of the test results is significantly affected by human factors. Furthermore, the testing process is relatively complex and slow, making it difficult to meet the needs of rapid testing on large-scale production lines.

[0007] In summary, existing methods for quality inspection of FC flip-soldering chip components all have their limitations, failing to efficiently, accurately, and comprehensively detect various potential defects. Therefore, developing an innovative quality inspection method for FC flip-soldering chip components that effectively overcomes these problems is of crucial practical significance for improving chip component production quality, reducing production costs, and promoting the high-quality development of the electronics industry. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a method, apparatus, equipment, and medium for quality inspection of FC flip-soldering chip assemblies. Through steps such as multimodal data acquisition, fusion, defect identification, dynamic detection feedback, adaptive parameter adjustment, and multi-scale feature analysis and defect localization, the invention overcomes the shortcomings of existing inspection technologies, improves the production quality and reliability of chip assemblies, enhances inspection accuracy and production efficiency, and reduces production costs.

[0009] In a first aspect, the present invention proposes a quality inspection method for chip assemblies after FC flip-soldering, specifically including the following steps: Step S1: Real-time synchronous acquisition of multimodal data of the chip assembly after FC flip-soldering; preprocessing of the multimodal data to generate multimodal data to be fused; Step S2: Extracting high-precision features from the multimodal data to be fused; normalizing the feature vectors of different modalities; and splicing them according to dynamic weight allocation rules to generate image data to be identified; Step S3: Inputting the image data to be identified into a pre-constructed deep learning-based defect identification model for defect identification; and outputting defect identification results based on defect type, defect location coordinates, and defect severity. Step S4: Input the defect identification results into the constructed welding process parameter optimization strategy network, and output the real-time adjustment strategy results for adaptive adjustment of welding process parameters; The multimodal data includes machine vision image data, infrared thermal imaging data, and ultrasonic scanning image data.

[0010] Furthermore, the step S1 of preprocessing the multimodal data to generate the multimodal data to be fused specifically includes: For the acquired machine vision image data, after Gaussian filtering for noise reduction, the image contrast and clarity are enhanced by an adaptive histogram equalization algorithm. Then, an edge detection algorithm is used to segment the solder joints and chips to generate the first data to be fused. For the collected infrared thermal imaging data, the infrared imaging data is calibrated by a temperature calibration model based on deep learning to generate a second data to be fused. The second data to be fused is then aligned with the first data to be fused in spatial position to realize that the temperature data in the infrared imaging data corresponds to the actual position of the chip component. For ultrasound scan image data, a third set of data to be fused is generated after denoising the ultrasound scan image data through wavelet transform and filtering. Multimodal data to be fused is constructed from the first data to be fused, the second data to be fused, and the third data to be fused.

[0011] Furthermore, step S2 specifically includes: Extract a first feature based on the geometry and grayscale of the solder joints from the first data to be fused; Extract a second feature based on temperature gradient, hotspot location, and temperature change trend from the second data to be fused; Extract a third feature based on the reflection signal parameters from the third set of data to be fused; A multimodal feature vector based on a first feature, a second feature, and a third feature is constructed. After normalizing the multimodal feature vector, a weight adaptive adjustment model is used to dynamically assign corresponding weights to the first feature, the second feature, and the third feature, respectively. The weights and corresponding features are concatenated and fused to generate the image data to be recognized.

[0012] Furthermore, the weight adaptive adjustment model employs a Bayesian neural network and a Monte Carlo Dropout mechanism. It obtains the probability distribution of the feature vector output of each modality through multiple forward propagations, and estimates the uncertainty of each modality based on the variance of this distribution, thereby achieving dynamic weight allocation.

[0013] Furthermore, in step S3, the deep learning-based defect recognition model adopts a multimodal fusion architecture, including a spatial attention mechanism (SAM) module, a channel attention mechanism (CAM) module, and a feature fusion layer. The SAM module generates spatial weights by calculating the spatial correlation of each position in the feature map of the image data to be recognized. The CAM module calculates the statistical information of the feature map in the image data to be recognized and generates channel weights based on the statistical information. The feature fusion layer realizes the feature fusion of the spatial weights and the channel weights through a weighted summation process and outputs the fused feature map.

[0014] Furthermore, in step S4, the welding process parameter optimization strategy network uses the near-end strategy optimization (PPO) framework. The state space of the optimization strategy network is configured as a three-dimensional state space including defect type, defect location coordinates, and defect severity. The action space is configured as a three-dimensional continuous variable action space including welding temperature, pressure, and time. The multi-objective reward function R is set as follows: In the above formula, Acc represents the improvement rate of detection accuracy, Eff represents production efficiency, and Ene represents energy consumption cost. , , This represents the weighting coefficient.

[0015] Furthermore, the deep learning-based defect recognition model also includes convolutional neural networks based on different receptive fields, specifically including: Small-scale convolutional layers with receptive fields of 2×2 to 4×4 employ deformable convolutional kernels, adaptively adjusting the shape and position of the kernels to extract subtle local feature information of chip components and locate the position and shape of minute defects; Large-scale convolutional layers with receptive fields of 8×8 to 10×10 employ a combination of dilated convolution and grouped convolution to identify defects or structural anomalies over a large area based on the overall structural characteristics of the chip assembly.

[0016] In a second aspect, the present invention also provides a quality inspection device for FC flip-soldering chip assemblies to perform the method as described in the first aspect, the quality inspection device comprising: A multimodal data acquisition module is used to synchronously acquire multimodal data of the chip assembly after FC flip-soldering in real time, and generate multimodal data to be fused after preprocessing the multimodal data; a multimodal data fusion module is used to extract high-precision features from the multimodal data to be fused, normalize the feature vectors of different modalities, and then stitch them together according to dynamic weight allocation rules to generate image data to be identified; a defect identification module is used to input the image data to be identified into a pre-constructed deep learning-based defect identification model for defect identification, and output defect identification results based on defect type, defect location coordinates, and defect severity. An adaptive adjustment module is used to input the defect identification results into the constructed welding process parameter optimization strategy network and output real-time adjustment strategy results for adaptive adjustment of welding process parameters. The multimodal data includes machine vision image data, infrared thermal imaging data, and ultrasonic scanning image data.

[0017] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in the first aspect.

[0018] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0019] The beneficial effects of this invention are: This invention reduces the number of model parameters and computational complexity while ensuring multi-task learning effectiveness. It has vital practical significance for improving the production quality of chip components, reducing production costs, and promoting the high-quality development of the electronics industry. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the quality inspection method for FC flip-soldering chip components proposed in this invention. Figure 2 This is a schematic diagram of the quality inspection device framework for the FC flip-soldering chip assembly proposed in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown in the figure, this embodiment proposes a quality inspection method for FC flip-soldering chip components, which specifically includes the following steps: Step S1: Real-time synchronous acquisition of multimodal data of the chip assembly after FC flip soldering, and preprocessing of the multimodal data to generate multimodal data to be fused.

[0023] In this embodiment, the step S1 of preprocessing the multimodal data to generate the multimodal data to be fused specifically includes: For the acquired machine vision image data, after Gaussian filtering for noise reduction, the image contrast and clarity are enhanced by an adaptive histogram equalization algorithm. Then, an edge detection algorithm is used to segment the solder joints and chips to generate the first data to be fused. For the collected infrared thermal imaging data, the infrared imaging data is calibrated by a temperature calibration model based on deep learning to generate a second data to be fused. The second data to be fused is then aligned with the first data to be fused in spatial position to realize that the temperature data in the infrared imaging data corresponds to the actual position of the chip component. For ultrasound scan image data, a third set of data to be fused is generated after denoising the ultrasound scan image data through wavelet transform and filtering. Multimodal data to be fused is constructed from the first data to be fused, the second data to be fused, and the third data to be fused.

[0024] In this embodiment, a Basler acA2040-90um high-resolution industrial camera with a custom-designed 50mm focal length optical lens was used for machine vision image data acquisition. This lens employs advanced optical materials to effectively reduce light scattering and chromatic aberration, improving image clarity. For the FC-soldered chip components, after experimental testing and algorithm optimization, the exposure time was precisely set to 80 microseconds, the aperture to F4.0, and the camera frame rate stabilized at 30 frames per second to obtain clear surface images. A PCI-Express 3.0 x4 interface image acquisition card was used, with a data transfer rate of 32Gbps, stably transmitting uncompressed image data at 1920×1080 resolution, avoiding data loss and delay. The acquired machine vision image data was first processed using a deep learning-based Gaussian filtering algorithm to remove noise. This algorithm adaptively identifies and removes various types of noise in the image while preserving image details. Then, an adaptive histogram equalization algorithm was used to enhance image contrast and clarity, dynamically adjusting the histogram distribution to make image details clearer. Finally, the improved Canny edge detection algorithm is used to segment the solder joints and the chip. This algorithm optimizes the edge detection threshold selection rules and can more accurately obtain key feature information.

[0025] The infrared thermal imaging data acquisition process is as follows: While the chip component is powered on and operating normally, a FLIR A325sc infrared thermal imager is used to measure the temperature field. This imager has a high-precision temperature measurement capability of ±0.5℃ and a resolution of 640×480. The focus is quickly adjusted to 120mm using an autofocus system, and the temperature measurement range is optimized to 30℃ to 120℃ using an intelligent algorithm, with an emissivity calibration of 0.95. 20 frames of infrared thermal images are acquired per second. The acquired data is calibrated for temperature based on high-precision blackbody calibration source data. A registration algorithm based on deep learning feature point matching is used to more accurately align the infrared thermal imaging data with the machine vision image in spatial position, ensuring that the temperature data accurately corresponds to the actual position of the chip component.

[0026] The acquisition process for ultrasonic scanning image data was as follows: An Olympus OmniScan MX2 ultrasonic scanning device was used, paired with an ultrasonic probe with a center frequency of 7.5MHz. Based on the structural characteristics of the chip assembly and the detection requirements, the scanning angle was set from 0° to 360°, and the scanning step size was 0.1mm, performing point-by-point scanning of the chip assembly. The ultrasonic signal transmission power was controlled at 2W, and the receiving gain was adjusted to 40dB to ensure high-quality ultrasonic signal acquisition. After scanning, a joint filtering algorithm based on wavelet transform and deep learning was used to filter the ultrasonic scanning data. This algorithm combines the advantages of wavelet transform multi-resolution analysis and deep learning feature extraction, effectively removing noise and enhancing the signal. Simultaneously, a wavelet transform enhancement algorithm was used to improve signal strength and clarity, highlighting defect features.

[0027] It should be noted that this embodiment does not specifically limit the algorithm used in the processing of the above image data.

[0028] Step S2: Extract high-precision features from the multimodal data to be fused, normalize the feature vectors of different modalities, and then stitch them together according to the dynamic weight allocation rules to generate image data to be recognized.

[0029] In this embodiment, step S2 specifically includes: Extract a first feature based on the geometry and grayscale of the solder joints from the first data to be fused; Extract a second feature based on temperature gradient, hotspot location, and temperature change trend from the second data to be fused; Extract a third feature based on the reflection signal parameters from the third set of data to be fused; A multimodal feature vector based on a first feature, a second feature, and a third feature is constructed. After normalizing the multimodal feature vector, a weight adaptive adjustment model is used to dynamically assign corresponding weights to the first feature, the second feature, and the third feature, respectively. The weights and corresponding features are concatenated and fused to generate the image data to be recognized.

[0030] The adaptive weight adjustment model employs a Bayesian neural network (BNN) with a Monte Carlo Dropout mechanism. It obtains the probability distribution of each modality's feature vector output through multiple forward propagations and estimates the uncertainty of each modality based on the variance of this distribution, thereby dynamically allocating weights. The BNN introduces the Monte Carlo Dropout mechanism, randomly activating some neurons in the network multiple times during the inference phase to generate multiple output predictions. By analyzing the statistical distribution (such as mean and variance) of these prediction results, it quantifies the prediction uncertainty of each modality feature from machine vision image data, infrared thermal imaging data, and ultrasonic scanning image data under the current input conditions, and dynamically adjusts the weight allocation of each modality during the fusion process accordingly. The Bayesian Neural Network (BNN) contains three hidden layers, each with 512 neurons, and a dropout rate of 0.2. The mean and variance of the weight distribution are calculated through variational inference. The weight update frequency is ≥200Hz, and knowledge distillation technology is introduced during model training. A teacher network with >50M parameters guides a student network with <10M parameters, enabling the fusion model to maintain a classification accuracy of over 98.5% while reducing the inference latency to ≤15ms.

[0031] In this embodiment, the SIFT (Scale Invariant Feature Transform) algorithm, combined with a deep learning network, is used to extract the first features of the geometric and grayscale features of the solder joint. The geometric features specifically involve the solder joint area, perimeter, and aspect ratio. The extraction accuracy of the solder joint area is improved to ±0.008 mm², the perimeter to ±0.015 mm, and the aspect ratio to ±0.04. The grayscale features specifically involve the average grayscale value and grayscale gradient. Combined with a deep learning texture analysis model, in this embodiment, the extraction accuracy of the average grayscale value is improved to ±1.5, and the extraction accuracy of the grayscale gradient is improved to ±0.08.

[0032] In the process of infrared thermal imaging data processing, to eliminate temperature measurement deviations caused by factors such as differences in material emissivity, environmental reflection, and non-uniformity of the optical system, this embodiment employs a deep learning-based temperature calibration model to correct the original thermal imaging data. This temperature calibration model uses a reference temperature point collected by a high-precision contact temperature probe (such as a platinum resistance thermometer PT100) under the same operating conditions as a monitoring signal. It combines the material information of the chip components, surface morphology features, and environmental parameters as inputs. A nonlinear mapping relationship is constructed using a convolutional neural network (CNN) or a graph neural network (GNN) to achieve pixel-level precise temperature field calibration, significantly improving the absolute temperature measurement accuracy of the thermal image. In this embodiment, the temperature gradient accuracy reaches ±0.08℃ / mm. The improved centroid algorithm combined with deep learning is used to locate hotspot positions with an accuracy of ±0.4mm. Furthermore, a deep learning-based linear regression algorithm optimizes the temperature change trend analysis, achieving an accuracy of ±0.15℃ / s.

[0033] In the process of ultrasonic scanning image data processing, feature extraction methods based on time-frequency analysis and deep learning are used to perform in-depth processing on the reflected signal parameters. The amplitude accuracy of the reflected signal reaches ±0.04V. The phase difference measurement algorithm combined with deep learning phase feature extraction is used to achieve a phase accuracy of ±4°. The propagation time accuracy reaches ±0.08μs by using a time delay estimation algorithm combined with deep learning time series analysis.

[0034] In this embodiment, the process of splicing and fusing feature vectors from different modalities according to a dynamic weight allocation rule specifically includes: constructing a weight adaptive adjustment model based on Bayesian inference, dynamically adjusting the weights of machine vision, infrared thermal imaging, and ultrasonic scanning features according to the type of chip component, manufacturing process, and feature changes in real-time acquired data. Before splicing, a Z-score normalization algorithm is used, introducing an adaptive threshold adjustment mechanism to dynamically adjust the normalization parameters according to the distribution of feature vectors, making the feature vectors of each modality more representative under a unified scale. During splicing, a dimensional splicing method is adopted to ensure the integrity of the feature vectors. A fusion model is constructed, using a machine learning algorithm combining Support Vector Machine (SVM), Random Forest, and Deep Belief Network to construct a multi-model collaborative fusion model. 15,000 accurately labeled multi-modal fusion feature vectors of normal and defective chip components are used as training samples, covering chip components from different production batches and under different process conditions. Through 800 iterations of training, an adaptive learning rate adjustment strategy is adopted, dynamically adjusting the learning rate according to the model performance during training, so that the model's classification accuracy reaches over 98%.

[0035] Step S3: Input the image data to be identified into a pre-built deep learning-based defect identification model for defect identification, and output the defect identification result based on the defect type, defect location coordinates and defect severity.

[0036] In this embodiment, the deep learning-based defect recognition model in step S3 adopts a multimodal fusion architecture, including a spatial attention mechanism (SAM) module, a channel attention mechanism (CAM) module, and a feature fusion layer. The SAM module generates spatial weights by calculating the spatial correlation of each position in the feature map of the image data to be recognized. The CAM module calculates the statistical information of the feature map in the image data to be recognized and generates channel weights based on the statistical information. The feature fusion layer realizes the feature fusion of the spatial weights and the channel weights through a weighted summation process and outputs the fused feature map.

[0037] In this embodiment, the SAM module generates spatial weights by calculating the spatial correlation of each location in the input feature map. Specifically, it measures the similarity between different spatial locations by performing autocorrelation operations on the feature map, thereby highlighting potential defect areas. The CAM module focuses on evaluating the information importance between channels. It calculates the statistical information (such as mean and variance) of the feature map for each channel and then uses this statistical information to generate channel weights to emphasize channels that contribute more to defect identification. Finally, the feature fusion layer combines the spatial and channel weights generated by the SAM and CAM modules, achieving effective fusion of multi-scale features through a weighted summation process. The fusion weights are automatically learned by the model during training to optimize the final defect identification accuracy. The entire model architecture aims to effectively extract and fuse features from different data sources, improving the accuracy of chip component quality inspection.

[0038] The deep learning-based defect recognition model also includes convolutional neural networks based on different receptive fields, specifically including: Small-scale convolutional layers with receptive fields of 2×2 to 4×4 employ deformable convolutional kernels, which adaptively adjust the shape and position of the kernels to extract subtle local feature information of chip components more accurately, thereby precisely locating the position and shape of minute defects. Large-scale convolutional layers with receptive fields of 8×8 to 10×10 employ a combination of dilated convolution and grouped convolution to identify defects or structural anomalies over a large area based on the overall structural characteristics of the chip assembly.

[0039] In this embodiment, the fully connected layer of the defect type identification branch has 4-5 hidden layers with 256, 128, 64, and 32 neurons respectively, and uses the Softmax activation function; the fully connected layer of the location coordinate prediction branch has 3-4 hidden layers with 128, 64, and 32 neurons respectively, and uses the linear activation function; the fully connected layer of the severity assessment branch has 3-4 hidden layers with 128, 64, and 32 neurons respectively, and uses the sigmoid activation function. This approach improves model efficiency, reduces the number of model parameters, lowers computational complexity, and accelerates model inference speed while maintaining multi-task learning performance.

[0040] It should be noted that after constructing the framework of the deep learning-based defect identification model, the model still needs to be trained. In this embodiment, a dataset containing 8000 training samples and 2000 test samples is established, and classified and managed according to multiple dimensions such as chip component type, manufacturing process, defect severity, and possible causes of defects, to facilitate subsequent model training and evaluation. This embodiment adopts a novel deep learning network architecture that integrates convolutional neural networks (CNN), recurrent neural networks (RNN), and attention mechanisms. Convolutional layers use convolutional kernels of different sizes and strides, such as 3×3 and 5×5 kernels, and combine grouped convolution and dilated convolution to increase the diversity of feature extraction. Pooling layers use adaptive pooling, automatically adjusting the pooling region according to the size and distribution of the feature map. Fully connected layers have 128-256 neurons and add multi-layer attention mechanism modules to enhance the model's ability to extract and focus on key features. The model input is the multimodal fused feature vector, and the output is the predicted result of defect type, location, and severity. Model training and optimization: The deep neural network model was trained using the training dataset, employing the AdamW optimization algorithm with an initial learning rate of 0.0008 and a weight decay coefficient of 0.00008. Overfitting was prevented by using L1 regularization (0.0008), L2 regularization (0.0008), Dropout probability of 0.15, and Layer Normalization. The model was trained for 800 iterations, with model parameters saved every 30 iterations to facilitate backtracking and analysis of the training process for different chip component types. The finely preprocessed and deep feature-fused test data was input into the trained deep neural network model. The model could output detailed information such as defect type, location coordinates (accuracy ±0.08mm), and severity assessment (accuracy ±4%) within 0.08 seconds. Simultaneously, the model utilized time series analysis and deep learning prediction algorithms to predict defect development trends and provide early warnings of potential quality issues.

[0041] Step S4: Input the defect identification results into the constructed welding process parameter optimization strategy network, and output the real-time adjustment strategy results for adaptive adjustment of welding process parameters.

[0042] In this embodiment, the welding process parameter optimization strategy network in step S4 uses the near-end strategy optimization (PPO) as a framework. The state space of the optimization strategy network is configured as a three-dimensional state space including defect type, defect location coordinates, and defect severity. The action space is configured as a three-dimensional continuous variable action space including welding temperature, pressure, and time. The multi-objective reward function R is set as follows: In the above formula, Acc represents the improvement rate of detection accuracy, Eff represents production efficiency, and Ene represents energy consumption cost. , , This represents the weighting coefficient.

[0043] It should be noted that before performing the aforementioned steps in this embodiment, the process control system parameters need to be initialized. Basic information such as the type, size, and material of the chip components is automatically identified through various methods such as image recognition and sensor detection. Combined with real-time parameters of the current detection environment, such as ambient temperature accuracy of ±0.08℃ and humidity accuracy of ±1.5%, the aforementioned detection method in this embodiment automatically calculates and adjusts the machine vision camera's shooting parameters within 0.8 seconds. Key detection parameters include: exposure time adjusted to 75-85 microseconds and frame rate adjusted to 40-45 frames / second; infrared thermal imager's temperature measurement range adjusted to 30℃-105℃ and resolution adjusted to 1280×720-1920×1080; and ultrasonic scanning equipment's probe frequency adjusted to 7MHz-8MHz and scanning speed adjusted to 0.8mm / s-1.2mm / s.

[0044] This embodiment also enables real-time parameter optimization. During the detection process, the system monitors the quality and stability of the detection data every 3 seconds. Using the aforementioned detection method, the accuracy, completeness, and consistency of the data are checked. This embodiment can quickly and accurately identify outliers and noise in the data. If data anomalies or poor detection results are detected, the detection parameters are readjusted in real-time within 0.4 seconds by comparing with pre-set standard data, ensuring that each detection device is always in optimal working condition. Furthermore, this embodiment can also adjust the detection parameters in advance based on the changing trends of the detection data, preventing detection errors.

[0045] Figure 2 This is a schematic diagram of a quality inspection device frame for a FC flip-soldering chip assembly according to one embodiment. The quality inspection device 200 includes: The multimodal data acquisition module 201 is used to acquire multimodal data of the chip assembly after FC flip soldering in real time and generate multimodal data to be fused after preprocessing the multimodal data. The multimodal data fusion module 202 is used to extract high-precision features from the multimodal data to be fused, normalize the feature vectors of different modalities, and then splice them according to the dynamic weight allocation rules to generate image data to be recognized. The chip component defect identification module 203 is used to input the image data to be identified into a pre-constructed deep learning-based defect identification model for defect identification, and output defect identification results based on defect type, defect location coordinates and defect severity. The adaptive adjustment module 204 is used to input the defect identification result into the constructed welding process parameter optimization strategy network and output the real-time adjustment strategy result for adaptive adjustment of welding process parameters. The multimodal data includes machine vision image data, infrared thermal imaging data, and ultrasonic scanning image data.

[0046] This embodiment also discloses an electronic device, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, it causes the processor to perform the steps of a quality inspection method for FC flip-soldering chip components.

[0047] Specifically, the electronic device includes a processor and a memory connected via a system bus. The processor provides computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. These computer programs can be executed by the processor to implement the quality inspection methods for FC flip-soldering chip assemblies provided in the following embodiments. The internal memory provides a cached runtime environment for the operating system computer programs in the non-volatile storage media.

[0048] The various modules in the apparatus provided in this application embodiment can be implemented in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements the steps of the method described in the embodiments of this application.

[0049] This embodiment also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of a quality inspection method for FC flip-soldering chip assemblies. A computer program product containing instructions, which, when run on a computer, causes the computer to perform quality inspection of FC flip-soldering chip assemblies.

[0050] Any references to memory, storage, database, or other media used in the embodiments of this application may include non-volatile and / or volatile memory. Suitable non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0051] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0052] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A quality inspection method for FC flip-soldering chip components, characterized in that, Specifically, the steps include the following: Step S1: Real-time synchronous acquisition of multimodal data of the chip assembly after FC flip-soldering; preprocessing of the multimodal data to generate multimodal data to be fused; Step S2: Extracting high-precision features from the multimodal data to be fused; normalizing the feature vectors of different modalities; and splicing them according to dynamic weight allocation rules to generate image data to be identified; Step S3: Inputting the image data to be identified into a pre-constructed deep learning-based defect identification model for defect identification; and outputting defect identification results based on defect type, defect location coordinates, and defect severity. Step S4: Input the defect identification results into the constructed welding process parameter optimization strategy network, and output the real-time adjustment strategy results for adaptive adjustment of welding process parameters; The multimodal data includes machine vision image data, infrared thermal imaging data, and ultrasonic scanning image data.

2. The quality inspection method according to claim 1, characterized in that, The step S1, which involves preprocessing the multimodal data to generate the multimodal data to be fused, specifically includes: For the acquired machine vision image data, after Gaussian filtering for noise reduction, the image contrast and clarity are enhanced by an adaptive histogram equalization algorithm. Then, an edge detection algorithm is used to segment the solder joints and chips to generate the first data to be fused. For the collected infrared thermal imaging data, the infrared imaging data is calibrated by a temperature calibration model based on deep learning to generate a second data to be fused. The second data to be fused is then aligned with the first data to be fused in spatial position to realize that the temperature data in the infrared imaging data corresponds to the actual position of the chip component. For ultrasound scan image data, a third set of data to be fused is generated after denoising the ultrasound scan image data through wavelet transform and filtering. Multimodal data to be fused is constructed from the first data to be fused, the second data to be fused, and the third data to be fused.

3. The quality inspection method according to claim 2, characterized in that, Step S2 specifically includes: Extract a first feature based on the geometry and grayscale of the solder joints from the first data to be fused; Extract a second feature based on temperature gradient, hotspot location, and temperature change trend from the second data to be fused; Extract a third feature based on the reflection signal parameters from the third set of data to be fused; A multimodal feature vector based on a first feature, a second feature, and a third feature is constructed. After normalizing the multimodal feature vector, a weight adaptive adjustment model is used to dynamically assign corresponding weights to the first feature, the second feature, and the third feature, respectively. The weights and corresponding features are concatenated and fused to generate the image data to be recognized.

4. The quality inspection method according to claim 3, characterized in that, The weight adaptive adjustment model adopts a Bayesian neural network and uses the Monte Carlo Dropout mechanism. It obtains the probability distribution of the feature vector output of each modality through multiple forward propagations, and estimates the uncertainty of each modality based on the variance of the distribution to achieve dynamic weight allocation.

5. The quality inspection method according to claim 4, characterized in that, In step S3, the deep learning-based defect recognition model adopts a multimodal fusion architecture, including a spatial attention mechanism (SAM) module, a channel attention mechanism (CAM) module, and a feature fusion layer. The SAM module generates spatial weights by calculating the spatial correlation of each position in the feature map of the image data to be recognized. The CAM module calculates the statistical information of the feature map in the image data to be recognized and generates channel weights based on the statistical information. The feature fusion layer realizes the feature fusion of the spatial weights and the channel weights through a weighted summation process and outputs the fused feature map.

6. The quality inspection method according to claim 5, characterized in that, The welding process parameter optimization strategy network in step S4 uses the near-end strategy optimization (PPO) framework. The state space of the optimization strategy network is configured as a three-dimensional state space including defect type, defect location coordinates, and defect severity. The action space is configured as a three-dimensional continuous variable action space including welding temperature, pressure, and time. The multi-objective reward function R is set as follows: In the above formula, Acc represents the improvement rate of detection accuracy, Eff represents production efficiency, and Ene represents energy consumption cost. , , This represents the weighting coefficient.

7. The quality inspection method according to claim 5, characterized in that, The deep learning-based defect recognition model also includes convolutional neural networks based on different receptive fields, specifically including: Small-scale convolutional layers with receptive fields of 2×2 to 4×4 employ deformable convolutional kernels, adaptively adjusting the shape and position of the kernels to extract subtle local feature information of chip components and locate the position and shape of minute defects; Large-scale convolutional layers with receptive fields of 8×8 to 10×10 employ a combination of dilated convolution and grouped convolution to identify defects or structural anomalies over a large area based on the overall structural characteristics of the chip assembly.

8. A quality inspection device for FC flip-soldering chip assemblies, used to perform the method as described in any one of claims 1-7, characterized in that, The quality testing device includes: A multimodal data acquisition module is used to synchronously acquire multimodal data of the chip assembly after FC flip-soldering in real time, and generate multimodal data to be fused after preprocessing the multimodal data; a multimodal data fusion module is used to extract high-precision features from the multimodal data to be fused, normalize the feature vectors of different modalities, and then stitch them together according to dynamic weight allocation rules to generate image data to be identified; a defect identification module is used to input the image data to be identified into a pre-constructed deep learning-based defect identification model for defect identification, and output defect identification results based on defect type, defect location coordinates, and defect severity. An adaptive adjustment module is used to input the defect identification results into the constructed welding process parameter optimization strategy network and output real-time adjustment strategy results for adaptive adjustment of welding process parameters. The multimodal data includes machine vision image data, infrared thermal imaging data, and ultrasonic scanning image data.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images