Automated control system and method for semiconductor packaging

Abstract

The present application relates to an automated control system and method for semiconductor packaging. The method includes: using deep learning-based image processing technology to perform image feature analysis on the device component welding state image, respectively extracting the local boundary detail features and global structural semantic features of the device component welding area, and using the component welding state boundary features to assist in adjusting its global structural semantic features, so as to integrate multi-modal complementary information and enhance the feature expression of the welding area, thereby intelligently judging whether the component welding quality is qualified, and generating early warning prompts for unqualified products. In this way, real-time and automated detection of welding quality in the semiconductor packaging process can be achieved, effectively reducing the risk of device failure caused by poor welding, and improving the quality and production efficiency of semiconductor products.

Description

Technical Field

[0001] The present application relates to the field of automated control technology, and in particular, to an automated control system and method for semiconductor packaging. Background Art

[0002] In the current semiconductor packaging industry, with the rapid development of microelectronics technology, the requirements for the welding process of device components are getting higher and higher. In the prior art, the welding of semiconductor components usually adopts a single reflow composite secondary reflow process. The first reflow welds the chip and the DBC board together, and the second reflow welds the DBC board and the heat sink together. This welding process helps to control the void rate of the product. However, the use of the two-time welding process leads to a longer process of semiconductor power devices, resulting in a waste of resources and increased production costs. At the same time, due to the change in the thermal conductivity of the DBC board during the two welding processes, the welding temperature requirements are strict during the second reflow, and experienced operators are required to perform the second reflow operation, resulting in increased labor costs.

[0003] In this regard, patent CN117594460A discloses a one-time welding forming method for semiconductor power device packaging. By changing the reflow soldering process, the semiconductor power device can be welded in place at one time during packaging, thereby shortening the process of semiconductor power devices, reducing resource waste, and reducing production costs. However, although this method simplifies the welding process, it lacks detection of component welding quality. In the prior art, the detection of welding quality mainly relies on manual visual inspection. This detection method is inefficient, easily affected by subjective factors, and cannot achieve real-time automatic early warning and control of welding quality. In actual applications, there may be a risk of semiconductor device failure due to poor welding.

[0004] Therefore, an optimized automated control method for semiconductor packaging is desired. Summary of the invention

[0005] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] In a first aspect, the present application provides an automated control method for semiconductor packaging, the method comprising: placing a plurality of first solder sheets on the top surface of a heat sink, placing a DBC board on the top of a single first solder sheet, placing a plurality of second solder sheets on the top surface of a single DBC board, and placing a chip on the top of a single second solder sheet, thereby obtaining a device assembly; placing the device assembly in a reflow oven, heating the preheating chamber of the reflow oven from room temperature to 120°C, and then heating the chamber from 120°C to 275°C within a set heating time to preheat the device assembly; after a first heat preservation time, heating the reflow oven to a temperature of 120°C; and heating the chamber from 120°C to 275°C. The device assembly in the preheating chamber of the flow soldering furnace is transferred to the soldering chamber of the reflow soldering furnace for soldering to obtain a soldered device assembly, wherein the temperature in the soldering chamber of the reflow soldering furnace is 290°C±10°C; after the second insulation time, the soldered device assembly in the soldering chamber of the reflow soldering furnace is transferred to the cooling chamber of the reflow soldering furnace, and taken out after the set cooling time to obtain a formed device assembly; the formed device assembly is subjected to a welding quality inspection to generate an unqualified early warning prompt; wherein the welding quality inspection of the formed device assembly to generate an unqualified early warning prompt includes:

[0007] Acquire a device component welding state image captured by a camera;

[0008] Performing welding region multi-scale feature extraction on the welding state image of the device component to obtain a welding region boundary feature vector and a welding region structure semantic feature map;

[0009] Performing attention enhancement on the welding region structure semantic feature map to obtain an enhanced welding region structure semantic feature map;

[0010] Using the welding area boundary feature vector as the auxiliary modal condition vector, batch normalization of the enhanced welding area structure semantic feature map based on multimodal auxiliary information is performed to obtain a conditionally constrained normalized welding area structure semantic feature map;

[0011] Based on the conditionally constrained normalized welding area structural semantic feature map, it is determined whether to generate the unqualified early warning prompt.

[0012] Optionally, multi-scale feature extraction of the welding area is performed on the welding state image of the device component to obtain a welding area boundary feature vector and a welding area structure semantic feature map, including: performing welding area detection on the welding state image of the device component to obtain an image of interest in the welding area; inputting the image of interest in the welding area into a boundary feature extraction branch based on a convolutional layer to obtain the welding area boundary feature vector; inputting the image of interest in the welding area into a welding structure feature extractor based on a void convolutional neural network model to obtain the welding area structure semantic feature map.

[0013] Optionally, performing welding area detection on the welding state image of the device assembly to obtain an image of interest in the welding area includes: inputting the welding state image of the device assembly into a welding area detector based on a Yolo model to obtain the image of interest in the welding area.

[0014] Optionally, the welding area structure semantic feature map is subjected to attention enhancement to obtain an enhanced welding area structure semantic feature map, including: inputting the welding area structure semantic feature map into a feature attention enhancement network under a compression-suppression architecture to obtain the enhanced welding area structure semantic feature map.

[0015] Optionally, the welding area structure semantic feature map is input into a feature attention enhancement network under a compression-suppression architecture to obtain the enhanced welding area structure semantic feature map, including: calculating the global mean of each feature matrix of the welding area structure semantic feature map along the channel dimension to obtain a welding area structure semantic compression information representation vector; performing multi-scale feature extraction on the welding area structure semantic compression information representation vector to obtain a welding area structure semantic compression information multi-scale association feature vector; using a Sigmoid function to normalize the welding area structure semantic compression information multi-scale association feature vector to obtain a welding area structure semantic feature weight vector; based on the welding area structure semantic feature weight vector, performing feature amplification and suppression operations on the welding area structure semantic feature map to obtain the enhanced welding area structure semantic feature map.

[0016] Optionally, multi-scale feature extraction is performed on the welding area structure semantic compression information representation vector to obtain a multi-scale associated feature vector of the welding area structure semantic compression information, including: performing one-dimensional convolution encoding on the welding area structure semantic compression information representation vector to obtain a welding area structure semantic compression information association representation feature vector; cascading the welding area structure semantic compression information representation vector and the welding area structure semantic compression information association representation feature vector to obtain a welding area structure semantic compression information multi-scale representation vector; inputting the welding area structure semantic compression information multi-scale representation vector into a compression information feature extraction module based on a multi-layer perceptron model to obtain the welding area structure semantic compression information multi-scale associated feature vector, wherein the multi-layer perceptron model comprises two fully connected layers and a SiLU activation function.

[0017] Optionally, based on the welding area structural semantic feature weight vector, the welding area structural semantic feature map is subjected to feature amplification and suppression operations to obtain the enhanced welding area structural semantic feature map, including: calculating the corresponding multiplication of each eigenvalue of the welding area structural semantic feature weight vector and each feature matrix of the welding area structural semantic feature map along the channel dimension to obtain the enhanced welding area structural semantic feature map.

[0018] Optionally, the welding area boundary feature vector is used as the auxiliary modal condition vector, and the enhanced welding area structural semantic feature map is batch normalized based on the feature map of multimodal auxiliary information to obtain a conditionally constrained normalized welding area structural semantic feature map, including: based on the mean and variance of each feature matrix of the enhanced welding area structural semantic feature map along the channel dimension, the enhanced welding area structural semantic feature map is batch normalized to obtain a normalized welding area structural semantic feature map; the normalized welding area structural semantic feature map is expanded into a one-dimensional vector and then feature cascaded with the auxiliary modal condition vector to obtain a welding area multimodal joint feature vector; the welding area multimodal joint feature vector is passed through a multilayer perceptron to obtain a scaling factor and a translation factor, wherein the multilayer perceptron includes a first fully connected layer and a second fully connected layer; based on the scaling factor and the translation factor, the enhanced welding area structural semantic feature map is batch normalized to obtain the conditionally constrained normalized welding area structural semantic feature map.

[0019] Optionally, based on the conditionally constrained normalized welding area structural semantic feature map, determining whether to generate the unqualified early warning prompt includes: inputting the conditionally constrained normalized welding area structural semantic feature map into a classifier-based welding detector to obtain a detection result, and the detection result is used to indicate whether the welding quality is qualified; in response to the detection result that the welding quality is unqualified, generating the unqualified early warning prompt.

[0020] In a second aspect, the present application provides an automated control system for semiconductor packaging, the system comprising:

[0021] A welding state image acquisition module is used to acquire a welding state image of a device component collected by a camera;

[0022] A multi-scale feature extraction module, used for performing multi-scale feature extraction of the welding area of ​​the device assembly welding state image to obtain a welding area boundary feature vector and a welding area structure semantic feature map;

[0023] An attention enhancement module, used for enhancing the attention of the welding area structure semantic feature map to obtain an enhanced welding area structure semantic feature map;

[0024] A feature map batch normalization module is used to use the welding area boundary feature vector as an auxiliary modal condition vector to perform feature map batch normalization on the enhanced welding area structure semantic feature map based on multimodal auxiliary information to obtain a conditionally constrained normalized welding area structure semantic feature map;

[0025] The early warning prompt generation module is used to determine whether to generate an unqualified early warning prompt based on the conditional constraint normalized welding area structure semantic feature map.

[0026] The above technical solution is adopted to analyze the image features of the device component welding state image by using deep learning-based image processing technology, extract the local boundary detail features and global structural semantic features of the device component welding area, and use the component welding state boundary features to assist in adjusting its global structural semantic features, so as to integrate multi-modal complementary information and enhance the feature expression of the welding area, so as to intelligently judge whether the component welding quality is qualified, and generate early warning prompts for unqualified products. In this way, real-time and automated detection of welding quality in the semiconductor packaging process can be achieved, effectively reducing the risk of device failure caused by poor welding, and improving the quality and production efficiency of semiconductor products.

[0027] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0029] Figure 1 The invention is a flow chart showing an automatic control method for semiconductor packaging according to an exemplary embodiment.

[0030] Figure 2 The invention is a block diagram showing an automatic control system for semiconductor packaging according to an exemplary embodiment.

[0031] Figure 3 It is a block diagram of an electronic device according to an exemplary embodiment.

[0032] Figure 4 It is an application scenario diagram of an automatic control method for semiconductor packaging according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0034] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0035] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0036] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0037] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0038] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0039] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0040] Figure 1 is a flow chart of an automatic control method for semiconductor packaging according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0041] Step S101, placing a plurality of first soldering sheets on the top surface of the heat sink, placing a DBC board on the top of a single first soldering sheet, placing a plurality of second soldering sheets on the top surface of the single DBC board, and placing a chip on the top of a single second soldering sheet, thereby obtaining a device assembly;

[0042] Step S102, placing the device assembly into a reflow oven, heating the preheating chamber of the reflow oven from room temperature to 120° C., and then heating from 120° C. to 275° C. within a set heating time, to preheat the device assembly;

[0043] Step S103, after the first insulation time, the device assembly in the preheating chamber of the reflow oven is transferred to the welding chamber of the reflow oven for welding to obtain a welded device assembly, wherein the temperature in the welding chamber of the reflow oven is 290°C±10°C;

[0044] Step S104, after the second holding time, the soldered device assembly in the soldering chamber of the reflow soldering furnace is transferred to the cooling chamber of the reflow soldering furnace, and taken out after a set cooling time to obtain a formed device assembly;

[0045] Step S105, performing welding quality inspection on the molded device assembly to generate an unqualified early warning prompt; wherein step S105, performing welding quality inspection on the molded device assembly to generate an unqualified early warning prompt, includes:

[0046] Step S1051, obtaining a device assembly welding state image captured by a camera;

[0047] Step S1052, performing welding region multi-scale feature extraction on the welding state image of the device assembly to obtain a welding region boundary feature vector and a welding region structure semantic feature map;

[0048] Step S1053, performing attention enhancement on the welding region structure semantic feature map to obtain an enhanced welding region structure semantic feature map;

[0049] Step S1054, using the welding area boundary feature vector as the auxiliary modal condition vector, performing batch normalization of the feature map of the enhanced welding area structure semantic feature map based on multimodal auxiliary information to obtain a conditionally constrained normalized welding area structure semantic feature map;

[0050] Step S1055: Determine whether to generate the unqualified warning prompt based on the conditionally constrained normalized welding area structural semantic feature map.

[0051] Patent CN117594460A discloses a one-time welding molding method for semiconductor power device packaging, comprising: placing a plurality of first welding sheets on the top surface of a heat sink, placing a DBC board on the top of a single first welding sheet, placing a plurality of second welding sheets on the top surface of a single DBC board, and placing a chip on the top of a single second welding sheet, thereby obtaining a device assembly; placing the device assembly in a reflow oven, heating the preheating chamber of the reflow oven from room temperature to 120°C, and then heating the preheating chamber from 120°C to 275°C within a set heating time to preheat the device assembly; after a first insulation time, transferring the device assembly in the preheating chamber of the reflow oven to a welding chamber of the reflow oven for welding, thereby obtaining a welded device assembly, wherein the temperature in the welding chamber of the reflow oven is 290°C±10°C; after a second insulation time, transferring the welded device assembly in the welding chamber of the reflow oven to a cooling chamber of the reflow oven, and taking it out after a set cooling time, thereby obtaining a molded device assembly.

[0052] Although this method simplifies the welding process, it lacks the detection of component welding quality. In the prior art, the detection of welding quality mainly relies on manual visual inspection, which is inefficient, easily affected by subjective factors, and cannot achieve real-time automatic warning and control of welding quality. In practical applications, there may be a risk of semiconductor device failure due to poor welding.

[0053] Based on this, in the technical solution of the present application, the welding quality of the molded device assembly is further inspected to generate an unqualified early warning prompt. Specifically, the technical concept of the present application is to use deep learning-based image processing technology to perform image feature analysis on the device assembly welding state image, respectively extract the local boundary detail features and global structural semantic features of the device assembly welding area, and use the component welding state boundary features to assist in adjusting its global structural semantic features, so as to integrate multi-modal complementary information and enhance the feature expression of the welding area, so as to intelligently judge whether the assembly welding quality is qualified, and generate early warning prompts for unqualified products. In this way, real-time and automated detection of welding quality in the semiconductor packaging process can be achieved, effectively reducing the risk of device failure due to poor welding, and improving the quality and production efficiency of semiconductor products.

[0054] Specifically, in the technical solution of the present application, first, the device assembly welding state image captured by the camera is obtained. It should be understood that by capturing the device assembly welding state image by the camera, the boundary clarity of the welding area, the shape and size of the solder joint, the distribution of the welding material, etc. can be clearly displayed, which provides instant data input for the automated detection and control of welding quality, and helps to achieve timely and efficient welding quality detection, thereby ensuring the continuity and efficiency of the semiconductor packaging process.

[0055] In one embodiment of the present application, multi-scale feature extraction of the welding area is performed on the welding state image of the device component to obtain a welding area boundary feature vector and a welding area structure semantic feature map, including: performing welding area detection on the welding state image of the device component to obtain an image of interest in the welding area; inputting the image of interest in the welding area into a boundary feature extraction branch based on a convolutional layer to obtain the welding area boundary feature vector; inputting the image of interest in the welding area into a welding structure feature extractor based on a void convolutional neural network model to obtain the welding area structure semantic feature map.

[0056] Next, considering that the device assembly welding state image may contain a large amount of background information, it not only increases the complexity of data processing, but also may interfere with the judgment of welding quality. Therefore, the welding area in the device assembly welding state image is further positioned and processed by the Yolo model to generate an image of interest in the welding area, so that the welding area itself can be focused on in the subsequent image feature extraction process, irrelevant background information can be eliminated, and the accuracy and efficiency of subsequent feature extraction can be improved.

[0057] In one embodiment of the present application, welding area detection is performed on the welding state image of the device component to obtain a welding area image of interest, including: inputting the welding state image of the device component into a welding area detector based on a Yolo model to obtain the welding area image of interest.

[0058] Then, considering that the boundary information of the welding area is of key significance for the judgment of welding quality, the clarity, continuity and shape of the boundary can directly reflect the quality of the welding process. Therefore, in the technical solution of the present application, a boundary feature extraction branch based on a convolution layer is used to extract boundary features of the image of interest in the welding area. Among them, the boundary feature extraction branch can automatically learn and extract the local boundary features of the welding area by using a small-scale convolution kernel to perform sliding convolution processing on the image of interest in the welding area, capture the fine texture and slight changes in the welding area, and thus generate a welding area boundary feature vector, which provides important boundary detail information for subsequent welding quality judgment.

[0059] At the same time, considering that in addition to boundary features, the global structural information of the welding area also has important characterization significance for the evaluation of welding quality. Therefore, in the technical solution of the present application, a welding structure feature extractor based on a dilated convolutional neural network (DCNN) is introduced to process the image of interest in the welding area, so as to utilize the unique dilated convolutional neural network structure, without sacrificing spatial resolution, through multi-layer convolution operations, to capture a wider range of deep contextual semantic information of the image, revealing the macro layout of the welding area, the distribution of weld points, and possible potential defects such as voids, cracks or lack of fusion, so as to obtain a structural semantic feature map of the welding area.

[0060] Considering that as the number of network layers increases, small welding defect information is usually gradually diluted or lost during the feature transmission process. In order to effectively retain these key information and enhance the network's sensitivity to important features, in this application, a feature attention enhancement network under a compression-suppression architecture is introduced to enhance the welding area structural semantic feature map to obtain an enhanced welding area structural semantic feature map. Specifically, the feature attention enhancement network first uses a global average pooling operation to perform feature compression on the welding area structural semantic feature map to obtain global compression information, and then through a series of nonlinear transformations and multi-scale feature learning, deep mining of the global compression information is achieved to generate attention weights corresponding to each feature channel of the welding area structural semantic feature map, and then through weighted operations, each feature channel of the welding area structural semantic feature map is amplified and suppressed to achieve selective enhancement of small defect information, thereby significantly improving the discriminability of feature representation and providing a more reliable feature basis for subsequent intelligent judgment.

[0061] In one embodiment of the present application, attention is enhanced on the welding area structure semantic feature map to obtain an enhanced welding area structure semantic feature map, including: inputting the welding area structure semantic feature map into a feature attention enhancement network under a compression-suppression architecture to obtain the enhanced welding area structure semantic feature map.

[0062] Furthermore, in one embodiment of the present application, the welding area structure semantic feature map is input into a feature attention enhancement network under a compression-suppression architecture to obtain the enhanced welding area structure semantic feature map, including: calculating the global mean of each feature matrix of the welding area structure semantic feature map along the channel dimension to obtain a welding area structure semantic compression information representation vector; performing multi-scale feature extraction on the welding area structure semantic compression information representation vector to obtain a welding area structure semantic compression information multi-scale association feature vector; using a Sigmoid function to perform a normalization operation on the welding area structure semantic compression information multi-scale association feature vector to obtain a welding area structure semantic feature weight vector; based on the welding area structure semantic feature weight vector, performing feature amplification and suppression operations on the welding area structure semantic feature map to obtain the enhanced welding area structure semantic feature map.

[0063] Furthermore, in one embodiment of the present application, multi-scale feature extraction is performed on the welding area structure semantic compression information representation vector to obtain a multi-scale associated feature vector of the welding area structure semantic compression information, including: performing one-dimensional convolution encoding on the welding area structure semantic compression information representation vector to obtain a welding area structure semantic compression information association representation feature vector; cascading the welding area structure semantic compression information representation vector and the welding area structure semantic compression information association representation feature vector to obtain a welding area structure semantic compression information multi-scale representation vector; inputting the welding area structure semantic compression information multi-scale representation vector into a compression information feature extraction module based on a multi-layer perceptron model to obtain the welding area structure semantic compression information multi-scale associated feature vector, wherein the multi-layer perceptron model comprises two fully connected layers and a SiLU activation function.

[0064] Furthermore, based on the welding area structural semantic feature weight vector, the welding area structural semantic feature map is subjected to feature amplification and suppression operations to obtain the enhanced welding area structural semantic feature map, including: calculating the corresponding multiplication of each eigenvalue of the welding area structural semantic feature weight vector and each feature matrix of the welding area structural semantic feature map along the channel dimension to obtain the enhanced welding area structural semantic feature map.

[0065] Specifically, the welding area structure semantic feature map is subjected to attention selection using the following feature attention selection and enhancement formula to obtain the enhanced welding area structure semantic feature map, wherein the feature attention selection and enhancement formula is:

[0066]

[0067] V s =concat(V s1, Conv 1d (V s1 ))

[0068]

[0069] Wherein, F represents the structural semantic feature map of the welding area, The xth pixel represents the semantic feature map of the welding area structure o The coordinates in the channel are (i o , j o ), H represents the height of the welding area structure semantic feature map, W represents the width of the welding area structure semantic feature map, is the xth vector representing the semantic compression information of the welding area structure o The characteristic value, V s1 The semantic compression information representation vector of the welding area structure is represented by Conv 1d (·) represents a one-dimensional convolution operation, concat(·,·) represents a cascade operation, V s represents the multi-scale representation vector of the semantic compression information of the welding area structure, MLP represents the multi-layer perceptron, sigmoid represents the sigmoid function, represents the channel-by-channel dot product, and F′ represents the structural semantic feature map of the enhanced welding area.

[0070] Then, the feature vector of the welding area boundary is used as the auxiliary modal condition vector, and the feature map of the enhanced welding area structure semantic feature map is batch normalized based on the multimodal auxiliary information to obtain the conditional constraint normalized welding area structure semantic feature map. It should be understood that since the boundary features of the welding area and its global structure semantic features have significant complementarity in welding quality assessment, the organic fusion of the two can more comprehensively and accurately reflect the welding state. To this end, in this application, a feature map batch normalization method based on multimodal auxiliary information is introduced, and the enhanced welding area structure semantic feature map is conditionally constrained based on the welding area boundary feature vector, and the normalization parameters (such as scaling factor and offset) of the enhanced welding area structure semantic feature map are dynamically adjusted by performing association analysis on the welding area boundary feature vector and the enhanced welding area structure semantic feature map, and its feature distribution is adjusted so that the enhanced welding area structure semantic feature map can more accurately reflect the subtle changes of the welding boundary while maintaining the global structure information, and enhance the sensitivity of the model to subtle differences in welding quality. In this way, the quality of complementary feature fusion is effectively improved, laying a solid foundation for the comprehensive evaluation of welding quality.

[0071] In one embodiment of the present application, the welding area boundary feature vector is used as the auxiliary modal condition vector, and the enhanced welding area structure semantic feature map is batch normalized based on the feature map of multimodal auxiliary information to obtain a conditionally constrained normalized welding area structure semantic feature map, including: based on the mean and variance of each feature matrix of the enhanced welding area structure semantic feature map along the channel dimension, the enhanced welding area structure semantic feature map is batch normalized to obtain a normalized welding area structure semantic feature map; the normalized welding area structure semantic feature map is expanded into a one-dimensional vector and then feature cascaded with the auxiliary modal condition vector to obtain a welding area multimodal joint feature vector; the welding area multimodal joint feature vector is passed through a multilayer perceptron to obtain a scaling factor and a translation factor, wherein the multilayer perceptron includes a first fully connected layer and a second fully connected layer; based on the scaling factor and the translation factor, the enhanced welding area structure semantic feature map is batch normalized to obtain the conditionally constrained normalized welding area structure semantic feature map.

[0072] Specifically, the enhanced welding region structure semantic feature map is processed by the following conditional constraint batch normalization formula to obtain the conditional constraint normalized welding region structure semantic feature map, wherein the conditional constraint batch normalization formula is:

[0073]

[0074] Among them, μ B and σ B 2 represent the mean and variance of the Bth feature matrix of the enhanced welding area structural semantic feature map along the channel dimension, x Bi represents the i-th eigenvalue in the B-th eigenmatrix, represents the i-th eigenvalue of the B-th feature matrix along the channel dimension of the normalized welding area structure semantic feature map, represents the multimodal joint feature vector of the welding area, MLP γ Represents the first fully connected layer of the multi-layer perceptron model, MLP β represents the second fully connected layer of the multilayer perceptron model, γ mlp represents the scaling factor, β mlp represents the translation factor, Represents the i-th eigenvalue of the B-th feature matrix along the channel dimension of the conditionally constrained normalized welding area structural semantic feature map.

[0075] Next, the conditional constraint normalized welding area structure semantic feature map is input into a welding detector based on a classifier to obtain a detection result, and the detection result is used to indicate whether the welding quality is qualified. It should be understood that after training with a large number of welding quality samples, the classifier has effectively learned the complex mapping relationship between welding quality features and target categories, so that it can accurately map the input conditional constraint normalized welding area structure semantic feature map to the preset welding quality category, and output a detection result indicating whether the welding quality is qualified. Furthermore, in response to the detection result that the welding quality is unqualified, an unqualified warning prompt is generated so as to promptly notify relevant personnel to handle it.

[0076] In one embodiment of the present application, based on the conditionally constrained normalized welding area structure semantic feature map, determining whether to generate the unqualified early warning prompt includes: inputting the conditionally constrained normalized welding area structure semantic feature map into a classifier-based welding detector to obtain a detection result, and the detection result is used to indicate whether the welding quality is qualified; in response to the detection result that the welding quality is unqualified, generating the unqualified early warning prompt.

[0077] Preferably, inputting the conditionally constrained normalized welding region structure semantic feature map into a classifier-based welding detector to obtain a detection result comprises:

[0078] Calculating the sum of eigenvalues ​​of the welding region structural semantic feature map and the mean feature map of the conditionally constrained normalized welding region structural semantic feature map and the square root of the sum of squares of the eigenvalues;

[0079] Calculating a weighted sum of the inverse of the square root of the sum of squares of the eigenvalues ​​and the sum of the eigenvalues, and performing point multiplication with the point addition and sum feature map of the welding region structure semantic feature map and the conditional constraint normalized welding region structure semantic feature map to obtain a first conditional constraint normalized welding region structure semantic sub-feature map;

[0080] Performing a dot multiplication of the dot product feature map of the welding region structure semantic feature map and the conditional constraint normalized welding region structure semantic feature map and the square root of the scale of the conditional constraint normalized welding region structure semantic feature map to obtain a second conditional constraint normalized welding region structure semantic sub-feature map;

[0081] Calculating the weighted point sum of the first conditionally constrained normalized welding region structure semantic sub-feature graph and the second conditionally constrained normalized welding region structure semantic sub-feature graph to obtain a corrected conditionally constrained normalized welding region structure semantic feature graph; and

[0082] The correction condition constraint normalized welding area structure semantic feature map is input into a classifier-based welding detector to obtain a detection result.

[0083] Among them, the correction condition constrained normalized welding area structure semantic feature map is expressed as:

[0084]

[0085] F1 and F2 are the welding area structure semantic feature map and the conditional constraint normalized welding area structure semantic feature map, respectively. μ is the mean feature map of the welding region structure semantic feature map and the conditional constraint normalized welding region structure semantic feature map, ‖F μ || L1 represents the sum of the eigenvalues ​​of the mean feature map, ‖F μ || L2 represents the square root of the sum of squares of the eigenvalues ​​of the mean feature map, S is the scale of the conditionally constrained normalized welding region structure semantic feature map, i.e., width multiplied by height multiplied by the number of channels, and α and β are weighted sums of weights as hyperparameters, represents point-wise addition, ⊙ represents point-wise multiplication, and F2′ represents the normalized welding area structural semantic feature map constrained by the correction condition.

[0086] Here, considering that the welding area boundary feature vector is used as the auxiliary modal condition vector, after the enhanced welding area structural semantic feature map is batch normalized based on the feature map of multimodal auxiliary information, the conditionally constrained normalized welding area structural semantic feature map obtained has an expression offset relative to the source image feature semantics of the welding area image of interest, thereby causing category offset. The present application hopes to compensate for the category offset caused by the imbalance of feature correspondence ratio under the fine-grained distribution of image semantics by fusing the welding area structural semantic feature map and the conditionally constrained normalized welding area structural semantic feature map, thereby improving the accuracy of the classification results.

[0087] Therefore, based on the constrained representation of the structured foreground and background distinction of the supermanifold of the welding area structural semantic feature map and the conditionally constrained normalized welding area structural semantic feature map based on the mean feature map norm between the welding area structural semantic feature map and the conditionally constrained normalized welding area structural semantic feature map, the feature-level key correspondence between the feature maps of the welding area structural semantic feature map and the conditionally constrained normalized welding area structural semantic feature map is modeled, and the global association relationship between the corresponding features is adjusted, so as to make positive fine-grained correspondence suggestions through the unbalanced proportion control between the corresponding feature values ​​of the welding area structural semantic feature map and the conditionally constrained normalized welding area structural semantic feature map, so as to avoid the imbalance of feature map level categories of the conditionally constrained normalized welding area structural semantic feature map by focusing on the focus, thereby improving the accuracy of the detection results obtained by inputting the conditionally constrained normalized welding area structural semantic feature map into the classifier-based welding detector.

[0088] In summary, the above scheme uses deep learning-based image processing technology to analyze the image features of the device component welding state image, extracts the local boundary detail features and global structural semantic features of the device component welding area, and uses the component welding state boundary features to assist in adjusting its global structural semantic features, so as to integrate multi-modal complementary information and enhance the feature expression of the welding area, so as to intelligently judge whether the component welding quality is qualified, and generate early warning prompts for unqualified products. In this way, real-time and automated detection of welding quality in the semiconductor packaging process can be achieved, effectively reducing the risk of device failure caused by poor welding, and improving the quality and production efficiency of semiconductor products.

[0089] Figure 2 The invention is a block diagram showing an automatic control system for semiconductor packaging according to an exemplary embodiment.

[0090] like Figure 2 As shown, the system 200 includes:

[0091] The welding state image acquisition module 201 is used to acquire the welding state image of the device component collected by the camera;

[0092] A multi-scale feature extraction module 202 is used to extract multi-scale features of the welding area of ​​the device assembly welding state image to obtain a welding area boundary feature vector and a welding area structure semantic feature map;

[0093] An attention enhancement module 203 is used to enhance the attention of the welding region structure semantic feature map to obtain an enhanced welding region structure semantic feature map;

[0094] A feature map batch normalization module 204 is used to use the welding region boundary feature vector as an auxiliary modal condition vector to perform feature map batch normalization on the enhanced welding region structure semantic feature map based on multimodal auxiliary information to obtain a conditionally constrained normalized welding region structure semantic feature map;

[0095] The early warning prompt generating module 205 is used to determine whether to generate an unqualified early warning prompt based on the conditional constraint normalized welding area structural semantic feature map.

[0096] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present application. The terminal device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0097] like Figure 3 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0098] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0099] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0100] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0101] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0102] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0103] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The modules involved in the embodiments described in this application can be implemented by software or hardware. The name of the module does not limit the module itself in some cases. For example, the test parameter acquisition module can also be described as a "module for acquiring device test parameters corresponding to the target device".

[0106] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0107] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] Figure 4 FIG. 1 is an application scenario diagram of an automatic control method for semiconductor packaging according to an exemplary embodiment. Figure 4 As shown, in this application scenario, first, a device component welding state image captured by a camera is obtained (for example, Figure 4 Then, the obtained device component welding state image is input to a server (for example, Figure 4 In the S) shown in , the server is capable of processing the device component welding status image based on an automated control algorithm for semiconductor packaging to determine whether to generate the unqualified early warning prompt.

[0109] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0110] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or to be performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0111] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely exemplary forms of implementation. With respect to the apparatus in the above-mentioned embodiments, the specific manner in which each module performs the operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

Claims

1. An automated control method for semiconductor packaging, comprising: A plurality of first soldering sheets are placed on the top surface of the heat sink, a DBC board is placed on the top of a single first soldering sheet, a plurality of second soldering sheets are placed on the top surface of the single DBC board, and a chip is placed on the top of a single second soldering sheet, thereby obtaining a device assembly; the device assembly is placed in a reflow oven, and the temperature of the preheating chamber of the reflow oven is raised from room temperature to 120°C, and then the temperature is raised from 120°C to 275°C within a set heating time to preheat the device assembly; after a first heat preservation time, the device assembly in the preheating chamber of the reflow oven is transferred to the reflow oven. The device assembly is welded in a welding chamber of a welding furnace to obtain a welded device assembly, wherein the temperature in the welding chamber of the reflow oven is 290°C±10°C; after a second holding time, the welded device assembly in the welding chamber of the reflow oven is transferred to a cooling chamber of the reflow oven, and taken out after a set cooling time to obtain a formed device assembly; the formed device assembly is welded and tested for quality to generate an unqualified early warning prompt, wherein the welding quality of the formed device assembly is welded and tested to generate an unqualified early warning prompt, including: Acquire a device component welding state image captured by a camera; Performing welding region multi-scale feature extraction on the welding state image of the device component to obtain a welding region boundary feature vector and a welding region structure semantic feature map; Performing attention enhancement on the welding region structure semantic feature map to obtain an enhanced welding region structure semantic feature map; Taking the welding area boundary feature vector as the auxiliary modal condition vector, performing batch normalization of the feature map of the enhanced welding area structure semantic feature map based on multimodal auxiliary information to obtain a conditionally constrained normalized welding area structure semantic feature map; Based on the conditionally constrained normalized welding area structural semantic feature map, determining whether to generate the unqualified early warning prompt; Among them, performing multi-scale feature extraction of the welding area on the welding state image of the device component to obtain a welding area boundary feature vector and a welding area structure semantic feature map includes: Performing welding area detection on the welding state image of the device assembly to obtain an image of interest in the welding area; Inputting the welding area image of interest into the boundary feature extraction branch based on the convolution layer to obtain the welding area boundary feature vector; The welding area image of interest is input into a welding structure feature extractor based on a hole convolutional neural network model to obtain a structural semantic feature map of the welding area.

2. The automated control method for semiconductor packaging according to claim 1, characterized in that: Performing welding area detection on the welding state image of the device assembly to obtain an image of interest in the welding area includes: The device assembly welding state image is input into a welding area detector based on a Yolo model to obtain an image of interest in the welding area.

3. The automated control method for semiconductor packaging according to claim 2, characterized in that: Performing attention enhancement on the welding region structure semantic feature map to obtain an enhanced welding region structure semantic feature map, including: The welding area structural semantic feature map is input into a feature attention enhancement network under a compression-suppression architecture to obtain the enhanced welding area structural semantic feature map.

4. The automated control method for semiconductor packaging according to claim 3, characterized in that: Inputting the welding area structure semantic feature map into the feature attention enhancement network under the compression-suppression architecture to obtain the enhanced welding area structure semantic feature map, including: Calculating the global mean of each feature matrix of the welding area structure semantic feature map along the channel dimension to obtain a welding area structure semantic compression information representation vector; Performing multi-scale feature extraction on the welding region structure semantic compression information representation vector to obtain a welding region structure semantic compression information multi-scale associated feature vector; Using a Sigmoid function to normalize the multi-scale associated feature vector of the welding area structure semantic compression information to obtain a welding area structure semantic feature weight vector; Based on the welding region structural semantic feature weight vector, feature amplification and suppression operations are performed on the welding region structural semantic feature map to obtain the enhanced welding region structural semantic feature map.

5. The automated control method for semiconductor packaging according to claim 4, characterized in that: Performing multi-scale feature extraction on the welding region structure semantic compression information representation vector to obtain a welding region structure semantic compression information multi-scale associated feature vector, including: Performing one-dimensional convolution coding on the welding region structure semantic compression information representation vector to obtain a feature vector representing the association between the welding region structure semantic compression information; Cascading the welding region structure semantic compression information representation vector and the welding region structure semantic compression information association representation feature vector to obtain a welding region structure semantic compression information multi-scale representation vector; The multi-scale representation vector of the welding area structure semantic compression information is input into a compression information feature extraction module based on a multi-layer perceptron model to obtain the multi-scale associated feature vector of the welding area structure semantic compression information, wherein the multi-layer perceptron model includes two fully connected layers and a SiLU activation function.

6. The automated control method for semiconductor packaging according to claim 5, characterized in that: Based on the welding region structure semantic feature weight vector, feature amplification and suppression operations are performed on the welding region structure semantic feature map to obtain the enhanced welding region structure semantic feature map, including: The enhanced welding region structural semantic feature map is obtained by calculating and multiplying the corresponding feature values ​​of the welding region structural semantic feature weight vector and the feature matrices of the welding region structural semantic feature map along the channel dimension.

7. The automated control method for semiconductor packaging according to claim 6, characterized in that: The welding region boundary feature vector is used as an auxiliary modal condition vector, and the enhanced welding region structure semantic feature map is batch normalized based on multimodal auxiliary information to obtain a conditionally constrained normalized welding region structure semantic feature map, including: Based on the mean and variance of each feature matrix of the enhanced welding region structure semantic feature map along the channel dimension, the enhanced welding region structure semantic feature map is batch normalized to obtain a normalized welding region structure semantic feature map; Expanding the normalized welding area structural semantic feature map into a one-dimensional vector and then performing feature concatenation with the auxiliary modal condition vector to obtain a multi-modal joint feature vector of the welding area; Passing the multimodal joint feature vector of the welding area through a multilayer perceptron to obtain a scaling factor and a translation factor, wherein the multilayer perceptron includes a first fully connected layer and a second fully connected layer; Based on the scaling factor and the translation factor, the enhanced welding region structural semantic feature map is batch normalized to obtain the conditionally constrained normalized welding region structural semantic feature map.

8. The automated control method for semiconductor packaging according to claim 7, characterized in that: Based on the conditionally constrained normalized welding area structure semantic feature map, determining whether to generate the unqualified early warning prompt includes: Inputting the conditionally constrained normalized welding region structure semantic feature map into a classifier-based welding detector to obtain a detection result, wherein the detection result is used to indicate whether the welding quality is qualified; In response to the detection result that the welding quality is unqualified, the unqualified early warning prompt is generated.

9. An automated control system for semiconductor packaging, used to execute the automated control method for semiconductor packaging according to claim 1, characterized in that: include: A welding state image acquisition module is used to acquire a welding state image of a device component collected by a camera; A multi-scale feature extraction module, used for performing multi-scale feature extraction of the welding area of ​​the device assembly welding state image to obtain a welding area boundary feature vector and a welding area structure semantic feature map; An attention enhancement module, used for enhancing the attention of the welding area structure semantic feature map to obtain an enhanced welding area structure semantic feature map; A feature map batch normalization module is used to use the welding area boundary feature vector as an auxiliary modal condition vector to perform feature map batch normalization on the enhanced welding area structure semantic feature map based on multimodal auxiliary information to obtain a conditionally constrained normalized welding area structure semantic feature map; The early warning prompt generation module is used to determine whether to generate an unqualified early warning prompt based on the conditional constraint normalized welding area structure semantic feature map.

Images