Flip chip ultrasonic signal denoising method and system based on adaptive DUN

Abstract

The present application relates to the technical field of signal processing, and discloses a flip chip ultrasonic signal denoising method and system based on adaptive DUN, which comprises the following steps: obtaining the ultrasonic signal reflected by the flip chip to be measured, constructing an overcomplete dictionary, sparse representation coefficients and a target function, regarding each iteration optimization of the target function as a stage to construct an adaptive deep unfolding network model with multiple stages; using the model to solve the optimal solution of the sparse representation coefficients stage by stage, wherein the model comprises: an adaptive gradient descent module for independently optimizing each element in the sparse representation coefficients, an adaptive threshold mechanism for enhancing the correlation between the channel and the input content, and a time sparsity enhancement convolution for extracting sparse features and time sequence correlation features; finally, using the trained model to obtain the optimal solution of the sparse representation coefficients, and reconstructing the ultrasonic signal by combining the overcomplete dictionary. The present application can adaptively remove different levels of noise in the ultrasonic signal, ensure robustness, and improve the upper limit of the denoising effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of signal processing, in particular to a flip-chip ultrasonic signal denoising method and system based on adaptive DUN. BACKGROUND

[0002] Flip-chip technology is a rapidly developing and widely used microelectronic packaging technology. Since the flip-chip directly bonds the solder joints on the chip and the corresponding solder joints on the packaging substrate at the same time, and omits the metal leads, the bonding speed is fast, the electrical performance is good, the reliability and yield are high. However, due to the mismatch of the thermal expansion coefficients between the chip and the substrate, as well as the packaging stress and external vibration, etc., cracks, voids and other micro-defects are easily induced in the chip solder joints, thereby reducing the reliability of the chip, and even causing the overall failure of the chip. In order to ensure the performance of the flip-chip in practical application, it is of great significance to detect the potential defects of the chip solder joints.

[0003] Ultrasonic testing is a non-destructive testing method for detecting the internal defects of materials using high-frequency ultrasonic waves, which has been widely used in flip-chip packaging quality testing. However, the solder joints of advanced packaging are becoming smaller and smaller, and the reflected ultrasonic signals of the internal defects of the solder joints are weak, accompanied by the influence of the noise of the detection system and the noise of the material grain, the reflected ultrasonic signals are complex, which affects the accuracy of the defect recognition of the solder joints. Therefore, it is necessary to denoise the ultrasonic signals reflected by the chip.

[0004] Flip-chip packaging is a typical multi-layer composite structure, and the number of internal defects is limited in space. Only a limited number of reflection interfaces and defects will produce echo signals, so that the collected ultrasonic signals have the characteristics of sparsity in physical nature. Therefore, sparse representation method can be used for denoising. Deep unfolding network (DUN) can combine the interpretability of iterative algorithms based on sparse models with the excellent performance of deep learning, and has been applied to inverse problems in multiple fields such as denoising, deblurring and super-resolution reconstruction. However, when using deep unfolding network to denoise the ultrasonic signals of the chip, it is difficult to effectively reconstruct the ultrasonic signals under different noise levels, and the denoising effect is limited. SUMMARY

[0005] Therefore, the technical problem to be solved by the present application is to overcome the shortcomings in the prior art, and to provide a flip-chip ultrasonic signal denoising method and system based on adaptive DUN, which can adaptively remove different levels of noise in the ultrasonic signal, ensure robustness, and improve the upper limit of the denoising effect.

[0006] To solve the above technical problems, the present application provides a flip-chip ultrasonic signal denoising method based on adaptive DUN, comprising:

[0007] An ultrasonic signal reflected by the flip chip under test is acquired, a fitting parameter is obtained by fitting the ultrasonic signal, and an over-complete dictionary is constructed using the fitting parameter;

[0008] A sparse representation model is constructed according to the inherent sparse characteristics of the ultrasonic signal and the over-complete dictionary, and an objective function of the sparse representation model is constructed, the objective function being used to solve an optimal solution of sparse representation coefficients of the sparse representation model;

[0009] Each iterative optimization of the objective function is regarded as a stage, an adaptive deep unfolding network model including K stages of forward propagation in series is constructed, K being a total number of iterations, the over-complete dictionary, the ultrasonic signal and the sparse representation coefficients are input into the adaptive deep unfolding network model, and the adaptive deep unfolding network model is used to solve the optimal solution of the sparse representation coefficients stage by stage, each stage including an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module being used to independently optimize each element in the sparse representation coefficients, and the adaptive soft threshold module including an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism being used to enhance the association between a channel and input content, and the temporal sparsity enhancement convolution being used to extract sparse features and time sequence correlation features;

[0010] The adaptive deep unfolding network model is trained, the optimal solution of the sparse representation coefficients is obtained using the trained adaptive deep unfolding network model, the ultrasonic signal is reconstructed in combination with the over-complete dictionary, and a denoised ultrasonic signal is obtained.

[0011] Further, the optimal solution of the sparse representation coefficients is solved stage by stage using the adaptive deep unfolding network model, and specifically:

[0012] A working mechanism of the kth stage of the adaptive deep unfolding network model is as follows: the sparse representation coefficients output by the k-1th stage, the ultrasonic signal and the over-complete dictionary are input into the kth stage, and an intermediate feature of the kth stage is output by the adaptive gradient descent module of the kth stage; the adaptive soft threshold module includes two nonlinear transformations, and the temporal sparsity enhancement convolution is embedded in the two nonlinear transformations, the two nonlinear transformations of the kth stage being denoted as and respectively; the intermediate feature of the kth stage is input into the adaptive soft threshold module of the kth stage, and the intermediate feature of the kth stage is sequentially input into , the adaptive threshold mechanism and , and the sparse representation coefficients of the kth stage are output;

[0013] The adaptive deep unfolding network model has K stages in total, and the sparse representation coefficients output after all the K stages are output are regarded as the optimal solution of the sparse representation coefficients.

[0014] Further, the calculation method of the intermediate feature of the kth stage is:

[0015] ,

[0016] wherein, is the intermediate feature of the kth stage, is the sparse representation coefficient obtained in the k-1th stage, is the gradient step of the sparse representation coefficient of the kth stage, is an over-complete dictionary, and T is a transpose operation, is an ultrasonic signal.

[0017] Further, the gradient step of the sparse representation coefficient of the kth stage is specifically:

[0018] ;

[0019] wherein, is the gradient step of the ith element in the sparse representation coefficient of the kth stage, and n is the number of elements in the sparse representation coefficient;

[0020] The calculation method of is:

[0021] ,

[0022] wherein, is a basic step, is a smoothing factor, is the gradient of the ith element in the sparse representation coefficient of the kth stage.

[0023] The calculation method of is:

[0024] ,

[0025] wherein, is the ith element in the sparse representation coefficient obtained in the k-1th stage.

[0026] Further, the intermediate feature of the kth stage is sequentially subjected to , an adaptive threshold mechanism and , and the sparse representation coefficient of the kth stage is output, and the calculation method is specifically:

[0027] Let the intermediate feature of the kth stage be , After being subjected to , the feature matrix obtained is denoted as ;

[0028] Let The feature matrix obtained after the adaptive thresholding mechanism is denoted as... , The calculation method is as follows:

[0029] ,

[0030] Where soft() is the soft thresholding operation. The feature matrix of all channels is input for the soft thresholding operation in the k-th stage. , Let C be the feature matrix of the c-th channel of the input soft thresholding operation in the k-th stage, where C is the number of channels; This represents the feature matrix of all channels after the k-th stage soft thresholding operation. It is an L1 norm. It is an L2 norm; Let k be the soft threshold vector. , The adaptive threshold for the c-th channel in the k-th stage;

[0031] Will go through The resulting feature matrix is ​​denoted as , The sparse representation coefficients of the k-th stage are obtained by reducing the C-channel to a single channel through convolution operations.

[0032] Furthermore, The calculation method is as follows:

[0033] ,

[0034] in, This represents the threshold learned in the c-th channel during the k-th stage. This represents the adaptive threshold calculated based on the input content for the c-th channel in the k-th stage. For element-wise dot product operation;

[0035] The calculation method is as follows:

[0036] ,

[0037] in, In order to be in Location The element value at the specified location, where n is the length and width of the channel.

[0038] Furthermore, the structure of each of the aforementioned nonlinear transformations comprises four layers: the first layer is an adaptive spatial parallel convolution, the second layer is a ReLU function, the third layer is a temporal sparsity enhancement convolution, and the fourth layer is an adaptive spatial parallel convolution.

[0039] The time sparsity enhanced convolutional structure comprises four convolutions: the first convolution is a standard convolution, the second convolution is composed of three series of dilated causal convolutions, the third convolution and the fourth convolution are dilated convolutions, and the dilated rates of the third convolution and the fourth convolution are different.

[0040] Further, when training the adaptive deep unfolding network model, the total loss function during training is constructed as:

[0041] ,

[0042] wherein, is the total loss function, is the difference loss function, is the symmetry constraint loss function.

[0043] Further, the calculation method of the difference loss function is:

[0044] ,

[0045] wherein, represents the data amount of each training, m is the length of the ultrasonic signal, is an over-complete dictionary, is the sparse coefficient of the i-th data output in the K-th stage, is the i-th data in the ultrasonic signal, is an L2 norm;

[0046] The calculation method of the symmetry constraint loss function is:

[0047] ,

[0048] wherein, n is the length of the sparse representation coefficient, K is the iteration number, and are two nonlinear transformation operations in the adaptive soft threshold module of the k-th stage, is the result of the i-th data in the k-th stage obtained through the adaptive gradient descent module.

[0049] The application also provides a flip-chip ultrasonic signal denoising system based on an adaptive DUN, comprising:

[0050] A signal acquisition module is configured to acquire ultrasonic signals reflected by a flip-chip under test.

[0051] An over-complete dictionary construction module is configured to fit the ultrasonic signals to obtain fitting parameters and construct an over-complete dictionary using the fitting parameters.

[0052] a sparse representation model construction module, configured to construct a sparse representation model according to an inherent sparse feature of the ultrasonic signal and an overcomplete dictionary;

[0053] a target function optimization module, configured to construct a target function of the sparse representation model and iteratively optimize the target function, the target function being used to solve an optimal solution of sparse representation coefficients of the sparse representation model;

[0054] an adaptive deep unfolding network model construction module, configured to take each iterative optimization of the target function as a stage, and construct an adaptive deep unfolding network model including K stages of forward propagation in series, K being a total number of iterations; a working mechanism of the adaptive deep unfolding network model is that the overcomplete dictionary, the ultrasonic signal and the sparse representation coefficients are input into the adaptive deep unfolding network model, and the adaptive deep unfolding network model is used to solve the optimal solution of the sparse representation coefficients stage by stage; each stage includes an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module is configured to independently optimize each element in the sparse representation coefficients, and the adaptive soft threshold module includes an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between a channel and input content, and the temporal sparsity enhancement convolution extracts sparse features and time sequence correlation features;

[0055] an ultrasonic signal denoising module, configured to train the adaptive deep unfolding network model, obtain the optimal solution of the sparse representation coefficients by using the trained adaptive deep unfolding network model, reconstruct the ultrasonic signal in combination with the overcomplete dictionary, and obtain a denoised ultrasonic signal.

[0056] The above technical solution of the present application has the following beneficial effects compared with the prior art:

[0057] The adaptive gradient descent module is configured to independently optimize each element in the sparse representation coefficients, so that more accurate sparse representation coefficients are obtained, and the original ultrasonic signal is fitted by using the more accurate sparse representation coefficients; the adaptive threshold mechanism enhances the association between a channel and input content, and the temporal sparsity enhancement convolution extracts sparse features and time sequence correlation features, and noise atoms are deleted to achieve a denoising effect, so that different levels of noise in the ultrasonic signal can be removed while the ability to reconstruct weak echoes is improved, strong robustness can be maintained under a high noise level, and the upper limit of the denoising effect is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments of the present application and in combination with the drawings, in which:

[0059] Figure 1A flow chart of the method in the preferred embodiment of the present application.

[0060] Figure 2 A framework diagram of the adaptive deep unfolding network model in the preferred embodiment of the present application.

[0061] Figure 3 A framework diagram of the kth stage of the adaptive deep unfolding network model in the preferred embodiment of the present application.

[0062] Figure 4 A comparison diagram of the signal-to-noise ratio of the ultrasonic signal denoised by the method of the present application and other methods in the simulation experiment.

[0063] Figure 5 A comparison diagram of the root mean square error of the ultrasonic signal denoised by the method of the present application and other methods in the simulation experiment.

[0064] Figure 6 A waveform diagram of the actual ultrasonic signal denoised by the method of the present application and other methods. DETAILED DESCRIPTION

[0065] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not limiting to the present application.

[0066] Referring to Figure 1 The present application discloses a flip-chip ultrasonic signal denoising method based on adaptive DUN, comprising the following steps:

[0067] S1: Obtain the ultrasonic signal reflected by the defect of the flip-chip to be tested, and the expression of the ultrasonic signal is:

[0068] ,

[0069] Among them, represents the amplitude of the actually collected ultrasonic signal at time t, represents the reflection coefficient of the ith interface, represents the amplitude of the ultrasonic signal of the ith interface at time t, represents the noise amplitude from the flip-chip to be tested and the system at time t, and M represents the total number of echoes.

[0070] S2: Use the least square method to fit the ultrasonic signal to determine the Gabor atom fitting parameters, and use the fitting parameters to construct an over-complete dictionary (Gabor dictionary).

[0071] S3: constructing a sparse representation model according to inherent sparse characteristics of the ultrasonic signal and an overcomplete dictionary, and constructing an objective function containing an L1 norm regularization term for solving an optimal solution of sparse representation coefficients of the sparse representation model.

[0072] S3-1: sparse reconstruction of the ultrasonic signal to obtain the sparse representation model as follows:

[0073] ,

[0074] wherein, is an original ultrasonic signal containing noise actually obtained, , is the i-th data in the ultrasonic signal; is an overcomplete dictionary, , represents is a m-dimensional space, m represents a length of the ultrasonic signal, and n represents a number of atoms in the overcomplete dictionary, , is a sparse representation coefficient, , is noise.

[0075] S3-2: constructing an objective function containing an L1 norm regularization term to solve optimal sparse representation coefficients, specifically as follows:

[0076] ,

[0077] wherein, is an optimal sparse representation coefficient, is a data fitting term, is a sparse regularization term, is a regularization parameter for adjusting a sparse degree, is an L1 norm, is an L2 norm.

[0078] S4: using an Iterative Shrinkage-Thresholding Algorithm (ISTA) to iteratively optimize the objective function, and gradually approaching an optimal solution by updating each iteration.

[0079] Two steps of gradient descent operation and soft threshold operation are performed in each iteration to update, specifically as follows:

[0080] ,

[0081] ;

[0082] where the first equation performs gradient descent operation and the second equation performs soft threshold operation, denotes the result of gradient descent update in the k-th iteration, denotes the sparse solution of sparse representation coefficients updated by soft threshold operation in the k-th iteration, denotes the gradient descent step size, denotes the transpose operation, denotes the soft threshold.

[0083] The second equation is equivalently expressed as:

[0084] ,

[0085] where, denotes the soft threshold operation.

[0086] The entire optimization process is an iterative update process of two steps of gradient descent operation and soft threshold operation. The gradient descent is used for fitting the original signal, and the soft threshold operation removes noise by deleting noise atoms. The above iterative solving process is expanded into a deep network architecture, and each iteration step corresponds to a stage of the network.

[0087] S5: Depth expansion is performed on the iterative shrinkage threshold algorithm, each iteration optimization of the target function is taken as a stage, an adaptive deep expansion network model is used to solve the optimal solution of the sparse representation coefficients stage by stage, the adaptive deep expansion network model is trained using back propagation, and a total loss function during training is constructed in combination with difference loss and symmetry constraint loss.

[0088] An adaptive deep expansion network model including K stages in series through forward propagation is constructed as shown in Figure 2 , and K is the total number of iterations. Each stage in the adaptive deep expansion network model corresponds to each iteration step in the proximal gradient descent algorithm. After K stages, K iterations of optimization are performed, Figure 3 is the k-th stage of the adaptive deep expansion network model. Figure 3 In the above formula, “Conv 1→C” denotes that the 3×3 convolution operation is reduced from C channels to a single channel, “Conv C→1” denotes that the 3×3 convolution operation is increased from a single channel to C channels, “ ” and “ ” denote the dimension of the current feature.

[0089] The working mechanism of the adaptive deep unfolding network model is that the overcomplete dictionary, the ultrasonic signal and the sparse representation coefficient are input into the adaptive deep unfolding network model, and the optimal solution of the sparse representation coefficient is solved stage by stage using the adaptive deep unfolding network model. Each stage includes an adaptive gradient descent module (AGDM) and an adaptive soft threshold module (ASTM), the adaptive gradient descent module optimizes each element in the sparse representation coefficient independently, and the adaptive soft threshold module includes an adaptive threshold mechanism and a time sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between the channel and the input content, and the time sparsity enhancement convolution extracts sparse features and time sequence correlation features.

[0090] The optimal solution of the sparse representation coefficient is solved stage by stage using the adaptive deep unfolding network model, and the specific process is as follows:

[0091] S5-1: The sparse representation coefficient (denoted as ) output by the last stage (i.e. the k-1 stage) is taken as the input of the kth stage together with the ultrasonic signal and the overcomplete dictionary, and the intermediate feature of the kth stage is output by the adaptive gradient descent module of the kth stage, denoted as . The adaptive gradient descent module expands the original scalar step into a vector form based on the gradient information to update each element in the sparse representation coefficient independently.

[0092] The calculation method of the intermediate feature of the kth stage is as follows:

[0093] ,

[0094] wherein, is the intermediate feature of the kth stage, is the sparse representation coefficient obtained in the k-1 stage, is the gradient step of the sparse representation coefficient of the kth stage, is the overcomplete dictionary, is the transpose operation, is the ultrasonic signal.

[0095] The gradient step of the sparse representation coefficient of the kth stage is as follows:

[0096] ;

[0097] wherein, is the gradient step of the i th element in the sparse representation coefficient of the kth stage, and n is the number of elements in the sparse representation coefficient;

[0098] The calculation method of is as follows:

[0099] ,

[0100] wherein, is the base step size, which is set to , is the smoothing factor to prevent division by zero, which is set to , is the gradient of the i-th element in the sparse representation coefficient of the k-th stage.

[0101] The calculation method of is:

[0102] ,

[0103] wherein, is the i-th element in the sparse representation coefficient obtained in the k-1-th stage, i.e., the sparse coefficient of the i-th data obtained in the k-1-th stage.

[0104] It can be seen that the adaptive gradient descent module expands the original scalar step size into a vector form, so that each element in the sparse representation coefficient is optimized independently, thereby enhancing the parameter adjustment capability of the network during gradient descent.

[0105] S5-2: The adaptive soft threshold module includes two nonlinear transformations, and the time sparsity enhancement convolution is embedded in the two nonlinear transformations.

[0106] The two nonlinear transformations have the same structure, and the structure of each nonlinear transformation includes four layers: the first layer is an adaptive spatial parallel convolution (ASPConv), the second layer is a ReLU function, the third layer is a time sparsity enhancement convolution (TSEConv), and the fourth layer is an adaptive spatial parallel convolution (ASPConv).

[0107] The adaptive spatial parallel convolution uses the “Adaptively Spatial Parallel Convolution Module” in Section 3.1 of Part 3 of the paper “QI G Q, ZHANG Y C, WANG K P, et al. Small object detection method based on adaptive spatial parallel convolution and fast multi-scale fusion[J]. Remote Sensing, 2022, 14(2): 420.”

[0108] The structure of the time sparsity enhanced convolution includes four convolutions: the first convolution is a standard convolution of 3x3, the second convolution is composed of three dilated causal convolutions in series; the third convolution and the fourth convolution are dilated convolutions, the dilation rates of the third convolution and the fourth convolution are different, in the embodiment, the third convolution is a dilated convolution with a dilation rate of 2, and the fourth convolution is a dilated convolution with a dilation rate of 4.

[0109] Let the two nonlinear transformations of the kth stage be respectively and ; then input the intermediate feature of the kth stage into the adaptive soft threshold value module of the kth stage, and the intermediate feature of the kth stage sequentially passes through the nonlinear transformation , the adaptive threshold mechanism and the nonlinear transformation , and outputs the sparse representation coefficient of the kth stage, denoted as .

[0110] S5-2-1: Let the intermediate feature of the kth stage be , After the feature matrix obtained after is denoted as , that is , is a nonlinear transformation operation.

[0111] S5-2-2: Let the feature matrix obtained after passes through the adaptive threshold mechanism be denoted as , The calculation method of

[0112] ,

[0113] Wherein, soft( ) is a soft threshold operation, is the feature matrix of all channels inputting the soft threshold operation of the kth stage, , is the feature matrix of the cth channel inputting the soft threshold operation of the kth stage, and C is the number of channels; is the feature matrix of all channels after the soft threshold operation of the kth stage, is the L1 norm, is the L2 norm; is the soft threshold vector of the kth stage, , is the adaptive threshold of the cth channel of the kth stage.

[0114] The calculation method of

[0115] ,

[0116] wherein, represents the threshold value learned by the kth stage cth channel (i.e., the trainable channel-level threshold value), represents the content adaptive threshold value calculated by the kth stage cth channel according to the input content (i.e., the channel-aware threshold value), is an element-wise multiplication operation;

[0117] According to the feature distribution of each channel, the corresponding , The calculation method of is as follows:

[0118] ,

[0119] wherein, is the element value at the position of , n is the length and width of the channel.

[0120] In the first forward propagation process in this embodiment, the trainable channel-level threshold value of each stage is initialized as 1, that is, the initial threshold value is completely calculated by the features in the channel. Let the learned threshold value of the kth stage be , As the training progresses, after each round of training, the is updated stage by stage through the back propagation of the loss function.

[0121] S5-2-3: The feature matrix obtained after is denoted as , After the 3x3 convolution operation, the kth stage sparse representation coefficient is obtained by reducing the C channels to a single channel. S5-1 and S5-2 are the working mechanism of the kth stage of the adaptive deep unfolding network model.

[0122] S5-3: The adaptive deep unfolding network model has K stages, and the method in S5-1 and S5-2 is repeatedly used. The sparse representation coefficient output after all K stages (denoted as ) is used as the optimal solution of the sparse representation coefficient.

[0123] In the training process of the adaptive deep unfolding network model, the adaptive deep unfolding network model is trained using back propagation, and the total loss function during training is constructed by combining the difference loss and the symmetry constraint loss as follows:

[0124]

[0125] ​​​​,

[0126] wherein, is a total loss function, is a difference loss function, is a symmetry constraint loss function.

[0127] The difference loss function trains the network parameters of the adaptive deep unfolding network model by measuring the difference between the reconstructed flip-chip ultrasonic signal and the original noisy ultrasonic signal The calculation method of the difference loss function is:

[0128] ,

[0129] wherein, represents the amount of data for each training, m is the length of the ultrasonic signal, is an over-complete dictionary, is the sparse coefficient of the i-th data output by the K-th stage, that is, the i-th element in the sparse representation coefficient obtained by the K-th stage, is the i-th data in the ultrasonic signal, is an L2 norm.

[0130] The symmetry constraint loss function is to ensure that the network follows the expectation of adaptive gradient descent and adaptive soft threshold operation, and the calculation method of the symmetry constraint loss function is:

[0131] ,

[0132] wherein, n is the length of the sparse representation coefficient, K is the iteration number, that is, the total stage number, and are two nonlinear transformation operations in the adaptive soft threshold module of the k-th stage, is the result obtained by the i-th data in the k-th stage through the adaptive gradient descent module.

[0133] S6: training the adaptive deep unfolding network model, using the trained adaptive deep unfolding network model to obtain the optimal solution of the sparse representation coefficient, reconstructing the ultrasonic signal by combining the over-complete dictionary, and obtaining the denoised ultrasonic signal:

[0134] ,

[0135] wherein, is the reconstructed ultrasonic signal, that is, the denoised ultrasonic signal.

[0136] The application also discloses a flip-chip ultrasonic signal denoising system based on adaptive DUN, comprising:

[0137] A signal acquisition module is configured to acquire an ultrasonic signal reflected by a defect of a flip chip to be tested;

[0138] A supercomplete dictionary construction module is configured to fit the ultrasonic signal to obtain fitting parameters, and construct a supercomplete dictionary using the fitting parameters;

[0139] A sparse representation model construction module is configured to construct a sparse representation model according to an inherent sparse feature of the ultrasonic signal and the supercomplete dictionary;

[0140] A target function optimization module is configured to construct a target function of the sparse representation model and iteratively optimize the target function, wherein the target function is used to solve an optimal solution of sparse representation coefficients of the sparse representation model;

[0141] An adaptive deep unfolding network model construction module is configured to regard each iterative optimization of the target function as a stage, and construct an adaptive deep unfolding network model including K stages of forward propagation in series, wherein K is a total number of iterations; a working mechanism of the adaptive deep unfolding network model is that the supercomplete dictionary, the ultrasonic signal and the sparse representation coefficients are input into the adaptive deep unfolding network model, and the adaptive deep unfolding network model is used to solve the optimal solution of the sparse representation coefficients stage by stage; each stage includes an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module is used to independently optimize each element in the sparse representation coefficients, and the adaptive soft threshold module includes an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism enhances an association between a channel and input content, and the temporal sparsity enhancement convolution extracts sparse features and time sequence correlation features;

[0142] An ultrasonic signal denoising module is configured to train the adaptive deep unfolding network model, use the trained adaptive deep unfolding network model to obtain the optimal solution of the sparse representation coefficients, reconstruct the ultrasonic signal in combination with the supercomplete dictionary, and obtain a denoised ultrasonic signal.

[0143] The application further discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the flip chip ultrasonic signal denoising method based on the adaptive DUN.

[0144] The application further discloses a device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the flip chip ultrasonic signal denoising method based on the adaptive DUN when executing the computer program.

[0145] Compared with the prior art, the application has the following advantages:

[0146] 1、The present application adjusts the gradient descent step size according to the sparsity of the ultrasonic signal, independently optimizes each element in the sparse representation coefficient, so as to obtain more accurate sparse representation coefficient, and uses the more accurate sparse representation coefficient to fit the ultrasonic signal.

[0147] 2、By strengthening the correlation between the adaptive threshold and the input content, the model can adaptively remove different levels of noise in the ultrasonic signal.

[0148] 3. According to the inherent sparsity and time correlation of the ultrasonic signal, a time sparsity enhanced convolution is proposed, which can effectively avoid ghosting problem, more accurately depict the natural attenuation characteristics of ultrasonic echo, improve the ability to reconstruct weak echo, and maintain strong robustness under high noise level, further improve the upper limit of denoising effect.

[0149] In order to further prove the advantages of the present application, the simulation signal and the actual signal of the flip-chip solder ball are sparse reconstructed by using the method of the present application in this embodiment. ISTA-Net+, SODAS-Net and MAUN based on deep unfolding method (for details, see the paper "Hu Q, Ma J, Gao Y, et al. MAUN: Memory-Augmented Deep Unfolding Network for Hyperspectral Image Reconstruction [J]. IEEE / CAA Journal of Automatica Sinica, 2024, 11(5): 1139-1150.") are selected, OMP (for details, see the paper "Tropp J A, Gilbert A C. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit [J]. IEEE Transactions on Information Theory, 2007, 53(12): 4655-4666. DOI: 10.1109 / TIT.2007.909108.") and MMP (for details, see the paper "Tropp J A, Gilbert A C. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit [J]. IEEE Transactions on Information Theory, 2007, 53(12): 4655-4666. DOI: 10.1109 / TIT.2007.909108.") based on sparse method, and wavelet denoising method based on traditional signal processing are used for comparison experiments.

[0150] The signal-to-noise ratio and the root mean square error are used as evaluation indexes, and the denoising results of the simulation ultrasonic signal of the flip-chip are shown in ADUN of Figure 4 、 Figure 5 , Figure 4 、 Figure 5 , wherein ADUN represents the method of the present application, and wavelet represents the wavelet denoising method. Figure 4 is a comparison diagram of the signal-to-noise ratio of the ultrasonic signal denoised by using the method of the present application and other methods respectively, Figure 5 is a comparison diagram of the root mean square error of the ultrasonic signal denoised by using the method of the present application and other methods respectively. From Figure 4 、 Figure 5It can be seen that the wavelet denoising method and SODAS-Net can achieve good denoising effect at low noise level, and SODAS-Net is slightly better than the wavelet denoising method, but with the increase of noise intensity, the reconstruction quality of both rapidly decreases, and the denoising performance of SODAS-Net and the wavelet denoising method is equivalent at high noise level. The denoising effect of OMP and MMP is basically the same, and the reconstruction quality is between the wavelet denoising method and SODAS-Net at low noise level, but the result is gradually better than SODAS-Net after the noise increases. ISTA-Net+ has relatively poor denoising effect at very low noise level compared with other comparative methods, but with the increase of noise level, the denoising performance is gradually better than the wavelet denoising method, SODAS-Net, OMP and MMP. MAUN shows better denoising ability on the whole, but its denoising performance decreases obviously at a certain stage from low noise level to high noise level, reaching the performance limit of the method itself; thereafter, with the further increase of noise level, the denoising effect maintains a certain noise reduction ability. The signal-to-noise ratio and root mean square error of the method of the application are better than those of other methods, especially at high noise level, which still maintains good denoising effect.

[0151] Then, the actual ultrasonic signal of flip-chip is denoised by using the method of the application, and the denoising result is as shown in Figure 6 Figure 6 In the figure, (a) represents the waveform diagram of the actual ultrasonic signal before denoising, Figure 6 (b) represents the waveform diagram after denoising by using the wavelet denoising method, Figure 6 (c) represents the waveform diagram after denoising by using OMP, Figure 6 (d) represents the waveform diagram after denoising by using MMP, Figure 6 (e) represents the waveform diagram after denoising by using ISTA-Net+, Figure 6 (f) represents the waveform diagram after denoising by using SODAS-Net, Figure 6 (g) represents the waveform diagram after denoising by using MAUN, Figure 6 (h) represents the waveform diagram after denoising by using the method of the application. From Figure 6 ​It can be seen that the wavelet denoising method still has some noise residues in the early stage of the characteristic waveform, which indicates that the method has insufficient effect on suppressing background noise; OMP and MMP have similar overall denoising effects, but there is incomplete noise removal in the tail of the signal; SODAS-Net has better denoising performance than the sparse method, but abnormal trends appear in the tail of the signal; ISTA-Net+ and MAUN also show obvious abnormalities in the tail of the signal, and there is waveform information loss at the circle mark, which will affect the judgment accuracy of the microstructure or defect position; the method of the application can not only effectively suppress most of the noise, but also reconstruct the weak echo signal, and has better overall denoising effect than other existing methods.

[0152] It can be seen from the simulation experiment that the denoising effect of the application has higher signal-to-noise ratio and lower root mean square error compared with the existing denoising methods, and the denoising effect of the application is better; at the same time, the denoising comparison of the actual ultrasonic signal also verifies that the application can effectively excavate weak echo characteristics while removing ultrasonic signal noise, and achieves superior denoising performance.

[0153] Those skilled in the art should understand that the embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can adopt the form of a computer program product implemented on one or more computer usable storage media containing computer usable program code (including but not limited to disk storage, CD-ROM, optical storage, etc.).

[0154] The application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one or more flows and / or blocks. Figure 1 The device that implements the functions specified in one or more flows and / or blocks.

[0155] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0156] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, so that the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0157] Obviously, the above embodiments are only examples for clearly illustrating, not limiting the embodiments. For those skilled in the art, on the basis of the above description, other different forms of changes or variations can also be made. Here, it is not necessary and also impossible to enumerate all the embodiments. The obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A method for denoising flip-chip ultrasonic signals based on adaptive DUN, characterized in that, include: Acquire the ultrasonic signal reflected by the flip chip under test, fit the ultrasonic signal to obtain fitting parameters, and use the fitting parameters to construct an overcomplete dictionary; A sparse representation model is constructed based on the inherent sparse characteristics of ultrasound signals and an overcomplete dictionary. An objective function for the sparse representation model is then constructed, and the objective function is used to solve for the optimal solution of the sparse representation coefficients of the sparse representation model. The objective function is iteratively optimized, with each iteration as a stage. An adaptive deep unfolded network model consisting of K stages connected sequentially via forward propagation is constructed, where K is the total number of iterations. The overcomplete dictionary, ultrasound signal, and sparse representation coefficients are input into the adaptive deep unfolded network model, and the optimal solution for the sparse representation coefficients is solved stage by stage using the adaptive deep unfolded network model. Each stage includes an adaptive gradient descent module and an adaptive soft thresholding module. The adaptive gradient descent module independently optimizes each element in the sparse representation coefficients. The adaptive soft thresholding module includes an adaptive thresholding mechanism and a temporal sparsity enhancement convolution. The adaptive thresholding mechanism enhances the correlation between channels and input content, and the temporal sparsity enhancement convolution extracts sparse features and temporal correlation features. The adaptive deep unfolding network model is trained, and the optimal solution of the sparse representation coefficients is obtained using the trained adaptive deep unfolding network model. The ultrasonic signal is then reconstructed using an overcomplete dictionary to obtain the denoised ultrasonic signal. The optimal solution for the sparse representation coefficients is obtained stage by stage using the adaptive depth unfolding network model, specifically: The working mechanism of the k-th stage of the adaptive deep unfolded network model is as follows: the sparse representation coefficients, ultrasound signal, and overcomplete dictionary output from the (k-1)-th stage are used as inputs to the k-th stage, and the intermediate features of the k-th stage are output through the adaptive gradient descent module. The adaptive soft thresholding module includes two nonlinear transformations, in which the temporal sparsity enhancement convolution is embedded. The two nonlinear transformations of the k-th stage are denoted as Fk, ... (k) and H (k) The intermediate features of stage k are input into the adaptive soft thresholding module of stage k, and the intermediate features of stage k are sequentially processed by F. (k) Adaptive threshold mechanism and H (k) Output the sparse representation coefficients for the k-th stage; The adaptive deep unfolding network model has K stages, and the sparse representation coefficients output after all K stages are taken as the optimal solution for the sparse representation coefficients.

2. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 1, characterized in that: The method for calculating the intermediate features of the k-th stage is as follows: r (k) =x (k-1) -ρ (k) D T (Dx (k-1) -y), Where, r (k) x is an intermediate feature of stage k. (k-1) ρ represents the sparse representation coefficients obtained in stage k-1. (k) Let be the gradient step size of the sparse representation coefficients in the k-th stage, D be the overcomplete dictionary, T be the transpose operation, and y be the ultrasound signal.

3. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 2, characterized in that: The gradient step size of the sparse representation coefficients in the k-th stage is specifically as follows: in, Let n be the gradient step size of the i-th element in the sparse representation coefficients of the k-th stage, and n be the number of elements in the sparse representation coefficients. The calculation method is as follows: Where, ρ base This is the base step size, and ε is the smoothing factor. Let be the gradient of the i-th element in the sparse representation coefficients of the k-th stage; The calculation method is as follows: Where, x i (k-1) It is the i-th element in the sparse representation coefficients obtained in the (k-1)-th stage.

4. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 1, characterized in that: The intermediate features of the k-th stage pass through F sequentially. (k) Adaptive threshold mechanism and H (k) Output the sparse representation coefficients for the k-th stage, specifically: Let r be the intermediate feature of the k-th stage. (k) r (k) After F (k) The resulting feature matrix is ​​denoted as p. (k) ; p (k) The feature matrix obtained after the adaptive thresholding mechanism is denoted as q. (k) q (k) The calculation method is as follows: Where soft() is the soft thresholding operation, p (k) Let p be the feature matrix of all channels for the soft thresholding operation in the k-th stage. (k) ={p1 (k) ,...,p c (k) ,...,p C (k) }, p c (k) q is the feature matrix of the c-th channel input to the soft thresholding operation in the k-th stage, where C is the number of channels; (k) Let λ be the feature matrix of all channels after the k-th stage soft thresholding operation, |||1| is the L1 norm, and |||2| is the L2 norm; λ (k) Let k be the soft threshold vector. The adaptive threshold for the c-th channel in the k-th stage; q (k) After H (k) The resulting feature matrix is ​​denoted as z. (k) , z (k) The sparse representation coefficients of the k-th stage are obtained by reducing the C-channel to a single channel through convolution operations.

5. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 4, characterized in that: The calculation method is as follows: in, This represents the threshold learned in the c-th channel during the k-th stage. This represents the adaptive threshold calculated based on the input content for the c-th channel in the k-th stage; ⊙ represents the element-wise multiplication operation. The calculation method is as follows: in, For in p c (k) The element value at position (i,j), where n is the length and width of the channel.

6. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 1, characterized in that: Each of the aforementioned nonlinear transformations comprises four layers: the first layer is an adaptive spatial parallel convolution, the second layer is a ReLU function, the third layer is a temporally sparsity-enhancing convolution, and the fourth layer is an adaptive spatial parallel convolution. The structure of the temporal sparsity-enhanced convolution includes four convolutions: the first convolution is a standard convolution, the second convolution consists of three dilated causal convolutions in series, and the third and fourth convolutions are dilated convolutions with different dilation rates.

7. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to any one of claims 1-6, characterized in that: When training the adaptive deep unfolded network model, the total loss function during training is constructed as follows: L total =L discrepancy +L constraint , Among them, L total Let L be the total loss function. discrepancy Let L be the difference loss function. constraint This is the loss function for symmetric constraints.

8. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 7, characterized in that: The method for calculating the difference loss function is as follows: Where, N b The data size represents the amount of data used in each training iteration, m is the length of the ultrasound signal, and D is the overcomplete dictionary. Let y be the sparsity coefficient of the i-th data output in the K-th stage. i Let be the i-th data in the ultrasound signal, and || ||2 be the L2 norm; The method for calculating the symmetric constraint loss function is as follows: Where n is the length of the sparse representation coefficients, K is the number of iterations, and F (k) ( ) and H (k) ( ) represent two nonlinear transformation operations in the adaptive soft thresholding module of the k-th stage, r i (k) This is the result obtained by applying the adaptive gradient descent module to the i-th data in the k-th stage.

9. A flip-chip ultrasonic signal denoising system based on adaptive DUN, characterized in that, include: The signal acquisition module is used to acquire the ultrasonic signal reflected by the flip chip under test; An overcomplete dictionary construction module is used to fit ultrasound signals to obtain fitting parameters, and then the fitting parameters are used to construct an overcomplete dictionary. The sparse representation model construction module is used to construct a sparse representation model based on the inherent sparse characteristics of ultrasound signals and an overcomplete dictionary. The objective function optimization module is used to construct the objective function of the sparse representation model and iteratively optimize the objective function, wherein the objective function is used to solve for the optimal solution of the sparse representation coefficients of the sparse representation model; An adaptive deep unfolded network model construction module is used to construct an adaptive deep unfolded network model consisting of K stages, where each iteration of the objective function is treated as a stage, and forward propagation is sequentially connected. K represents the total number of iterations. The working mechanism of the adaptive deep unfolded network model is as follows: the overcomplete dictionary, ultrasound signal, and sparse representation coefficients are input into the adaptive deep unfolded network model, and the optimal solution for the sparse representation coefficients is solved stage by stage using the adaptive deep unfolded network model. Each stage includes an adaptive gradient descent module and an adaptive soft thresholding module. The adaptive gradient descent module independently optimizes each element in the sparse representation coefficients. The adaptive soft thresholding module includes an adaptive thresholding mechanism and a temporal sparsity enhancement convolution. The adaptive thresholding mechanism enhances the correlation between channels and input content, and the temporal sparsity enhancement convolution extracts sparse features and temporal correlation features. The ultrasonic signal denoising module is used to train the adaptive deep unfolding network model, obtain the optimal solution of sparse representation coefficients using the trained adaptive deep unfolding network model, and reconstruct the ultrasonic signal by combining it with an overcomplete dictionary to obtain the denoised ultrasonic signal. The optimal solution for the sparse representation coefficients is obtained stage by stage using the adaptive depth unfolding network model, specifically: The working mechanism of the k-th stage of the adaptive deep unfolded network model is as follows: the sparse representation coefficients, ultrasound signal, and overcomplete dictionary output from the (k-1)-th stage are used as inputs to the k-th stage, and the intermediate features of the k-th stage are output through the adaptive gradient descent module. The adaptive soft thresholding module includes two nonlinear transformations, in which the temporal sparsity enhancement convolution is embedded. The two nonlinear transformations of the k-th stage are denoted as F( k ) and H (k) The intermediate features of stage k are input into the adaptive soft thresholding module of stage k, and the intermediate features of stage k are sequentially processed by F( k ), adaptive threshold mechanism and H (k) Output the sparse representation coefficients for the k-th stage; The adaptive deep unfolding network model has K stages, and the sparse representation coefficients output after all K stages are taken as the optimal solution for the sparse representation coefficients.

Images