Flip chip ultrasonic signal denoising method and system based on adaptive DUN

Abstract

The invention relates to the technical field of signal processing, and discloses a flip chip ultrasonic signal denoising method and system based on self-adaptive DUN, and the method comprises the steps: obtaining an ultrasonic signal reflected by a to-be-detected flip chip, constructing an over-complete dictionary, a sparse representation coefficient and a target function, constructing an adaptive deep expansion network model with multiple stages by taking each iterative optimization of the target function as one stage; solving the optimal solution of the sparse representation coefficient stage by stage by using a model, wherein the model comprises a self-adaptive gradient descent module for independently optimizing each element in the sparse representation coefficient, a self-adaptive threshold mechanism for enhancing correlation between a channel and input content, and a time sparsity enhanced convolution for extracting sparse features and time sequence correlation features; and finally, obtaining an optimal solution of the sparse representation coefficient by using the trained model, and reconstructing the ultrasonic signal in combination with the over-complete dictionary. According to the method, different levels of noise in the ultrasonic signal can be adaptively removed, the robustness is ensured, and the upper limit of the denoising effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of signal processing, in particular to a flip-chip ultrasonic signal denoising method and system based on adaptive DUN. BACKGROUND

[0002] Flip-chip technology is a rapidly developing and widely used microelectronic packaging technology. Since the flip-chip directly bonds the solder joints on the chip and the corresponding solder joints on the packaging substrate at the same time, and omits the metal leads, the bonding speed is fast, the electrical performance is good, the reliability and yield are high. However, due to the mismatch of the thermal expansion coefficients between the chip and the substrate, as well as the packaging stress and external vibration, etc., cracks, voids and other micro-defects are easily induced in the chip solder joints, thereby reducing the reliability of the chip, and even causing the overall failure of the chip. In order to ensure the performance of the flip-chip in practical application, it is of great significance to detect the potential defects of the chip solder joints.

[0003] Ultrasonic testing is a non-destructive testing method for detecting the internal defects of materials using high-frequency ultrasonic waves, which has been widely used in flip-chip packaging quality testing. However, the solder joints of advanced packaging are becoming smaller and smaller, and the reflected ultrasonic signals of the internal defects of the solder joints are weak, accompanied by the influence of the noise of the detection system and the noise of the material grain, the reflected ultrasonic signals are complex, which affects the accuracy of the defect recognition of the solder joints. Therefore, it is necessary to denoise the ultrasonic signals reflected by the chip.

[0004] Flip-chip packaging is a typical multi-layer composite structure, and the number of internal defects is limited in space. Only a limited number of reflection interfaces and defects will produce echo signals, so that the collected ultrasonic signals have the characteristics of sparsity in physical nature. Therefore, sparse representation method can be used for denoising. Deep unfolding network (DUN) can combine the interpretability of iterative algorithms based on sparse models with the excellent performance of deep learning, and has been applied to inverse problems in multiple fields such as denoising, deblurring and super-resolution reconstruction. However, when using deep unfolding network to denoise the ultrasonic signals of the chip, it is difficult to effectively reconstruct the ultrasonic signals under different noise levels, and the denoising effect is limited. SUMMARY

[0005] Therefore, the technical problem to be solved by the present application is to overcome the shortcomings in the prior art, and to provide a flip-chip ultrasonic signal denoising method and system based on adaptive DUN, which can adaptively remove different levels of noise in the ultrasonic signal, ensure robustness, and improve the upper limit of the denoising effect.

[0006] To solve the above technical problems, the present application provides a flip-chip ultrasonic signal denoising method based on adaptive DUN, comprising: Obtain the ultrasonic signal reflected by the flip chip to be tested, fit the ultrasonic signal to obtain fitting parameters, and use the fitting parameters to construct an over-complete dictionary; A sparse representation model is constructed based on the inherent sparse characteristics of the ultrasonic signal and an overcomplete dictionary, and an objective function of the sparse representation model is constructed, wherein the objective function is used to solve the optimal solution of the sparse representation coefficients of the sparse representation model; The objective function is iteratively optimized, and each iterative optimization of the objective function is regarded as a stage. An adaptive deep expansion network model is constructed, which includes K stages of forward propagation connected in series, where K is the total number of iterations; the overcomplete dictionary, the ultrasonic signal, and the sparse representation coefficient are input into the adaptive deep expansion network model, and the adaptive deep expansion network model is used to solve the optimal solution of the sparse representation coefficient stage by stage; each stage includes an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module independently optimizes each element in the sparse representation coefficient, the adaptive soft threshold module includes an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between the channel and the input content, and the temporal sparsity enhancement convolution extracts sparse features and temporal correlation features; The adaptive deep expansion network model is trained, and the optimal solution of the sparse representation coefficient is obtained using the trained adaptive deep expansion network model. The ultrasonic signal is reconstructed in combination with an overcomplete dictionary to obtain a denoised ultrasonic signal.

[0007] Furthermore, the adaptive deep expansion network model is used to solve the optimal solution of the sparse representation coefficients stage by stage, specifically: The working mechanism of the kth stage of the adaptive deep expansion network model is as follows: the sparse representation coefficients, ultrasonic signals and over-complete dictionary outputted in the k-1th stage are used as the inputs of the kth stage, and the intermediate features of the kth stage are outputted through the adaptive gradient descent module of the kth stage; the adaptive soft threshold module includes two nonlinear transformations, and the temporal sparsity enhanced convolution is embedded in the two nonlinear transformations. The two nonlinear transformations of the kth stage are respectively denoted as and ; Input the intermediate features of the kth stage into the adaptive soft threshold module of the kth stage, and the intermediate features of the kth stage are sequentially , adaptive threshold mechanism and , output the sparse representation coefficient of the kth stage; The adaptive deep expansion network model has a total of K stages, and the sparse representation coefficients output after all K stages are used as the optimal solution of the sparse representation coefficients.

[0008] Furthermore, the calculation method of the intermediate features of the kth stage is: , wherein, is the intermediate feature of the kth stage, is the sparse representation coefficient obtained in the k-1th stage, is the gradient step size of the sparse representation coefficient of the kth stage, is an overcomplete dictionary, and T is a transpose operation, is an ultrasound signal.

[0009] Further, the gradient step size of the sparse representation coefficient of the kth stage is specifically: ; wherein, is the gradient step size of the i-th element in the sparse representation coefficient of the kth stage, and n is the number of elements in the sparse representation coefficient; The calculation method of is: , wherein, is a basic step size, is a smoothing factor, is the gradient of the i-th element in the sparse representation coefficient of the kth stage; The calculation method of is: , wherein, is the i-th element in the sparse representation coefficient obtained in the k-1th stage.

[0010] Further, the intermediate feature of the kth stage is sequentially subjected to an adaptive threshold mechanism and , and the sparse representation coefficient of the kth stage is output, and the calculation method is specifically: Let the intermediate feature of the kth stage be , After the operation of , the feature matrix obtained is denoted as ; Let the feature matrix obtained by subjecting to the adaptive threshold mechanism be , The calculation method of is: , wherein, soft( ) is a soft threshold operation, is the feature matrix of all channels input to the soft threshold operation of the kth stage, , is the feature matrix of the c-th channel input to the soft threshold operation of the kth stage, and C is the number of channels; is the feature matrix of all channels after the soft threshold operation in the kth stage, is the L1 norm, is the L2 norm; is the soft threshold vector of the kth stage, , is the adaptive threshold of the cth channel in the kth stage; Will go through The obtained feature matrix is ​​recorded as , After the convolution operation, the C channels are reduced to a single channel to obtain the sparse representation coefficient of the kth stage.

[0011] Furthermore, The calculation method is: , in, represents the threshold learned by the cth channel in the kth stage, represents the content adaptive threshold calculated by the c-th channel in the k-th stage based on the input content, It is the element-wise multiplication operation; The calculation method is: , in, For Location The element value at , n is the length and width of the channel.

[0012] Furthermore, the structure of each nonlinear transformation includes four layers: the first layer is adaptive spatial parallel convolution, the second layer is ReLU function, the third layer is temporal sparsity enhanced convolution, and the fourth layer is adaptive spatial parallel convolution; The structure of the temporal sparsity enhanced convolution includes four convolutions: the first convolution is a standard convolution, the second convolution is composed of three dilated causal convolutions in series, the third and fourth convolutions are dilated convolutions, and the dilation rates of the third and fourth convolutions are different.

[0013] Furthermore, when training the adaptive deep expansion network model, the total loss function during training is constructed as: , in, is the total loss function, is the difference loss function, is the symmetric constraint loss function.

[0014] Furthermore, the difference loss function is calculated as follows: , wherein, m represents the amount of data for each training, and m is the length of the ultrasonic signal, is an over-complete dictionary, is the sparse coefficient of the i-th data output in the K-th stage, is the i-th data in the ultrasonic signal, is an L2 norm; The calculation method of the symmetric constraint loss function is: , wherein, n is the length of the sparse representation coefficient, and K is the number of iterations, and are two nonlinear transformation operations in the adaptive soft threshold value module of the k-th stage, is the result of the i-th data in the k-th stage obtained through the adaptive gradient descent module.

[0015] The application also provides a flip-chip ultrasonic signal denoising system based on an adaptive DUN, comprising: A signal acquisition module is configured to acquire the ultrasonic signal reflected by the flip-chip under test; An over-complete dictionary construction module is configured to fit the ultrasonic signal to obtain fitting parameters and construct an over-complete dictionary using the fitting parameters; A sparse representation model construction module is configured to construct a sparse representation model according to the inherent sparse characteristics of the ultrasonic signal and the over-complete dictionary; A target function optimization module is configured to construct a target function of the sparse representation model and iteratively optimize the target function, wherein the target function is used to solve the optimal solution of the sparse representation coefficient of the sparse representation model; An adaptive deep unfolding network model construction module is configured to take each iterative optimization of the target function as a stage, construct an adaptive deep unfolding network model comprising K stages connected in series through forward propagation, and set K as the total number of iterations; the working mechanism of the adaptive deep unfolding network model is that the over-complete dictionary, the ultrasonic signal and the sparse representation coefficient are input into the adaptive deep unfolding network model, and the adaptive deep unfolding network model is used to solve the optimal solution of the sparse representation coefficient stage by stage; each stage comprises an adaptive gradient descent module and an adaptive soft threshold value module, the adaptive gradient descent module is configured to independently optimize each element in the sparse representation coefficient, and the adaptive soft threshold value module comprises an adaptive threshold mechanism and a time sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between the channel and the input content, and the time sparsity enhancement convolution extracts the sparse feature and the time sequence correlation feature; The ultrasonic signal denoising module is configured to train the adaptive deep unfolding network model, obtain an optimal solution of sparse representation coefficients by using the trained adaptive deep unfolding network model, reconstruct the ultrasonic signal by combining an overcomplete dictionary, and obtain a denoised ultrasonic signal.

[0016] The above technical solution of the present application has the following beneficial effects compared with the prior art:

[0017] The adaptive gradient descent module is configured to independently optimize each element in the sparse representation coefficients, thereby obtaining more accurate sparse representation coefficients, and the original ultrasonic signal is fitted by using the more accurate sparse representation coefficients. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments of the present application and in conjunction with the accompanying drawings, in which: Figure 1 The flowchart of the method in the preferred embodiment of the present application.

[0019] Figure 2 The framework diagram of the adaptive deep unfolding network model in the preferred embodiment of the present application.

[0020] Figure 3 The framework diagram of the kth stage of the adaptive deep unfolding network model in the preferred embodiment of the present application.

[0021] Figure 4 The comparison diagram of the signal-to-noise ratio of the ultrasonic signal denoised by using the method of the present application and other methods in the simulation experiment.

[0022] Figure 5 The comparison diagram of the root mean square error of the ultrasonic signal denoised by using the method of the present application and other methods in the simulation experiment.

[0023] Figure 6 The waveform diagram of the actual ultrasonic signal denoised by using the method of the present application and other methods in the flip-chip. DETAILED DESCRIPTION

[0024] The present application will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not limiting the present application.

[0025] Referring to Figure 1 The application discloses a flip-chip ultrasonic signal denoising method based on adaptive DUN, which comprises the following steps: S1: obtaining an ultrasonic signal reflected by a defect of a flip-chip to be measured, and an expression of the ultrasonic signal is: , wherein, represents an actual amplitude of the ultrasonic signal collected at t, represents a reflection coefficient of the i th interface, represents an amplitude of the ultrasonic signal of the i th interface at t, represents an amplitude of noise from the flip-chip to be measured and a system at t, and M represents a total number of echoes.

[0026] S2: fitting the ultrasonic signal to determine Gabor atom fitting parameters by using a least square method, and constructing an over-complete dictionary (a Gabor dictionary) by using the fitting parameters.

[0027] S3: constructing a sparse representation model according to inherent sparse characteristics of the ultrasonic signal and the over-complete dictionary, and constructing an objective function containing an L1 norm regularization term of the sparse representation model, wherein the objective function is used to solve an optimal solution of sparse representation coefficients of the sparse representation model.

[0028] S3-1: performing sparse reconstruction on the ultrasonic signal, and obtaining the sparse representation model as: , wherein, is an original noisy ultrasonic signal actually obtained, , is the i th data in the ultrasonic signal; is an over-complete dictionary, , represents a m-dimensional space, m represents a length of the ultrasonic signal, and n represents a number of atoms in the over-complete dictionary, , is a sparse representation coefficient, , is noise.

[0029] S3-2: constructing an objective function containing an L1 norm regularization term to solve optimal sparse representation coefficients, and the objective function is specifically: , wherein, is an optimal sparse representation coefficient, is a data fitting term, is a sparse regularization term, is a regularization parameter, and is used to adjust a sparse degree, L1 norm, L2 norm.

[0030] S4: iteratively optimizing the objective function using an Iterative Shrinkage-Thresholding Algorithm (ISTA), gradually approaching the optimal solution through each iteration update.

[0031] Two steps of gradient descent operation and soft threshold operation are performed in each iteration update, specifically: The first formula performs gradient descent operation, and the second formula performs soft threshold operation, denotes the result of gradient descent update in the kth iteration, denotes the sparse solution of the sparse representation coefficient obtained by the soft threshold operation in the kth iteration, denotes the gradient descent step size, is a transpose operation, is a soft threshold.

[0032] The second formula is equivalently expressed as: wherein, denotes the soft threshold operation.

[0033] The entire optimization process is an iterative update process of two steps of gradient descent operation and soft threshold operation. Gradient descent is used for fitting the original signal, and soft threshold operation removes noise by deleting noise atoms. The above iterative solving process is expanded into a deep network architecture, and each iteration step corresponds to a stage of the network.

[0034] S5: depth expansion of the Iterative Shrinkage-Thresholding Algorithm, each iteration optimization of the objective function is regarded as a stage, and the optimal solution of the sparse representation coefficient is solved stage by stage using an adaptive deep expansion network model. The adaptive deep expansion network model is trained using back propagation, and the total loss function during training is constructed by combining the difference loss and the symmetry constraint loss.

[0035] An adaptive deep expansion network model including K stages of forward propagation in series is constructed as shown in Figure 2 K is the total number of iterations. Each stage in the adaptive deep expansion network model corresponds to each iteration step in the proximal gradient descent algorithm. After K stages, it is equivalent to performing K times of iteration optimization, Figure 3 is the kth stage of the adaptive deep expansion network model. Figure 3 ​​​In the following description, "Conv 1→C" means a 3x3 convolution operation from single channel to C channels, "Conv C→1" means a 3x3 convolution operation from C channels to single channel, "dim" and "dim" represent the dimension of the current feature.

[0036] The working mechanism of the adaptive deep unfolding network model is that the overcomplete dictionary, the ultrasonic signal and the sparse representation coefficient are input into the adaptive deep unfolding network model, and the optimal solution of the sparse representation coefficient is solved stage by stage using the adaptive deep unfolding network model. Each stage includes an adaptive gradient descent module (AGDM) and an adaptive soft threshold module (ASTM). The adaptive gradient descent module optimizes each element in the sparse representation coefficient independently, and the adaptive soft threshold module includes an adaptive threshold mechanism and a time sparsity enhancement convolution. The adaptive threshold mechanism enhances the association between the channel and the input content, and the time sparsity enhancement convolution extracts sparse features and time sequence correlation features.

[0037] The optimal solution of the sparse representation coefficient is solved stage by stage using the adaptive deep unfolding network model, and the specific process is as follows: S5-1: The sparse representation coefficient (denoted as ) output by the last stage (i.e. the k-1 stage) is input into the kth stage together with the ultrasonic signal and the overcomplete dictionary. First, the adaptive gradient descent module of the kth stage outputs the intermediate feature of the kth stage, denoted as . The adaptive gradient descent module expands the original scalar step into a vector form based on the gradient information to update each element in the sparse representation coefficient independently.

[0038] The calculation method of the intermediate feature of the kth stage is as follows: , wherein, is the intermediate feature of the kth stage, is the sparse representation coefficient obtained in the k-1 stage, is the gradient step of the sparse representation coefficient of the kth stage, is the overcomplete dictionary, is the transpose operation, is the ultrasonic signal.

[0039] The gradient step of the sparse representation coefficient of the kth stage is as follows: ; wherein, is the gradient step of the i-th element in the sparse representation coefficient of the kth stage, and n is the number of elements in the sparse representation coefficient. ​​ The calculation method is: , in, is the basic step length, which is taken as , To prevent the smoothing factor from being divided by 0, the value in this embodiment is , is the gradient of the i-th element in the sparse representation coefficient of the k-th stage; The calculation method is: , in, is the i-th element in the sparse representation coefficient obtained in the k-1th stage, that is, the sparse coefficient of the i-th data obtained in the k-1th stage.

[0040] It can be seen that the adaptive gradient descent module expands the original scalar step size into a vector form, so that each element in the sparse representation coefficient is optimized independently, thereby enhancing the network's parameter adjustment ability during gradient descent.

[0041] S5-2: The adaptive soft threshold module includes two nonlinear transformations, and the temporal sparsity enhanced convolution is embedded in the two nonlinear transformations.

[0042] The structures of the two nonlinear transformations are the same, and the structure of each nonlinear transformation includes four layers: the first layer is adaptive spatial parallel convolution (ASPConv), the second layer is ReLU function, the third layer is temporal sparsity enhanced convolution (TSEConv), and the fourth layer is adaptive spatial parallel convolution (ASPConv).

[0043] The adaptive spatial parallel convolution uses the “Adaptively Spatial Parallel Convolution Module” in Section 3.1 of the paper “QI GQ, ZHANG YC, WANG KP, et al. Small object detection method based on adaptive spatial parallel convolutionand fast multi-scale fusion[J]. Remote Sensing, 2022, 14(2): 420.”

[0044] The structure of the time sparsity enhanced convolution includes four convolutions: the first convolution is a standard convolution of 3x3, the second convolution is composed of three dilated causal convolutions in series; the third convolution and the fourth convolution are dilated convolutions, the dilation rates of the third convolution and the fourth convolution are different, in the embodiment, the third convolution is a dilated convolution with a dilation rate of 2, and the fourth convolution is a dilated convolution with a dilation rate of 4.

[0045] The two nonlinear transformations of the kth stage are denoted as and respectively; then the intermediate feature of the kth stage is input into the adaptive soft threshold module of the kth stage, and the intermediate feature of the kth stage sequentially passes through the nonlinear transformation , the adaptive threshold mechanism and the nonlinear transformation , and outputs the sparse representation coefficient of the kth stage, denoted as .

[0046] S5-2-1: the intermediate feature of the kth stage is denoted as , The feature matrix obtained after is denoted as , that is , is a nonlinear transformation operation.

[0047] S5-2-2: the feature matrix obtained after passes through the adaptive threshold mechanism is denoted as , The calculation method of , wherein soft( ) is a soft threshold operation, is the feature matrix of all channels input into the soft threshold operation of the kth stage, , is the feature matrix of the cth channel input into the soft threshold operation of the kth stage, and C is the number of channels; is the feature matrix of all channels after the soft threshold operation of the kth stage, is the L1 norm, is the L2 norm; is the soft threshold vector of the kth stage, , is the adaptive threshold of the cth channel of the kth stage.

[0048] The calculation method of , wherein denotes the threshold learned by the c-th channel in the k-th stage (i.e., the trainable channel-level threshold), denotes the content-adaptive threshold calculated by the c-th channel in the k-th stage according to the input content (i.e., the channel-aware threshold), is an element-wise multiplication operation; According to the feature distribution of each channel, the corresponding is adaptively estimated. The calculation method of is as follows: , wherein, is the element value at the position of , n is the length and width of the channel.

[0049] In this embodiment, in the first forward propagation process, the trainable channel-level threshold of each stage is initialized to 1, that is, the initial threshold is completely calculated by the features in the channel. Let the threshold learned in the k-th stage be , With the training, after each round of training, the threshold is updated stage by stage through the back propagation of the loss function.

[0050] S5-2-3: The feature matrix obtained after the operation of is denoted as , After the 3x3 convolution operation, the sparse representation coefficient of the k-th stage is obtained by reducing the C channels to a single channel. S5-1 and S5-2 are the working mechanism of the k-th stage of the adaptive deep unfolding network model.

[0051] S5-3: The adaptive deep unfolding network model has K stages, and the method in S5-1 and S5-2 is repeatedly used. The sparse representation coefficient output after all K stages (denoted as ) is taken as the optimal solution of the sparse representation coefficient.

[0052] In the training process of the adaptive deep unfolding network model, the adaptive deep unfolding network model is trained using back propagation, and the total loss function during training is constructed by combining the difference loss and the symmetry constraint loss as follows:

[0053] , wherein, is the total loss function, is the difference loss function, is the symmetry constraint loss function. ​​​​

[0054] The difference loss function trains the network parameters of the adaptive deep unfolding network model by measuring the difference between the reconstructed flip-chip ultrasonic signal and the original noisy ultrasonic signal The calculation method of the difference loss function is: , Wherein, represents the amount of data for each training, m is the length of the ultrasonic signal, is an overcomplete dictionary, indicates the sparse coefficient of the i-th data output in the K-th stage, that is, the i-th element in the sparse representation coefficient obtained in the K-th stage, is the i-th data in the ultrasonic signal, is the L2 norm.

[0055] The symmetry constraint loss function is used to ensure that the network follows the expectation of adaptive gradient descent and adaptive soft threshold operation, and the calculation method of the symmetry constraint loss function is: , Wherein, n is the length of the sparse representation coefficient, K is the iteration number, that is, the total number of stages, and are two nonlinear transformation operations in the adaptive soft threshold module of the k-th stage, is the result obtained by the adaptive gradient descent module for the i-th data in the k-th stage.

[0056] S6: training the adaptive deep unfolding network model, using the trained adaptive deep unfolding network model to obtain the optimal solution of the sparse representation coefficient, reconstructing the ultrasonic signal by combining the overcomplete dictionary, and obtaining the denoised ultrasonic signal: , Wherein, is the reconstructed ultrasonic signal, that is, the denoised ultrasonic signal.

[0057] The application also discloses a flip-chip ultrasonic signal denoising system based on adaptive DUN, comprising: A signal acquisition module is used to acquire the ultrasonic signal reflected by the defect of the flip-chip to be tested. An overcomplete dictionary construction module is used to fit the ultrasonic signal to obtain fitting parameters, and the fitting parameters are used to construct an overcomplete dictionary. A sparse representation model construction module is used to construct a sparse representation model according to the inherent sparse characteristics of the ultrasonic signal and the overcomplete dictionary. A target function optimization module is configured to construct a target function of a sparse representation model and iteratively optimize the target function, and the target function is used to solve an optimal solution of sparse representation coefficients of the sparse representation model. An adaptive deep unfolding network model construction module is configured to construct an adaptive deep unfolding network model including K stages of forward propagation in series, with each iteration of the optimization of the target function being a stage, and K being a total number of iterations; and a working mechanism of the adaptive deep unfolding network model is that the overcomplete dictionary, the ultrasonic signal and the sparse representation coefficients are input into the adaptive deep unfolding network model, and the adaptive deep unfolding network model is used to solve the optimal solution of the sparse representation coefficients stage by stage; each stage includes an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module is configured to independently optimize each element in the sparse representation coefficients, and the adaptive soft threshold module includes an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between a channel and input content, and the temporal sparsity enhancement convolution extracts sparse features and time sequence correlation features. An ultrasonic signal denoising module is configured to train the adaptive deep unfolding network model, use the trained adaptive deep unfolding network model to obtain the optimal solution of the sparse representation coefficients, combine the overcomplete dictionary to reconstruct the ultrasonic signal, and obtain the denoised ultrasonic signal.

[0058] The application further discloses a computer readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the flip-chip ultrasonic signal denoising method based on the adaptive DUN.

[0059] The application further discloses a device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the flip-chip ultrasonic signal denoising method based on the adaptive DUN when executing the computer program.

[0060] Compared with the prior art, the application has the following advantages. 1. The application adaptively adjusts a gradient descent step according to the sparsity of the ultrasonic signal, independently optimizes each element in the sparse representation coefficients, and thus more accurate sparse representation coefficients are obtained, and the ultrasonic signal is fitted using the more accurate sparse representation coefficients.

[0061] 2. The adaptive threshold strengthens the association between the threshold and the input content, so that the model can adaptively remove different levels of noise in the ultrasonic signal.

[0062] 3. According to the inherent sparsity and time correlation of the ultrasonic signal, a time sparsity enhanced convolution is proposed, which effectively avoids ghosting problem, more accurately describes the natural attenuation characteristics of the ultrasonic echo, improves the ability to reconstruct weak echoes, and can maintain strong robustness under high noise level, further improving the upper limit of denoising effect.

[0063] In order to further prove the advantages of the present application, the sparse reconstruction of the simulation signal and the actual signal of the flip-chip solder ball using the method of the present application is carried out in this embodiment. ISTA-Net+, SODAS-Net and MAUN based on deep unfolding method (for details, see the paper "Hu Q, Ma J, Gao Y, et al. MAUN: Memory-Augmented Deep Unfolding Network for Hyperspectral Image Reconstruction [J]. IEEE / CAA Journal of Automatica Sinica, 2024, 11(5): 1139-1150.") are selected, OMP (for details, see the paper "Tropp J A, Gilbert A C. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit [J]. IEEE Transactions on Information Theory, 2007, 53(12): 4655-4666. DOI: 10.1109 / TIT.2007.909108.") and MMP (for details, see the paper "Tropp J A, Gilbert AC. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit [J]. IEEE Transactions on Information Theory, 2007, 53(12): 4655-4666. DOI: 10.1109 / TIT.2007.909108.") based on sparse method, and wavelet denoising method based on traditional signal processing are used for comparison experiments.

[0064] The signal-to-noise ratio and the root mean square error are used as evaluation indexes, and the denoising results of the simulation ultrasonic signal of the flip-chip are shown in Figure 4 、 Figure 5 , Figure 4 、 Figure 5ADUN in the figure represents the method of the present application, and the wavelet denoising method. Figure 4 A comparison chart of signal-to-noise ratios of ultrasonic signals denoised by the method of the present application and other methods respectively, Figure 5 A comparison chart of root mean square errors of ultrasonic signals denoised by the method of the present application and other methods respectively. Figure 4 、 Figure 5 It can be seen that the wavelet denoising method and SODAS-Net can achieve good denoising effects at low noise levels, and SODAS-Net is slightly better than the wavelet denoising method, but as the noise level increases, the reconstruction quality of both rapidly decreases, and the denoising performance of SODAS-Net is equivalent to that of the wavelet denoising method at high noise levels. The denoising effects of OMP and MMP are basically the same, and the reconstruction quality is between that of the wavelet denoising method and SODAS-Net at low noise levels, but the results gradually become better than those of SODAS-Net as the noise level increases. ISTA-Net+ has relatively poor denoising effect at very low noise levels compared with other comparison methods, but as the noise level increases, the denoising performance gradually becomes better than that of the wavelet denoising method, SODAS-Net, OMP and MMP. MAUN shows better denoising ability overall, but its denoising performance decreases significantly at a certain stage when the noise level changes from low to high, reaching the performance limit of the method itself; thereafter, as the noise level further increases, the denoising effect maintains a certain noise reduction ability. The signal-to-noise ratio and root mean square error of the method of the present application are better than those of other methods, especially at high noise levels, which still maintains good denoising effect.

[0065] Then, the actual ultrasonic signal of the flip-chip is denoised using the method of the present application, and the denoising result is as shown in Figure 6 , Figure 6 (a) in the figure represents a waveform diagram of the actual ultrasonic signal before denoising, Figure 6 (b) in the figure represents a waveform diagram after denoising using the wavelet denoising method, Figure 6 (c) in the figure represents a waveform diagram after denoising using OMP, Figure 6 (d) in the figure represents a waveform diagram after denoising using MMP, Figure 6 (e) in the figure represents a waveform diagram after denoising using ISTA-Net+, Figure 6 (f) in the figure represents a waveform diagram after denoising using SODAS-Net, Figure 6 (g) in the figure represents a waveform diagram after denoising using MAUN, Figure 6 (h) in the figure represents a waveform diagram after denoising using the method of the present application. From Figure 6It can be seen that the wavelet denoising method still has some noise residue in the area where the characteristic waveform should be stable in the early stage, indicating that its suppression effect on background noise is insufficient; OMP and MMP have similar performance in overall denoising effect, but both have incomplete noise removal at the tail of the signal; SODAS-Net has improved noise reduction performance compared with the sparse method, but has an abnormal trend at the tail of the signal; ISTA-Net+ and MAUN also show obvious abnormalities at the tail of the signal, and there is waveform information loss at the circled area, which will affect the accuracy of judging the location of small structures or defects; the method of the present invention can not only effectively suppress most of the noise, but also reconstruct weak echo signals, and is superior to other existing methods in overall noise reduction effect.

[0066] From the simulation experiments, it can be seen that the denoising effect of the present invention has a higher signal-to-noise ratio and a lower root mean square error than the existing denoising method, and the denoising effect of the present invention is better. At the same time, the denoising comparison of the actual ultrasonic signal also verifies that the present invention can effectively mine the weak echo features while removing the ultrasonic signal noise, thereby achieving superior denoising performance.

[0067] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0070] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, so that the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0071] Obviously, the above embodiments are only examples for clearly illustrating, not limiting the embodiments. For those skilled in the art, based on the above description, other different forms of changes or variations can also be made. Here, all the embodiments are not required to be exhausted, and the obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A flip-chip ultrasonic signal denoising method based on adaptive DUN, characterized in that: include: Obtain the ultrasonic signal reflected by the flip chip to be tested, fit the ultrasonic signal to obtain fitting parameters, and use the fitting parameters to construct an over-complete dictionary; A sparse representation model is constructed based on the inherent sparse characteristics of the ultrasonic signal and an overcomplete dictionary, and an objective function of the sparse representation model is constructed, wherein the objective function is used to solve the optimal solution of the sparse representation coefficients of the sparse representation model; The objective function is iteratively optimized, and each iterative optimization of the objective function is regarded as a stage. An adaptive deep expansion network model is constructed, which includes K stages of forward propagation connected in series, where K is the total number of iterations; the overcomplete dictionary, the ultrasonic signal, and the sparse representation coefficient are input into the adaptive deep expansion network model, and the adaptive deep expansion network model is used to solve the optimal solution of the sparse representation coefficient stage by stage; each stage includes an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module independently optimizes each element in the sparse representation coefficient, the adaptive soft threshold module includes an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between the channel and the input content, and the temporal sparsity enhancement convolution extracts sparse features and temporal correlation features; The adaptive deep expansion network model is trained, and the optimal solution of the sparse representation coefficient is obtained using the trained adaptive deep expansion network model. The ultrasonic signal is reconstructed in combination with an overcomplete dictionary to obtain a denoised ultrasonic signal.

2. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 1, characterized in that: The adaptive deep expansion network model is used to solve the optimal solution of the sparse representation coefficients stage by stage, specifically: The working mechanism of the k-th stage of the adaptive deep expansion network model is as follows: the sparse representation coefficients, ultrasonic signals and over-complete dictionary outputted in the k-1 stage are used as the inputs of the k-th stage, and the intermediate features of the k-th stage are outputted through the adaptive gradient descent module of the k-th stage; the adaptive soft threshold module includes two nonlinear transformations, and the temporal sparsity enhanced convolution is embedded in the two nonlinear transformations. The two nonlinear transformations of the k-th stage are respectively denoted as and ; Input the intermediate features of the kth stage into the adaptive soft threshold module of the kth stage, and the intermediate features of the kth stage are sequentially , adaptive threshold mechanism and , output the sparse representation coefficient of the kth stage; The adaptive deep expansion network model has a total of K stages, and the sparse representation coefficients output after all K stages are used as the optimal solution of the sparse representation coefficients.

3. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 2, characterized in that: The calculation method of the intermediate features of the kth stage is: ， in, is the intermediate feature of the kth stage, is the sparse representation coefficient obtained in the k-1 stage, is the gradient step size of the sparse representation coefficient in the kth stage, is an overcomplete dictionary, T is a transposition operation, is an ultrasonic signal.

4. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 3 is characterized in that: The gradient step size of the sparse representation coefficient in the k-th stage is specifically: ； in, is the gradient step size of the i-th element in the sparse representation coefficient of the k-th stage, and n is the number of elements in the sparse representation coefficient; The calculation method is: ， in, is the basic step length, is the smoothing factor, is the gradient of the i-th element in the sparse representation coefficient of the k-th stage; The calculation method is: ， in, is the i-th element in the sparse representation coefficient obtained in the k-1th stage.

5. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 2, characterized in that: The intermediate features of the kth stage are sequentially , adaptive threshold mechanism and , output the sparse representation coefficient of the kth stage, specifically: The intermediate features of the kth stage are recorded as , go through The obtained feature matrix is ​​recorded as ; Will The feature matrix obtained by the adaptive threshold mechanism is recorded as , The calculation method is: ， Among them, soft( ) is the soft threshold operation, The feature matrix of all channels of the soft threshold operation is input to the kth stage, , The feature matrix of the cth channel of the soft threshold operation input in the kth stage, where C is the number of channels; is the feature matrix of all channels after the soft threshold operation in the kth stage, is the L1 norm, is the L2 norm; is the soft threshold vector of the kth stage, , is the adaptive threshold of the cth channel in the kth stage; Will go through The obtained feature matrix is ​​recorded as , After the convolution operation, the C channels are reduced to a single channel to obtain the sparse representation coefficient of the kth stage.

6. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 5, characterized in that: The calculation method is: ， in, represents the threshold learned by the cth channel in the kth stage, represents the content adaptive threshold calculated by the c-th channel in the k-th stage based on the input content, It is the element-wise multiplication operation; The calculation method is: ， in, For Location The element value at , n is the length and width of the channel.

7. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 2, characterized in that: The structure of each nonlinear transformation includes four layers: the first layer is adaptive spatial parallel convolution, the second layer is ReLU function, the third layer is temporal sparsity enhanced convolution, and the fourth layer is adaptive spatial parallel convolution; The structure of the temporal sparsity enhanced convolution includes four convolutions: the first convolution is a standard convolution, the second convolution is composed of three dilated causal convolutions in series, the third and fourth convolutions are dilated convolutions, and the dilation rates of the third and fourth convolutions are different.

8. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to any one of claims 1 to 7, characterized in that: When training the adaptive deep expansion network model, the total loss function during training is constructed as: ， in, is the total loss function, is the difference loss function, is the symmetric constraint loss function.

9. The flip-chip ultrasonic signal denoising method based on adaptive DUN according to claim 8, characterized in that: The calculation method of the difference loss function is: ， in, represents the amount of data for each training, m is the length of the ultrasonic signal, To complete the dictionary, is the sparse coefficient of the i-th data output in the K-th stage, is the i-th data in the ultrasonic signal, is the L2 norm; The calculation method of the symmetric constraint loss function is: ， Among them, n is the length of the sparse representation coefficient, K is the number of iterations, and are the two nonlinear transformation operations in the adaptive soft threshold module of the kth stage, It is the result obtained by the adaptive gradient descent module for the i-th data in the k-th stage.

10. A flip-chip ultrasonic signal denoising system based on adaptive DUN, characterized in that: include: A signal acquisition module is used to obtain the ultrasonic signal reflected by the flip chip to be tested; An overcomplete dictionary construction module is used to fit the ultrasonic signal to obtain fitting parameters, and use the fitting parameters to construct an overcomplete dictionary; A sparse representation model building module is used to build a sparse representation model based on the inherent sparse features of the ultrasound signal and an overcomplete dictionary; An objective function optimization module, configured to construct an objective function of a sparse representation model and iteratively optimize the objective function, wherein the objective function is used to find an optimal solution for the sparse representation coefficients of the sparse representation model; An adaptive deep expansion network model construction module is used to take each iterative optimization of the objective function as a stage, and construct an adaptive deep expansion network model including K stages of forward propagation connected in series, where K is the total number of iterations; the working mechanism of the adaptive deep expansion network model is as follows: the overcomplete dictionary, ultrasonic signal and sparse representation coefficient are input into the adaptive deep expansion network model, and the adaptive deep expansion network model is used to solve the optimal solution of the sparse representation coefficient stage by stage; each stage includes an adaptive gradient descent module and an adaptive soft threshold module, the adaptive gradient descent module independently optimizes each element in the sparse representation coefficient, the adaptive soft threshold module includes an adaptive threshold mechanism and a temporal sparsity enhancement convolution, the adaptive threshold mechanism enhances the association between the channel and the input content, and the temporal sparsity enhancement convolution extracts sparse features and temporal correlation features; The ultrasonic signal denoising module is used to train the adaptive deep expansion network model, use the trained adaptive deep expansion network model to obtain the optimal solution of the sparse representation coefficient, and reconstruct the ultrasonic signal in combination with the overcomplete dictionary to obtain the denoised ultrasonic signal.

Images