Automated defect classification method in scanning acoustic microscopy, software program product, and scanning acoustic microscope

Abstract

The present invention relates to an automatic defect classification method for a sample in a scanning acoustic microscope, a software program product, and a scanning acoustic microscope. In the method according to the present invention, a sample is scanned with a scanning acoustic microscope, a time sequence of recorded ultrasonic signals that are reflected and / or transmitted is digitized, analyzed for defects, and in particular, classified with respect to defects by at least one neural network pre-trained without a teacher in initial learning by a deep learning algorithm using a scanning acoustic microscope scan of one or more control samples of the same type as the sample or known labeled defects.

Description

Technical Field

[0001] The present invention relates to a method, a software program product, and a scanning acoustic microscope for automatically classifying defects in a sample by scanning acoustic microscopy inspection.

Background Art

[0002] Quality inspection by ultrasound (also called acoustic microscopy inspection) enables non-destructive measurement and inspection of materials and components. When generating an image in a scanning acoustic microscope, the sample to be inspected is scanned line by line, a short ultrasonic pulse is generated at each pixel, and the ultrasonic signal reflected from the sample is evaluated pixel by pixel. Usually, the temporal sequence of signals is analyzed within a specified time range (gate). The time range to be analyzed is defined in association with the surface signal of the sample and can also be made such that the sample is imaged at a predefined depth relative to the surface (surface trigger).

[0003] Generally, the maximum signal intensity or maximum amplitude within the selected time range is represented by the gray-scale value of each pixel or each raster point, thereby generating an image of the sample. However, complex data processing operators such as filters and transforms can also be used to generate this gray-scale value. The generated micrograph images the sample in a plane orthogonal to the transducer and is called a C-scan. Another image mode is the B-scan. The B-scan represents the acoustic cross-section of the sample. The position coordinates are plotted along the X-axis of the image, and the flight time of the acoustic signal is plotted along the Y-axis.

[0004] When scanning acoustic microscopy is used for production management and statistical quality control, C-scan images are typically evaluated using automated image processing. In addition to traditional algorithms such as threshold analysis, morphological filtering, and matrix-based image manipulation, artificial intelligence-based image-based solutions are also employed. The goal of image processing is to minimize misidentification of structures that are evaluated as false positives and to reliably detect critical defects in components.

[0005] For this purpose, one or more images from the ultrasound examination are transferred to image processing, where the received images are examined for various features. The coordinates and characteristics of any defects found are then transferred to the manufacturing management system. In some cases, a final good / bad product evaluation is performed, and defective components are sorted out. In particular, in the field of semiconductor manufacturing, the SECS / GEM communication standard is used for connection.

[0006] This method is suitable for traditional semiconductor components such as individual components using plastic bonding compounds, DCB-based power electronics using traditional solder joints, or wafers bonded by fusion bonding. In all of these applications, defects can be displayed with clear brightness compared to the remaining defect-free areas.

[0007] The methodological limitations of conventional image processing always arise when pixel luminance values ​​can no longer be clearly assigned to defects, and the same grayscale value can occur for both component defects and undamaged structures. In such cases, the capabilities of image processing can be improved by identifying component regions and performing only local analysis.

[0008] Analyzing ultrasonic data of complex components is often impossible with conventional image processing. For complex image content, machine learning-based image analysis is sometimes used. However, image-based machine learning requires a large amount of data to train the neural network. In particular, obtaining the necessary number of defect components is extremely difficult when training on low-frequency types of defects. For example, one workaround is to artificially augment a small number of typical defects. However, this can lead to network misinterpretation because non-representative data is used for training. Therefore, approaches using image-based artificial intelligence are very complex and unreliable in defect recognition applications in acoustic microscopy of materials and components because they cannot utilize sufficient image data for training artificial neural networks, and there are not enough good datasets available for comprehensive modeling.

[0009] In contrast, the objective of the present invention is to improve automated defect recognition in non-destructive testing of complex materials and components, particularly complex semiconductor samples. [Overview of the Initiative]

[0010] The objective is to automatically classify defects in a sample by scanning acoustic microscopy, particularly in the frequency range of 10 MHz to 2000 MHz, by scanning the sample with a scanning acoustic microscope equipped with one or more ultrasonic transducers, positioning the sample stepwise relative to the ultrasonic transducers at raster points, generating one or more ultrasonic signals at each raster point, and recording them after reflection on and / or within the sample and / or transmission through the sample, wherein the temporal sequence of reflected and / or transmitted and recorded ultrasonic signals is digitized and analyzed, and the classification with respect to defects is achieved in particular by scanning acoustic microscopy scans of one or more control samples of the same type as the sample, or by at least one neural network pre-trained unsupervised in initial training with a deep learning algorithm, using known labeled defects.

[0011] The method according to the present invention is applicable to a wide variety of samples, from simple material samples to complex semiconductor samples, and is based on the fact that current state-of-the-art scanning acoustic microscopes can detect, process, and store acoustic signals at each scan point in real time. These temporal sequences are also called A-scan signals. This allows for signal-based or volume-based analysis of the complete temporal signal sequence, replacing previous image-based analysis.

[0012] In principle, this avoids the unavoidable data reduction problem in previous image-based analyses, which occurs when the complex ultrasonic signal detected is reduced to a single value, such as the maximum amplitude within a considered time frame, during the image generation of each pixel, i.e., each raster point. This reduction prevents comprehensive information contained in the ultrasonic signal from being used for analysis. This is particularly problematic at high amplification rates, where the noise of a single grayscale value reaches critical levels compared to the threshold required in conventional image analysis, negatively impacting the reliability of defect detection.

[0013] The ultrasonic signals of complex semiconductor samples often include not only the signal portion directly corresponding to the coupled compression wave, but also signal portions generated by other waveforms. In high-speed materials, so-called mode conversion can occur. In this case, the coupled compression wave generates both a compression wave portion and a shear wave portion at the interface, and since these two propagate at different speeds within the sample, a time-shifted echo signal is generated. If two interfaces with high reflectivity face each other, the sound wave reflects back multiple times, forming multiple echoes. Transducers with large aperture angles can also generate Rayleigh waves and Lamb waves within the solid. All of these signals can be used to determine the presence or absence of defects. Therefore, weakly developed defects generate characteristic signatures at many different locations in the signal. These signatures provide a better defect identification database than a single grayscale value, as different parts of the signal can be used for analysis.

[0014] Using an artificial neural network capable of evaluating and classifying the overall signals of an interface increases the reliability of network decisions in data analysis, surpassing image-based methods. While labeling image data, necessary for training artificial neural networks for image data analysis, often involves limited data availability and significant manual effort, this method focuses on classifying individual A-scan signals. For this purpose, it is sufficient to train the artificial neural network with multiple A-scan signals from representative relevant regions. Even if the number of such signals is four orders of magnitude, this represents only a small fraction of the total data volume for a complete C-scan image.

[0015] The method according to the present invention aims to train a network with signals from relevant regions, thereby enabling the network to derive information and verify the usefulness of the complete information of the signals for training a deep learning network. Deep learning networks are highly efficient in automated feature extraction and often yield better results than manual selection can achieve. With proper optimization and continuous learning, deep learning networks can draw more conclusions than human operators. 3D data detected by a scanning acoustic microscope is used as raw data for training a statistical model. This data makes it possible to reconstruct all individual gate settings, such as A-scan, C-scan, B-scan, slices, and 3D scans (including amplitude and time-of-flight data), allowing samples to be displayed in individual layers.

[0016] Conventional methods, in their applicability, rely on clean signals from ultrasonic transducers and are incapable of handling broadband ultrasonic transducers resulting from numerous signals of different origins. In embodiments of the method according to the present invention, the ultrasonic transducer is designed as a broadband ultrasonic transducer having a bandwidth of at least 10%, and more particularly, at least 20%, and the ultrasonic transducer is designed to record and transmit to a receiver signals generated, in particular due to mode conversion, multiple echoes, Rayleigh waves, Lamb waves, and / or intrinsic characteristics of the transducer.

[0017] The method according to the present invention uses a neural network trained by supervised or unsupervised learning. In the first case, one control sample or several control samples, each with at least one defect, are scanned. The defect is either visually apparent to an expert or known from other inspections. The expert then marks one or more regions of the defect-free control sample and one or more defective regions. The neural network is trained to assign the corresponding ultrasonic signals to one of the categories. In the case of deep learning, the signal structures involved are not predetermined, and the neural network finds the most meaningful signal structures on its own.

[0018] In unsupervised learning, a control sample without defects is scanned. The ultrasonic signal of this control sample indicates to the neural network being trained what the ultrasonic signal should ideally look like. During the training phase, the neural network learns the entire range of acceptable signals for different structures in the control sample. This may consist of numerous different structures and their corresponding signal sequences that could not be imaged using conventional methods. Subsequently, when a defective sample is scanned, the ultrasonic signal at the location of the defect is not within the spectrum of acceptable signals and is different from them. The presence or absence of defects can be assessed by using a criterion that describes the deviation from the ideal signal image.

[0019] In this context, it is prudent to train the neural network with ultrasonic signals recorded with the same parameters as the signals to be classified later. This relates, in particular, to the bandwidth and transmission characteristics of the ultrasonic transducer, as well as the signal frequency of the ultrasonic pulse.

[0020] In the embodiment, the ultrasonic transducer is designed as a broadband ultrasonic transducer having a bandwidth of at least 10%, and more particularly, at least 20%, and the ultrasonic transducer is designed to record and transmit to a receiver signals generated, particularly due to mode conversion, multiple echoes, Rayleigh waves, Lamb waves, and / or the intrinsic characteristics of the transducer. Due to its high bandwidth, the ultrasonic signals transmitted by the broadband ultrasonic transducer contain far more information than ultrasonic transducers with small bandwidths or those operating with reduced bandwidths that are typically used. When using broadband ultrasonic transducers, this information density has made it impractical to apply corresponding image analysis based on neural networks to data generated using broadband ultrasonic transducers. On the other hand, the method according to the present invention improves the degree of classification separation by utilizing the additionally available information. Furthermore, the high information density of the broadband signal allows for training of the neural network with one or a very small number of scans.

[0021] In the embodiment, the neural network is a convolutional neural network (CNN), particularly a 1D-resnet architecture having one-dimensional convolutional blocks, and particularly the deep learning algorithm includes automated feature extraction. Alternatively, the neural network is a recurrent neural network (RNN), particularly having an architecture based on LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit), and particularly the deep learning algorithm includes adaptation of the feedforward stage of the RNN. The neural network may be designed as a hybrid of CNN and RNN.

[0022] In the embodiment, the ultrasonic signal is smoothed prior to the analysis, particularly by wavelet filtering, using a deterministic algorithm based on Daubechies wavelets or Mexican hat wavelets. The ultrasonic signal used to train the neural network should also be subjected to the same preprocessing in this case. Smoothing can help remove signal artifacts that may arise due to the design of the equipment.

[0023] In the embodiment, the temporal sequence of the ultrasonic signal is analyzed as a discrete time series of the form {signal length × 1}. Digitization causes the discrete time series to contain the lengths of thousands of data points, resulting in a vector of the same length. In this case, each raster point or each pixel is analyzed individually for whether or not it belongs to a defect. Additionally or alternatively, multiple ultrasonic signals from adjacent raster points may be combined and analyzed as a discrete time series of raster points, particularly as a dataset of the form {signal length × n × m}, arranged within a rectangle of side length n × m, where at least one of n and m is 2 or greater, in particular n=m, and in particular n and m are odd numbers. This includes combinations of multiple adjacent pixels row by row or column by column, or combinations of pixels in a rectangular or square arrangement. For example, in the case of a square arrangement, if odd-numbered pixels are selected as side lengths, the center of the square coincides with the central pixel. In the case of combinations of adjacent pixels, whether linear or planar, the central pixel is weighted more heavily than the surrounding pixels, allowing for the analysis or learning of the neural network to be adapted accordingly.

[0024] In the embodiment, processing performance is improved when the dataset of the discretized ultrasonic signals is two-dimensional, the discretized ultrasonic signals are in the first dimension, and a time step of 1 is in the second dimension. The actual value of the time step depends on the digitization rate of the analog ultrasonic signals.

[0025] In an embodiment, the classification includes one or more classes for the absence of defects, particularly one or more classes related to one or more structures, and one or more classes for defects, particularly classes of different types of defects.

[0026] In an embodiment, during the classification, a confidence level for each classification is determined, particularly expressed as an intensity value for each raster point.

[0027] In particular, for a systematic analysis of defects in the production of samples, in an embodiment, for raster points having defect classifications, the classification and particularly the confidence level therefor are also superimposed on a C-scan image as color or grayscale values. In particular, the confidence level is integrated as a luminance parameter, a color parameter, or a transparency parameter.

[0028] In an embodiment, before the processing by the neural network, the ultrasonic signal is converted by regression, particularly by interpolation, or by a learned statistical model, particularly without loss of information regarding the sample, into a signal of a predetermined length and / or a predetermined time increment, and particularly noise, signal artifacts, and / or interference signals and distortions due to design are reduced. According to the present invention, the loss of information is understood to be the loss of information indicating the sample, and the interference signals and distortions are inherent in the system and do not include any information regarding the sample itself. Ideally, the learning data of the reference sample is recorded under the same conditions and parameters as the ultrasonic signal of the sample to be tested later. However, in practice, since it is not possible to exactly repeat all test parameters, first, the system responses for different situations are recorded and analyzed, and it may be useful to remove parts of the system responses known during the preprocessing of the ultrasonic signal before learning and analysis, such as noise, signal artifacts, interference signals, and distortions.

[0029] Additionally or alternatively, the ultrasonic signal may be corrected for anomalies by an encoder-decoder architecture, where the ultrasonic signal is decomposed into multiple components, reconstructed based on learned domain knowledge, and the reconstruction error is derived from the difference between the input signal and the reconstructed signal. The encoder-decoder architecture converts the received ultrasonic signal into a good ultrasonic signal and is an example of regression. For this purpose, first, a statistical model, such as a neural network, is trained to convert a good signal into a good signal. This is a case of unsupervised learning, generating the aforementioned domain knowledge and incorporating it into the neural network. An abnormal signal deviating from the good signal due to the presence of defects is sequentially converted by the encoder-decoder architecture into the previously learned good signal, thereby correcting the anomalies. The difference between the output signal and the input signal forms the basis for defect classification and can be expressed, for example, by a threshold of the degree of deviation.

[0030] In an embodiment, the reconstructed 3D dataset of the sample volume of the sample is generated by a learned statistical model that takes into account the characteristics of ultrasonic propagation in the sample, corrected for the effects of defocus, multiple echoes, mode conversion signals, and transducer-specific signals, enabling the generation of cross-sectional images along any plane. Since these effects can be modeled and back-calculated from the ultrasonic signal, they can be removed.

[0031] In this embodiment, either an untrained neural network or a neural network pre-trained with at least one heterogeneous type of control sample from at least one heterogeneous sample type is used as the basis for the initial training. In the former case, training takes some time because the neural network is trained from an initial state, whereas in the latter case, there is already available information that only needs to be adapted to the current control sample. As a result, training converges relatively quickly depending on the similarity to the previously trained sample type. After training for the current sample type is complete, the current version of the neural network can be saved for later use with the same sample type, thereby setting up a database for different prototypes of neural networks without having to train the neural network anew each time.

[0032] In the embodiment, for the initial training of the neural network, the labeling of defects and other parts of the control sample is performed manually, in particular on the C-scan cross-section, and in particular, the manually marked area occupies less than 50%, particularly less than 20%, and particularly less than 10% of the scanned area of ​​the control sample. Therefore, in the method according to the present invention, the complete information content of the ultrasonic signal of each pixel is used, and thus the starting material can be greatly reduced. Thus, a limitation on a relatively small area of ​​the control sample is sufficient.

[0033] The object of the present invention can also be achieved by a software program product that, when executed on a data processing device for a scanning acoustic microscope, includes program code means for causing the scanning acoustic microscope to perform the method according to the present invention described above. Alternatively, the training of the neural network and the analysis of ultrasonic signals may be performed on a separate data processing device.

[0034] Furthermore, the object of the present invention is also achieved by a scanning acoustic microscope comprising a positioning system with a holder for a sample, at least one ultrasonic transducer, in particular a broadband ultrasonic transducer, a pulse generation and reception unit connected to or connectable to the at least one ultrasonic transducer, a data processing unit, and an analog-to-digital conversion unit, wherein the object of the present invention is also achieved by a scanning acoustic microscope designed and configured to carry out the aforementioned method according to the present invention by the aforementioned software program product which is executable and stored in the data processing unit.

[0035] The software program product and the scanning acoustic microscope achieve the same advantages, characteristics, and features as the method according to the present invention described above.

[0036] Further features of the present invention will become apparent from the description of embodiments, claims and accompanying drawings. Embodiments of the present invention may involve individual features or combinations of multiple features.

[0037] In the context of this invention, features designated as "particularly" or "preferably" should be understood as optional features. [Brief explanation of the drawing]

[0038] The following description, with reference to the drawings, uses exemplary embodiments to explain the general concept of the invention without limitation, and any further details of the invention not described herein are explicitly referenced in the drawings. These are shown below. [Figure 1] This is a schematic diagram of signal formation in scanning acoustic microscopy. [Figure 2] This is a schematic diagram of a one-dimensional CNN architecture. [Figure 3] This is a schematic diagram of the LSTM architecture. [Figure 4] This is a schematic diagram of unsupervised deep learning. [Figure 5] This is a schematic diagram of deep learning with preprocessing. [Figure 6] This diagram shows temporal sequence signals from different classifications. [Figure 7] This is a schematic diagram of a typical workflow in data analysis using machine learning. [Figure 8] This is a schematic diagram of the components of the method according to the present invention. [Modes for carrying out the invention]

[0039] In the drawings, identical or similar types of elements and / or parts are given the same reference numeral, so the corresponding locations are omitted.

[0040] Figure 1 shows a schematic of signal formation in scanning acoustic microscopy. As shown in the right half of the figure, an ultrasonic pulse 30 generated by a pulse generator (not shown), such as a piezoelectric crystal, typically containing frequencies between 10 MHz and 2000 MHz, is irradiated onto the sample 20 to be inspected by an ultrasonic transducer 12. At the front surface 22 of the sample 20, a portion of the pulse 30 is reflected as an echo 32, and another portion of the pulse is reflected at the back surface 24 as an echo 34. In between, another portion of the pulse 30 may be reflected as an echo 36 at the internal structure and / or defects of the internal volume 26 of the sample. The returned echoes 32, 34, and 36 are sequentially led by the ultrasonic transducer 12 to a receiver (not shown), which may be the same piezoelectric crystal that generated the pulse 30. The corresponding scanning acoustic microscope is known.

[0041] The left half of the figure shows the temporal sequence of the amplitude of the reflected signal, a so-called A-scan. Corresponding to the short propagation time, echo 32 returns first from the front, and this echo has the maximum amplitude. This is followed by a fairly weak echo 36 from inside the sample, and finally a pulsed echo 34 returns from the back 26. This signal sequence is also schematically shown vertically, corresponding to the depth cross-section of the sample 20.

[0042] In the analysis of a sample for defects, the middle portion of the signal is of interest. Therefore, a time gate 40 is applied to exclude echoes 32 and 34 of the pulse 30 from the front 22 and back 24. The signal located within the gate 40 contains various information about the internal state of the sample 20 at the current raster point or pixel.

[0043] Figure 2 schematically shows a one-dimensional CNN architecture of a convolutional network (CNN). A discrete temporal sequence of amplitude signals at gate 40 ("input time series") is introduced into the input layer 52 and serves as the input signal. This is followed by a convolutional plane 54, which is the so-called convolution base, an averaging layer 56 ("global average pooling"), and a classifier 58 having one or more fully coupled layers. The classifier 58 outputs a number of K initial classifications 60 indicating whether sample 20 contains a defect at its current position, and if so, what kind of defect or structure it is.

[0044] Figure 3 shows an alternative to the recurrent neural network 70, which includes a series of LSTM layers 72 on the input side 71, followed by a fully connected layer 74. The output of the fully connected layer 74 is processed, for example, by a softmax function 76 to derive a prediction 78 corresponding to the classification of the CNN.

[0045] Figure 4 schematically illustrates an example of unsupervised deep learning. A scanning acoustic microscope 10 scans a defect-free control sample 20 at each raster point ("single A-scan approach"). The temporal sequence is the gold standard because the control sample 20 is defect-free. Next, a neural network is trained with the entire A-scan data using a deep learning model. In this way, the neural network learns the features of the A-scan sequence of all structures contained in the control sample and is configured to optimally recognize these signals. Since the corresponding steps are known, the trained neural network has knowledge of what signals are predicted for the appropriate sample. Subsequently, A-scan signals from the location of defects can therefore be drawn to, identified, and marked. This can be seen in the partial image on the right, where defects are shown on the flip-chip sample.

[0046] Figure 5 shows a different method from Figure 4, in which the A-scan signal is preprocessed before training or analysis, for example, by wavelet filtering or upscaling.

[0047] Supervised learning may be performed instead of unsupervised learning. In this case, defective control samples are scanned instead of defect-free control samples. This is followed by a labeling step in which experts or other systems used for defect recognition mark characteristic regions of different defects, where applicable, on the control samples, and characteristic regions of different defect-free structures, where applicable. The neural network is then trained with the aim of reproducing the corresponding defect or structure classification as accurately as possible. The procedure for supervised learning is also publicly known.

[0048] Figure 6 shows the temporal ultrasonic signal sequences within gate 40 for some of the classifications described above. In this example, the sample is a defective flip-chip, corresponding to the right-hand representations in Figures 4 and 5, respectively. The gate start is indicated by a vertical line in both cases. From top to bottom, these are the signals from the defective bright solder joint, the defective dark solder joint, the normal solder joint, and the other structural regions. To the naked eye, the differences are barely perceptible. During training, the neural network learns which signal components or structures are characteristic of each classification.

[0049] Figure 7 shows a schematic of a typical workflow in machine learning-based data analysis. The scanning acoustic microscope 10 scans the sample, i.e., records a temporal sequence of ultrasonic signals at each scanning point on the sample. This is followed by procedure selection 110, and the analysis procedure 112 is selected. For example, each pixel may be analyzed individually, or neighbor evaluation may be performed on a square region with side lengths of, for example, 3 or 5 pixels. Multiple adjacent pixels in a row or column, or contiguous non-square regions of pixels, may be evaluated together, if applicable, with weights that slope from the center to the edges of the region.

[0050] After the selection in step 112, the amount of recorded signal is subjected to evaluation. This may be done online in real time, downstream, and offline if applicable. The recorded ultrasonic signal may be preprocessed 116, for example, by upsampling, downsampling, and / or filtering, particularly wavelet filtering. Regression 118 may be performed using, for example, an autoencoder 119. An example of this is a neural network trained by unsupervised learning. Alternatively, classification 120 may be performed using a CNN 50 or RNN 70 trained by supervised learning.

[0051] Figure 8 shows a schematic diagram of the components of the method according to the present invention. Referring to the remaining figures, particularly Figure 1, the scanning acoustic microscope 10 generates pulses 30, which are switched by the ultrasonic transducer 12 and returned in the form of various echoes 32, 34, and 36, which are digitized and analyzed. Possible but not limited to, the distribution of functions is single-signal based or context-based functions. For example, regression functions such as noise suppression (denoising) and upsampling are usually, but not necessarily, applied to individual ultrasonic signals. The individual ultrasonic signals may already be classified by a corresponding trained neural network.

[0052] Context-based functionality relates to the holistic evaluation of ultrasonic signals from adjacent scan points or pixels. These may be classified by a corresponding trained neural network. While the neural network becomes larger with more input data, the increased number of available signals also increases the likelihood of higher discriminative ability than with individual signal analysis. Because combining signals from adjacent raster points suppresses uncorrelated signal noise and generates a sharper reconstructed image, holistically evaluating these more complex signals is also well-suited for 2D or 3D reconstruction of samples. Of course, reconstruction can also be performed using individual signals.

[0053] All features cited, including those obtained solely from the drawings, and any individual features disclosed in combination with other features, are considered essential to the invention, either individually or in combination. Embodiments according to the present invention can be realized by individual features or combinations of features. [Explanation of symbols]

[0054] 10 Scanning Acoustic Microscope 12 Ultrasonic transducers 20 samples 22 Front 24 Back 26 Internal volume 30 Ultrasonic pulses 32 Echo from the front 34. Echo from the back 36 Echoes from internal volume 40 Gates 50 CNN 52 Input Layers 54 Folding Layers 56. Global Average Pooling 58 Classifier 60K output classification 70 RNN 71 Inputs 72 LSTM layer 74 Fully connected layer 76 Softmax rating 78 predictions 110 Step Selection 112 Selected Procedure 114 A scan 116 Preprocessing 118 Regression 119 Autoencoder 120 classification

Claims

1. A method for automatically classifying defects in a sample by scanning acoustic microscopy, comprising scanning the sample with a scanning acoustic microscope equipped with one or more ultrasonic transducers, positioning the sample stepwise at raster points relative to the ultrasonic transducers, generating one or more ultrasonic signals at each raster point, and recording them after reflection on and / or within the sample and / or transmission through the sample, The temporal sequence of reflected and / or transmitted ultrasonic signals is digitized and analyzed for defects by at least one neural network that has been unsupervisedly trained in advance by scanning acoustic microscopy scans of one or more control samples of the same type as the sample, or by scanning acoustic microscopy scans of one or more control samples of the same type as the sample, with known defects labeled. The temporal sequence of the ultrasonic signal is analyzed as a discrete time series of the form {signal length × 1} and / or volumetric analysis is performed, and multiple ultrasonic signals from adjacent raster points are combined and analyzed as a discrete time series of raster points arranged within a rectangle of side length n × m, as a dataset of the form {signal length × n × m}, where at least one of n and m is 2 or more, n = m or n <> m, and n and m are odd or even.

2. The method according to claim 1, wherein the scanning acoustic microscopy is performed in a frequency range of 10 MHz to 2000 MHz.

3. The method according to claim 1, wherein the analysis comprises classifying the temporal sequence of the recorded ultrasonic signals.

4. The method according to claim 1, wherein the ultrasonic transducer is designed as a broadband ultrasonic transducer having a bandwidth of at least 10% or at least 20%.

5. The method according to claim 4, wherein the ultrasonic transducer is designed to record and transmit to a receiver signals generated due to mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or intrinsic characteristics of the transducer.

6. The method according to claim 1, wherein the neural network is a convolutional neural network (CNN), a recurrent neural network (RNN), or a hybrid of a CNN and an RNN.

7. The CNN is a 1D-resnet architecture having one-dimensional convolutional blocks, and the deep learning algorithm includes automated feature extraction, or The method according to claim 6, wherein the RNN has an architecture based on LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit), and the deep learning algorithm includes adaptation of the feedforward stage of the RNN.

8. The method according to claim 1, wherein the ultrasonic signal is smoothed by a deterministic algorithm prior to the analysis.

9. The method according to claim 8, wherein the deterministic algorithm includes wavelet filtering based on Daubecies wavelets or Mexican hat wavelets.

10. The method according to claim 1, wherein the dataset of the discretized ultrasonic signals is two-dimensional, the discretized ultrasonic signals are in a first dimension, and one time step is in a second dimension.

11. The method according to claim 1, wherein the classification includes one or more classes for the absence of defects.

12. The method according to claim 11, wherein the classification comprises one or more classes for one or more defect-free structures in the sample, and classes for defects or defects of a different type.

13. The method according to claim 1, wherein the confidence level for each of the classifications is determined during the classification process.

14. The method according to claim 13, wherein the reliability is expressed by an intensity value for each raster point.

15. The method according to claim 1, wherein for raster points having defect classifications, the classification and / or confidence in the classification are superimposed on a C-scan image as color or grayscale values, and the confidence is integrated as a luminance parameter, a color parameter, or a transparency parameter.

16. The method according to claim 1, wherein, prior to processing by the neural network, the ultrasonic signal is converted into a signal of a predetermined length and / or a predetermined temporal increment by regression, interpolation, or a trained statistical model, with or without loss of information about the sample.

17. The method according to claim 16, wherein noise, signal artifacts and / or interference signals and distortions caused by the design are reduced.

18. The method according to claim 1, wherein the ultrasonic signal is corrected for anomalies by an encoder-decoder architecture, the ultrasonic signal is decomposed into a plurality of components, reconstructed based on learned domain knowledge, and the reconstruction error is derived from the difference between the input signal and the reconstructed signal.

19. The method according to claim 1, wherein a reconfigured 3D dataset of the sample volume of the sample is generated by a trained statistical model that takes into account the characteristics of ultrasonic propagation within the sample, correcting for the effects of defocus, multiple echoes, mode conversion signals and intrinsic transducer signals, thereby enabling the generation of a cross-sectional image along an arbitrary plane.

20. The method according to claim 1, wherein, as the basis for the initial training, an untrained neural network or a neural network pre-trained using at least one heterogeneous type of control sample from at least one heterogeneous sample type is used.

21. The method according to claim 1, wherein, for the initial training of the neural network, defects and other parts of the control sample are manually labeled, and the manually marked areas occupy less than 50%, less than 20%, or less than 10% of the scanned area of ​​the control sample.

22. The method according to claim 21, wherein the labeling is performed on a C-scan cross-section.

23. A software program product comprising program code means, which, when executed on a data processing device for a scanning acoustic microscope, causes the scanning acoustic microscope to execute the method according to any one of claims 1 to 22.

24. A scanning acoustic microscope comprising a positioning system with a holder for a sample, at least one ultrasonic transducer, a pulse generation / reception unit connected to or connectable to the at least one ultrasonic transducer, a data processing unit, and an analog-to-digital conversion unit, wherein a software program product executable and stored in the data processing unit is used to carry out the method according to any one of claims 1 to 22.

Images