Method, software program product and scanning acoustic microscope for automatic defect classification in scanning acoustic microscopy

Abstract

The invention relates to a method for automatic defect classification of a sample in scanning acoustic microscopy, a software program product and a scanning acoustic microscope. In the method according to the invention, a sample is scanned with a scanning acoustic microscope, in which a time sequence of recorded reflected and / or emitted ultrasound signals is digitized and analyzed, in particular classified, for defects by means of at least one neural network, the neural network is previously unsupervised trained in initial training by a deep learning algorithm with the aid of a scanning acoustic microscope scan of one or more control samples of the same type as the sample or with known and marked defects.

Description

Technical Field

[0001] The invention relates to a method for automatic defect classification of a sample by scanning acoustic microscopy, a software program product and a scanning acoustic microscope. Background Art

[0002] Quality inspection by ultrasound (also called acoustic microscopy) enables non-destructive measurement and inspection of materials and components. During image generation in a scanning acoustic microscope, the sample to be inspected is scanned line by line and a short ultrasound pulse is generated at each pixel, the ultrasound signal reflected by the sample being evaluated pixel by pixel. Typically, the time series of the signal is analyzed within a specified time range (gate). The time range to be analyzed can also be defined relative to the surface signal of the sample to ensure that the sample is imaged at a previously specified depth relative to the surface (surface trigger).

[0003] Typically, the maximum signal intensity and amplitude within the selected time range are represented by a grayscale value for each pixel and each raster point, respectively, and thus an image of the sample is generated. However, in order to generate the grayscale values, complex data processing operators such as filters or transformations can also be used. The resulting micro-image images the sample in a plane perpendicular to the transducer and is called a C-scan. Another imaging mode is the B-scan. The B-scan represents an acoustic cross-section through the sample. The position coordinates are plotted along the X-axis of the image and the flight time of the sound signal along the Y-axis.

[0004] In the case of the use of scanning acoustic microscopy in production control and statistical quality control, the C-scan images are usually evaluated by automated image processing. In this case, image-based solutions using artificial intelligence are employed in addition to conventional algorithms such as threshold analysis, morphological filters and matrix-based image operations. The goal of image processing is to reliably detect critical defects in components with as few erroneous evaluations as possible in the form of false positive evaluation structures.

[0005] To this end, one or more images are transmitted from the ultrasonic inspection to the image processing, which then checks various features of the received images. The coordinates or characteristics of the defects found are then transmitted to the manufacturing control system. In some cases, a final good / bad assessment is performed and defective parts are sorted out. Specifically, in the field of semiconductor manufacturing, the communication standard SECS / GEM is used for connection.

[0006] The method works well for conventional semiconductor components, such as individual components with plastic socket solder compounds, DCB-based power electronics with conventional solder joints, or wafers joined together by fusion bonding. In all of these applications, defects can be displayed with clear brightness compared to the remaining defect-free areas.

[0007] The methodological limits of conventional image processing are always reached when the brightness value of a pixel can no longer be clearly assigned to a defect, and the same grayscale values ​​occur both in the case of component defects and in the case of intact structures. Here, the capabilities of image processing can be improved by identifying component areas and performing only local analyses.

[0008] With conventional image processing, the analysis of ultrasonic data of complex components is generally no longer feasible. In the case of complex image content, machine learning-based image analysis can also be used. However, image-based machine learning requires a large amount of data to train the neural network. In particular, in order to train rarely occurring defect types, obtaining the required high number of defective components may be very delicate. For example, one solution is to artificially magnify some exemplary defects. However, this may lead to a misunderstanding of the network because non-representative data is used for training. Therefore, methods with image-based artificial intelligence are very complex and hardly reliable in the application of defect identification in acoustic microscopy on materials and components, because there is only insufficient image material available for training artificial neural networks and not good enough data sets for comprehensive modeling. Summary of the invention

[0009] In contrast, the object of the present invention is to improve the automatic defect recognition in the non-destructive inspection of (even complex) materials and components, in particular complex semiconductor specimens.

[0010] The object is achieved by a method for automatic defect classification of a sample by scanning acoustic microscopy, in particular in the frequency range of 10 MHz to 2000 MHz, wherein the sample is scanned using a scanning acoustic microscope with one or more ultrasonic transducers, the sample is progressively positioned at raster points relative to the one or more ultrasonic transducers and one or more ultrasonic signals are generated at each raster point and one or more ultrasonic signals are recorded after reflection on and / or in the sample and / or after transmission through the sample, wherein the time series of the recorded reflected and / or transmitted ultrasonic signals are digitized and analyzed (in particular for classification) for defects by at least one neural network, which has previously been unsupervisedly trained by a deep learning algorithm in an initial training with the help of scanning acoustic microscope scans of one or more control samples of the same type as the sample or with known and marked defects.

[0011] The method according to the invention is applicable to different types of samples, from simple material samples to complex semiconductor samples, and is based on the fact that a scanning acoustic microscope corresponding to the prior art allows the detection, real-time processing and storage of acoustic signals at each scanning point. These time series are also referred to as A-scan signals. Furthermore, a signal-based or volume-based analysis of the complete time signal series can be performed instead of the previous image-based analysis.

[0012] In principle, the problem of data reduction that was unavoidable in previous image-based analyses is thereby avoided, the cause of which is that during image generation for each pixel (i.e. each raster point), the individually detected and possibly complex ultrasound signal is reduced to a single value, such as the maximum value of the amplitude in the considered time frame. With this reduction, the comprehensive information contained in the ultrasound signal is lost in the analysis. This leads, among other things, to the noise of the individual gray values ​​reaching critical values ​​compared to the threshold values ​​required in conventional image analysis, especially at high magnifications, and has a negative impact on the certainty of the defect decision.

[0013] The ultrasonic signal of a complex semiconductor sample usually contains signal parts that not only directly correspond to the signal parts of the coupled compression wave, but also are generated by other wave types. In fast materials, so-called mode conversion may occur. Here, the coupled compression wave generates a compression wave part and a shear wave part at the interface, which move at different propagation speeds in the sample and thus generate time-shifted echo signals. If two interfaces with large reflection coefficients are opposite to each other, multiple echoes are formed caused by the sound waves reflected back and forth. Transducers with large aperture angles can also induce Rayleigh waves and Lamb waves in solid objects. All these signals can be used for defect / non-defect judgment. Weakly developed defects therefore produce characteristic tags at many different locations of the signal. These tags provide a better database for identifying defects than just a single grayscale value, because different parts of the signal can be used for analysis.

[0014] The use of an artificial neural network that can evaluate and classify the overall signal of the interface increases the reliability of the network's decisions in its data analysis and exceeds the reliability of image-based methods. In contrast to the usually limited data availability and considerable manpower required in the case of labeled image data for training artificial neural networks for image data analysis, the proposed method therefore focuses on the classification of individual A-scan signals. For this purpose, it is sufficient to train the artificial neural network with multiple A-scan signals from representative and relevant areas. Even 4-digit signals of this type represent only a small percentage of the total data volume relative to the complete C-scan image.

[0015] The method according to the present invention is intended to train the network with signals from the relevant area so that the network can derive information and check the complete information of the signal for the availability of deep learning network training. Deep learning networks are very effective in automatic feature extraction, and generally obtain better results here than those achievable in the case of manual selection. Through correct optimization and continuous learning, deep learning networks can draw more conclusions than human operators. 3D data detected by scanning acoustic microscopes are used as raw data for training statistical models. These data make it possible to reconstruct all individual gate settings, such as A-scans, C-scans, B-scans, slices and 3D scans (including amplitude and time of flight data), so that samples can be displayed in separate layers.

[0016] While previous methods rely on a clean signal from the ultrasonic transducer for applicability, this can be difficult to cope with in the case of a broadband ultrasonic transducer due to signals from many different sources. In an embodiment of the method according to the invention, the ultrasonic transducer(s) are designed as broadband ultrasonic transducers, specifically, the broadband ultrasonic transducer(s) have a bandwidth of at least 10%, specifically at least 20%, wherein the ultrasonic transducer(s) are specifically designed to record signals resulting from mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or due to the intrinsic characteristics of the transducer, and forward the signals to a receiver.

[0017] The method according to the invention uses a neural network that has been trained in a supervised or unsupervised manner. In the first case, a control sample or several control samples (at least one of which is defective) are scanned, the defects of which are visible to the expert or known from other examinations. The expert then marks one or more defect-free areas and one or more defective areas of the control sample. The training of the neural network prepares to assign the corresponding ultrasound signal to one of these classifications. In the case of deep learning, it is not preset which signal structures are of interest, but the neural network finds the most meaningful signal structures by itself.

[0018] In the case of unsupervised learning, a control sample without defects is scanned. The ultrasound signal of this control sample shows the ideal appearance of ultrasound signals to the neural network trained with it. During the training phase, the neural network has thus learned the full range of acceptable signals for different structures of the control sample. This can be a large number of different structures and corresponding signal sequences that cannot be imaged in a conventional way. If a defective sample is scanned later, the ultrasound signal at the defect location will not be in the spectrum of acceptable signals, but will be different from it. With the help of a criterion that describes the deviation from the ideal signal image, it can then be assessed whether a defect exists.

[0019] In this context, it is sensible to train the neural network with ultrasound signals that have been recorded with the same parameters as the signals to be classified later. This concerns, among other things, the bandwidth and transmission characteristics of the ultrasound transducer and the signal frequency of the ultrasound pulses.

[0020] In an embodiment, one or more ultrasonic transducers are designed as one or more broadband ultrasonic transducers, in particular, this or these broadband ultrasonic transducers have a bandwidth of at least 10%, in particular at least 20%, wherein this or these ultrasonic transducers are specifically designed to record signals generated from mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or due to the intrinsic characteristics of the transducer, and forward the signals to a receiver. Due to their high bandwidth, the ultrasonic signals emitted by the broadband ultrasonic transducers contain significantly more information than the commonly used ultrasonic transducers, which include a smaller bandwidth or operate with a smaller bandwidth. In the case of using broadband ultrasonic transducers, such information density has so far made it impractical to perform corresponding image analysis of data generated by means of broadband ultrasonic transducers based on neural networks. On the other hand, the method according to the invention utilizes additionally available information to improve the separation sharpness of the classification. The high information density of the broadband signal also enables the neural network to be trained by only one or very few scans.

[0021] In an embodiment, the neural network is a neural convolutional network (CNN), in particular having a 1-D resource network architecture with one-dimensional convolutional blocks, wherein in particular the deep learning algorithm in particular comprises automated feature extraction. Alternatively, the neural network is a recurrent neural network (RNN), in particular having a long short-term memory (LSTM) based architecture or a gateway recurrent unit (GRU) architecture, wherein in particular the deep learning algorithm in particular comprises an adaptation of the feedforward phase of the RNN. The neural network can also be designed as a hybrid of a CNN and a RNN.

[0022] In an embodiment, the ultrasound signal is smoothed before analysis by a deterministic algorithm, in particular by wavelet filtering, in particular based on the Daubeches wavelet or the Mexican Hat wavelet. Moreover, in this case, the ultrasound signal (with the help of a neural network trained on the ultrasound signal) should be subjected to the same preprocessing. Smoothing can be used to eliminate signal artifacts that may be due to the instrument design.

[0023] In an embodiment, the time series of the ultrasonic signal is analyzed as a discrete time series in the form of {signal length x 1}. Due to digitization, the discrete time series may contain a length of thousands of data points, resulting in a vector of the same length. In this case, each raster point (respectively each pixel) is analyzed individually as to whether it belongs to a defect. Additionally or alternatively, a volume analysis may be performed, in which several ultrasonic signals of adjacent raster points are combined and analyzed as a discrete time series of raster points, specifically arranged in a rectangle with an edge length of nxm, and several ultrasonic signals of adjacent raster points are combined and analyzed as a data set in the form of {signal length xnxm}, where at least one of n and m is ≥2, where specifically n=m, where specifically n and m are odd numbers. This includes a row-by-row or column-by-column combination of multiple adjacent pixels, a combination of pixels in a rectangular or square arrangement, respectively, where, for example, if an odd number of pixels is selected as the edge length in the case of a square arrangement, the center of the square then coincides with the center pixel. In the case of a combination of adjacent pixels, whether this occurs in a linear or planar form, the analysis is performed accordingly. The training of the neural network can be adapted because central pixels are weighted higher than peripheral pixels.

[0024] In an embodiment, a gain in processing performance is obtained if the dataset of discretized ultrasound signals is two-dimensional, with discretized ultrasound signals in a first dimension and a time step of 1 in a second dimension. The actual value of the time step depends on the digitization rate of the analog ultrasound signal.

[0025] In an embodiment, the classification includes one or more non-defective classifications and one or more defective classifications, the one or more non-defective classifications are specifically one or more classifications of one or more structures, and the one or more defective classifications are specifically different types of defects.

[0026] During classification, in an embodiment, a confidence level of the respective classification is determined, which confidence level is specifically represented by an intensity value in the representation of each raster point.

[0027] In order to systematically analyze defects in sample production, among other things, in an embodiment, the classification (and specifically also the confidence in the classification) is superimposed on the C-scan image as color or grayscale values ​​of the raster points with the defect classification, wherein specifically the confidence is integrated as a brightness parameter, a color parameter or a transparency parameter.

[0028] In an embodiment, before being processed by the neural network, the ultrasound signal is transformed by regression, in particular, the ultrasound signal is transformed by interpolation or by a trained statistical model, converting the ultrasound signal into a signal of a predefined length and / or predefined time increment, in particular, without losing information about the sample, wherein in particular, noise, signal artifacts and / or interference signals and distortion due to design are reduced. According to the present invention, information loss is understood to be indicative of information loss of the sample, while interference signals and distortion are inherent in the system and do not contain any information about the sample itself. Although ideally the training data on the control sample is recorded under the same conditions and parameters as the ultrasound signal on the sample to be tested later, in practice not all test parameters are exactly repeatable, so that it may be useful to first record and analyze the system response to different conditions and remove those parts of the system response that are known during preprocessing of the ultrasound signal before training and before analysis (such as noise, signal artifacts, interference signals and distortion).

[0029] Additionally or alternatively, the ultrasound signal may be subjected to anomaly correction by means of an encoder-decoder architecture, wherein the ultrasound signal is decomposed into components and reconstructed again based on trained domain knowledge, wherein the reconstruction error is derived from the difference between the input signal and the reconstructed signal. The encoder-decoder architecture images the received ultrasound signal onto a good ultrasound signal and is therefore an example of regression. For this purpose, a statistical model, such as a neural network, is first trained so that it images good signals onto good signals in turn. This is the case of unsupervised learning, which generates the mentioned domain knowledge, which is then included in the neural network. Abnormal signals that deviate from good signals due to the presence of defects are in turn imaged by the encoder-decoder architecture onto previously learned good signals and are thereby anomaly corrected. Therefore, the difference between the output signal and the input signal forms the basis for defect classification, which can be represented, for example, via a threshold value of the degree of deviation.

[0030] In an embodiment, a reconstructed 3D dataset of a sample volume of a sample is generated by a trained statistical model taking into account the characteristics of ultrasound propagation in the sample, which corrects for effects such as defocus, multiple echoes, mode conversion signals, and intrinsic transducer signals, allowing cross-sectional images to be generated along any plane. These effects can be modeled and calculated back from the ultrasound signal, thereby removing them.

[0031] In an embodiment, an untrained neural network or a pre-trained neural network on at least one control sample of an alien type of at least one alien sample type is used as a basis for initial training. Training in the first case will take slightly longer, since the neural network is trained from scratch, whereas in the latter case there is already available information that only needs to be adapted to the current control sample. Thus, learning converges relatively quickly, depending on the similarity to previously trained sample types. After training of the current sample type is completed, the current version of the neural network can be stored for later use on the same sample type, so that a database of neural networks of different prototypes is built up without the need to retrain the neural network each time.

[0032] For the initial training of the neural network, the labeling of defects and other parts of the control sample was performed manually, in particular on C-scan segments, wherein in particular the areas marked by manual labor accounted for less than 50%, in particular less than 20%, in particular less than 10% of the scanned area of ​​the control sample. Thus, the method according to the invention manages with significantly less starting material, since the entire information content of the ultrasound signal per pixel is used. Thus, a restriction to a relatively small area of ​​the control sample is sufficient.

[0033] The object of the invention is also achieved by a software program product having program code means which, when executed on a data processing facility of a scanning acoustic microscope, causes the scanning acoustic microscope to carry out the method according to the invention as described above. Alternatively, the training of the neural network and the analysis of the ultrasound signals can also be carried out on separate data processing facilities.

[0034] Furthermore, the object of the present invention is also through a scanning acoustic microscope, which has a positioning system, which has a holder for a sample, at least one ultrasonic transducer, a pulse generating and receiving unit, a data processing facility and an analog-to-digital converter unit, at least one ultrasonic transducer being specifically a broadband ultrasonic transducer, the pulse generating and receiving unit being connected or connectable to at least one ultrasonic transducer, wherein the scanning acoustic microscope is designed and configured for performing the previously described method according to the present invention, specifically, by means of the previously described software program product executable and stored in the data processing facility.

[0035] The software program product and the scanning acoustic microscope achieve the same advantages, properties and features as the method according to the invention described previously.

[0036] Further features of the invention will become clear from the description of embodiments according to the invention as well as from the claims and the drawings.Embodiments according to the invention may implement individual features or a combination of several features.

[0037] In the context of the present invention, features identified with "specifically" or "preferably" are to be understood as optional features. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The invention will be described below by means of exemplary embodiments with reference to the accompanying drawings, without limiting the general inventive concept, and any details according to the invention which are not further explained in the text will be explicitly described with reference to the accompanying drawings. They are shown below:

[0039] Figure 1 is a schematic diagram of signal formation in a scanning acoustic microscope.

[0040] Figure 2 is a schematic representation of the 1-D CNN architecture;

[0041] Figure 3 is a schematic representation of the LSTM architecture,

[0042] Figure 4 is a schematic representation of unsupervised deep learning,

[0043] Figure 5 is a schematic representation of deep learning with preprocessing;

[0044] Figure 6 Time series signals from different classifications;

[0045] Figure 7 is a schematic representation of the general workflow in data analysis via machine learning, and

[0046] Figure 8 is a schematic diagram of components of a method according to the present invention.

[0047] In the drawings, elements and / or parts of the same or similar types are provided with the same reference numerals, thereby omitting the corresponding re-introduction, respectively. DETAILED DESCRIPTION

[0048] Figure 1A schematic diagram of signal formation in scanning acoustic microscopy is shown. As can be seen from the right half of the figure, an ultrasonic pulse 30 (which typically comprises a frequency between 10 MHz and 2000 MHz) generated by a pulse generator (not shown), such as a piezoelectric crystal, is directed by an ultrasonic transducer 12 onto a sample 20 to be examined. On the front surface 22 of the sample 20, a portion of the pulse 30 is reflected as an echo 32, another portion of the pulse is reflected as an echo 34 on the rear surface 24. Meanwhile, another portion of the pulse 30 may be reflected as an echo 36 on internal structures and / or defects in the inner volume 26 of the sample. The returning echoes 32, 34, 36 are directed one by one by the ultrasonic transducer 12 to a receiver (not shown). It can therefore be the same piezoelectric crystal that has also generated the pulse 30. Corresponding scanning acoustic microscopes are known.

[0049] In the left half of the figure, a time sequence of the amplitude of the reflected signal is represented (the so-called A-scan). Corresponding to the shorter travel time, the echo 32 returns first from the front surface. This one also has the largest amplitude. The substantially weaker echo 36 from the interior of the sample follows, and the echo 34 of the last pulse comes from the rear surface 26. The signal sequence is also schematically shown in a vertically aligned manner corresponding to the depth profile of the sample 20.

[0050] For defect analysis of the sample, the middle part of the signal is of interest. Therefore, a time gate 40 is applied which excludes the echoes 32, 34 of the pulse 30 from the front surface 22 and the back surface 24. The signal located within the gate 40 contains various information about the internal lifetime of the sample 20 at each pixel on the current raster point.

[0051] exist Figure 2 , a 1-D CNN architecture of a convolutional network (CNN) is schematically represented. A discrete time series of amplitude signals (“input time series”) introduced into the gate 40 of the input layer 52 is used as input signal. This is followed by a sequence of convolution planes 54 (the so-called convolutional basis), followed by an averaging layer 56 (“global average pooling”) and a classifier 58 with one or more fully connected layers, which issues K initial classifications 60, from which it is concluded whether the sample 20 includes a defect at the current position and, if applicable, what type of defect or what type of structure.

[0052] Figure 3 An alternative recurrent neural network 70 is shown comprising a series of LSTM layers 72 on one side of an input 71 followed by a dense layer 74. The output of the dense layer 74 is processed, for example, by a softmax function 76, to obtain a prediction 78 corresponding to a classification in a CNN.

[0053] exist Figure 4In , an example of unsupervised deep learning is schematically represented. A scanning acoustic microscope 10 scans a defect-free control sample 20 at each raster point ("single A-scan method"). Since the control sample 20 has no defects, the time series forms the gold standard. In the following, a neural network is trained on the entire data of the A-scan by a deep learning model. In this way, the neural network learns the characteristics of the A-scan sequence of all structures contained in the control sample and builds itself so that it optimally recognizes these signals. The corresponding procedures are known, so that the trained neural network has knowledge of what kind of signals to expect in suitable samples. Later, the A-scan signal from the defect location will therefore attract attention and can be identified and marked. This can be recognized in the partial image on the right, where a defect is indicated on the flip chip sample.

[0054] exist Figure 5 In, an alternative method is shown, which is similar to Figure 4 The method of is different in that, before analysis, the A-scan signal is preprocessed accordingly before training, for example, by wavelet filtering, amplification, etc.

[0055] Instead of unsupervised learning, supervised learning can also be performed. In this case, a defective control sample is scanned instead of a defect-free control sample. This is followed by a marking step in which an expert or another system for defect recognition marks areas on the control sample that are characteristic of different defects on the one hand (if applicable) and of different defect-free structures of the sample on the other hand (if applicable). The neural network is then trained to reproduce the corresponding defects. The structural classification is performed as accurately as possible. The procedure for supervised learning is also known.

[0056] exist Figure 6 In FIG. 4 , a temporal sequence of ultrasound signals in the gate 40 is shown for each of the aforementioned classifications. In this example, the sample is a defective flip chip, corresponding to Figure 4 and 5 The corresponding right-hand representation in . The gate start is indicated in each case by a vertical line. From top to bottom, these are: the signal from a defective light solder joint, the signal from a defective dark solder joint, the signal from a complete solder joint, and the signal from other structural areas. The naked eye can hardly discern any differences. The neural network learns during its training which signal components or structures are characteristic of the different classifications.

[0057] Figure 7A schematic representation of a general workflow in data analysis by machine learning is shown. A scanning acoustic microscope 10 performs a scan of a sample, i.e. a time series of ultrasound signals is recorded at each scan point on the sample. This is followed by a process selection 110, in which an analysis process 112 is selected, e.g. analysis of each pixel individually or neighborhood evaluation of, e.g., a square area comprising, e.g., an edge length of 3 or 5 pixels. If applicable, multiple neighboring pixels of a row or series or non-square continuous pixel area may be evaluated together, with a weighting that is tilted from the center to the edge of the area.

[0058] After the selection procedure 112, the amount of the recorded signal is fed to the evaluation. This can be done online and in real time, but can also be done downstream and, if applicable, offline. Among them, the recorded ultrasound signal can be subjected to preprocessing 116, for example, upsampling, downsampling and / or filtering, specifically wavelet filtering. Regression 118 can occur, for example, by using an autoencoder 119. Such an example is a neural network that has been trained unsupervised. Moreover, classification 120 can be performed by using a CNN 50 or RNN 70, which is trained by supervised learning.

[0059] Figure 8 Schematic diagram of the components of the method according to the present invention is shown. Figure 1 , the scanning acoustic microscope 10 generates a pulse 30, which is switched through the ultrasonic transducer 12 and returned in the form of various echoes 32, 34, 36, and is digitized and analyzed. Possible but non-exclusive functional distributions are based on single signal or context-based functions. For example, regression functionality such as noise suppression (denoising) or upsampling is usually but not necessarily applied to an individual ultrasonic signal. The individual ultrasonic signal may also have been classified by a correspondingly trained neural network.

[0060] The context-based function involves a collective evaluation of the ultrasound signal at each pixel on adjacent scanning points. These neural networks can also be classified by correspondingly trained neural networks. Due to the large amount of input data, the size of the neural network is correspondingly large, but the large number of available signals also ensures that the recognition power is greater than that in the case of individual signal analysis. These more complex signals evaluated together are also very suitable for 2D or 3D reconstruction of the sample, because the unrelated signal noise is suppressed by the combination of the signals of adjacent grating points, so that a clearer reconstructed image can be produced. Of course, reconstruction can also be performed by these individual signals.

[0061] All cited features, including also those features taken from the drawings alone, and individual features disclosed in combination with other features, are considered essential to the invention both individually and in combination. Embodiments according to the invention can be realized by individual features or a combination of several features.

[0062] Reference numerals list

[0063] 10Scanning Acoustic Microscope

[0064] 12 Ultrasonic transducer

[0065] 20 samples

[0066] 22 front surface

[0067] 24 rear surface

[0068] 26 Internal volume

[0069] 30 ultrasonic pulses

[0070] 32Echo from the front surface

[0071] 34Echo from the rear surface

[0072] 36Echoes from the inner volume

[0073] 40 grid

[0074] 50CNN

[0075] 52 Input layer

[0076] 54 folding layers

[0077] 56 Global Average Pooling

[0078] 58 Classifier

[0079] 60K output classification

[0080] 70RNN

[0081] 71 Input terminal

[0082] 72 LSTM layers

[0083] 74Dense layer

[0084] 76Softmax evaluation

[0085] 78 predictions

[0086] 110 Program Selection

[0087] 112 selected programs

[0088] 114A Scanner

[0089] 116 Preprocessing

[0090] 118 Return

[0091] 119 Autoencoder

[0092] 120 categories

Claims

1. A method for automatic defect classification of a sample by scanning acoustic microscopy, in particular, the scanning acoustic microscopy performs automatic defect classification of a sample in a frequency range of 10 MHz to 2000 MHz, wherein: The sample is scanned using a scanning acoustic microscope with one or more ultrasonic transducers, the sample is progressively positioned at raster points relative to the one or more ultrasonic transducers, one or more ultrasonic signals are generated at each raster point, and the one or more ultrasonic signals are recorded after reflection on and / or in the sample and / or after transmission of the sample, wherein the time series of the recorded reflected and / or transmitted ultrasonic signals are digitized and analyzed for defects, in particular for classification, by at least one neural network, which was previously trained unsupervised by a deep learning algorithm in an initial training with the help of scanning acoustic microscope scans of one or more control samples of the same type as the sample or with known and marked defects.

2. The method according to claim 1, characterized in that The one or more ultrasonic transducers are designed as one or more broadband ultrasonic transducers, specifically, the one or more broadband ultrasonic transducers have a bandwidth of at least 10%, specifically at least 20%, wherein the one or more ultrasonic transducers are specifically designed to record signals resulting from mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or due to inherent characteristics of the transducer, and forward the signals to a receiver.

3. The method according to claim 1 or 2, characterized in that: The neural network is a neural convolutional network (CNN), specifically having a 1-D network architecture having one-dimensional convolutional blocks, and the deep learning algorithm specifically includes automated feature extraction, or is a recurrent neural network (RNN), in particular having a Long Short Term Memory (LSTM) based architecture or a Gateway Recurrent Unit (GRU) architecture, and the deep learning algorithm in particular comprises an adaptation of a feed-forward phase of said RNN, or It is a hybrid of the CNN and the RNN.

4. The method according to any one of claims 1 to 3, characterized in that Prior to the analysis, the ultrasound signal is smoothed by a deterministic algorithm, in particular by wavelet filtering, in particular based on the Daubeches wavelet or the Mexican Hat wavelet.

5. The method according to any one of claims 1 to 3, characterized in that The time series of the ultrasonic signal is analyzed as a discrete time series in the form of {signal length x 1}, and / or a volume analysis is performed, wherein several ultrasonic signals of adjacent grating points are combined and analyzed as a discrete time series of grating points, specifically arranged in a rectangle with an edge length of nxm, and several ultrasonic signals of adjacent grating points are combined and analyzed as a data set in the form of {signal length xnxm}, wherein at least one of n and m is ≥2, wherein specifically, n=m, wherein specifically, n and m are odd numbers.

6. The method according to any one of claims 1 to 5, characterized in that The dataset of discretized ultrasound signals is two-dimensional, having the discretized ultrasound signals in a first dimension and a time step of 1 in a second dimension.

7. The method according to any one of claims 1 to 6, characterized in that The classifications include one or more non-defective classifications and one or more defective classifications, wherein the one or more non-defective classifications are specifically one or more classifications of one or more structures, and the one or more defective classifications are specifically different types of defects.

8. The method according to any one of claims 1 to 7, characterized in that In the classification, a confidence level of the respective classification is determined, which confidence level is represented in particular by an intensity value in the representation of each raster point.

9. The method according to any one of claims 1 to 8, characterized in that The classification, and in particular also the confidence level for the classification, is superimposed on the C-scan image as the color or grayscale value of the raster points with the defect classification, wherein in particular the confidence level is integrated as a brightness parameter, a color parameter or a transparency parameter.

10. The method according to any one of claims 1 to 9, characterized in that Before being processed by the neural network, the ultrasound signal is converted by regression, specifically, the ultrasound signal is converted by interpolation or by a trained statistical model to convert the ultrasound signal into a signal of a predefined length and / or a predefined time increment, specifically without losing information about the sample, wherein specifically noise, signal artifacts and / or interfering signals and distortion due to design are reduced.

11. The method according to any one of claims 1 to 10, characterized in that The ultrasound signal can be anomaly corrected by an encoder-decoder architecture, wherein the ultrasound signal is decomposed into components and reconstructed again based on trained domain knowledge, wherein a reconstruction error is derived from the difference between the input signal and the reconstructed signal.

12. The method according to any one of claims 1 to 11, characterized in that A reconstructed 3D dataset of the sample volume of the sample is generated by a trained statistical model taking into account the characteristics of ultrasound propagation in the sample, and the trained statistical model corrects for the effects of defocus, multiple echoes, mode conversion signals and intrinsic transducer signals, allowing cross-sectional images to be generated along any plane.

13. The method according to any one of claims 1 to 12, characterized in that An untrained neural network or a pretrained neural network is used as basis for said initial training with the help of at least one control sample of an alien type of at least one alien sample type.

14. The method according to any one of claims 1 to 13, characterized in that For the initial training of the neural network, defects and other parts of the control sample are identified manually, specifically on a C-scan segment, wherein specifically the manually marked area accounts for less than 50% of the scan area of ​​the control sample, specifically less than 20%, specifically less than 10%.

15. A software program product having program code means which, when executed on a data processing installation of a scanning acoustic microscope, causes the scanning acoustic microscope to carry out the method according to any one of claims 1 to 14.

16. A scanning acoustic microscope, comprising a positioning system, the positioning system comprising a holder for a sample, at least one ultrasonic transducer, a pulse generating and receiving unit, a data processing facility and an analog-to-digital converter unit, the at least one ultrasonic transducer being in particular a broadband ultrasonic transducer, the pulse generating and receiving unit being connected or connectable to the at least one ultrasonic transducer, wherein: The scanning acoustic microscope is designed and arranged for carrying out the method according to any one of claims 1 to 14, in particular by means of a software program product executable and stored in the data processing facility.

Images