Automatic defect classification method in scanning acoustic microscopy inspection, software program product, and scanning acoustic microscope

Abstract

The present invention relates to an automatic defect classification method for a sample in a scanning acoustic microscope, a software program product, and a scanning acoustic microscope. In the method according to the present invention, a sample is scanned with a scanning acoustic microscope, a time sequence of recorded ultrasonic signals that are reflected and / or transmitted is digitized, analyzed for defects, and in particular, classified with respect to defects by at least one neural network pre-trained without a teacher in initial learning by a deep learning algorithm using a scanning acoustic microscope scan of one or more control samples of the same type as the sample or known labeled defects.

Description

Technical Field

[0001] The present invention relates to a method for automatically classifying defects in a sample by scanning acoustic microscopy inspection, a software program product, and a scanning acoustic microscope.

Background Art

[0002] Quality inspection by ultrasonic waves (also called acoustic microscopy inspection) enables non-destructive measurement and inspection of materials and components. When generating an image with a scanning acoustic microscope, the sample to be inspected is scanned line by line, a short ultrasonic pulse is generated at each pixel, and the ultrasonic signal reflected from the sample is evaluated pixel by pixel. Usually, the time sequence of the signal is analyzed within a specified time range (gate). The time range to be analyzed is defined in association with the surface signal of the sample, and it is also possible to image the sample at a depth specified in advance with respect to the surface (surface trigger).

[0003] Generally, the maximum signal intensity or maximum amplitude within the selected time range is represented by the gray scale value of each pixel or each raster point, thereby generating an image of the sample. However, in order to generate this gray scale value, complex data processing operators such as filters and conversions can also be used. The generated micrograph images the sample in a plane orthogonal to the transducer and is called a C-scan. Another image mode is the B-scan. The B-scan represents the acoustic cross-section of the sample. The position coordinates are plotted along the X-axis of the image, and the flight time of the acoustic signal is plotted along the Y-axis.

[0004] When using scanning acoustic microscopy inspection in production management and statistical quality control, the evaluation of C-scan images is usually performed by automatic image processing. In this case, in addition to conventional algorithms such as threshold analysis, morphological filters, and matrix-based image operations, image-based solutions using artificial intelligence are also adopted. The purpose of image processing is to minimize the misevaluation of structures evaluated as false positives and to reliably detect significant defects in components.

[0005] For this purpose, one or more images from ultrasonic inspection are transferred to image processing, and the received images are inspected for various features. The coordinates and characteristics of the detected defects are then transferred to the manufacturing management system. In some cases, a final good / bad evaluation is performed, and defective components are sorted. In particular, in the field of semiconductor manufacturing, the communication standard SECS / GEM is used for connection.

[0006] This method is suitable for traditional semiconductor components such as individual components using plastic connection compounds, DCB-based power electronics using traditional soldering, or wafers joined by fusion bonding. In all of these applications, defects can be displayed with a clear brightness compared to the remaining defect-free areas.

[0007] The methodological limitations of conventional image processing always occur when the pixel luminance values can no longer be clearly assigned to defects and the same gray-scale value can occur in both cases of component defects and non-damaged structures. In such cases, the image processing ability can be improved by identifying the component area and performing only local analysis.

[0008] The analysis of ultrasonic data of complex components is often no longer possible with conventional image processing. In the case of complex image content, image analysis based on machine learning may be used. However, in image-based machine learning, a large amount of data is required for training neural networks. In particular, when training on types of defects with low occurrence frequencies, it is very difficult to obtain a sufficient number of defective components. For example, as a countermeasure, there is a method of pseudo-expanding a small number of typical defects. However, since non-representative data is used for learning, this may lead to misinterpretation of the network. Therefore, the approach using image-based artificial intelligence can only utilize insufficient pixel materials for the training of artificial neural networks, and the available good datasets are not sufficient for comprehensive modeling, so it is very complicated and unreliable in the application of defect recognition in acoustic microscopy of materials and components.

[0009] In contrast, an object of the present invention is to improve automatic defect recognition in non-destructive inspection of complex materials and components, particularly complex semiconductor samples.

Summary of the Invention

[0010] This object is achieved by a method for automatically classifying defects in a sample by means of scanning acoustic microscopy, which comprises scanning the sample with a scanning acoustic microscope comprising one or more ultrasonic transducers, in particular in a frequency range from 10 MHz to 2000 MHz, positioning the sample stepwise relative to the ultrasonic transducer at raster points, generating one or more ultrasonic signals at each raster point, and recording them after reflection on and / or within the sample and / or after transmission through the sample, wherein the temporal sequence of the reflected and / or transmitted and recorded ultrasonic signals is digitized and analyzed, in particular by means of at least one neural network pre-trained without a teacher in an initial learning by means of a deep learning algorithm, by means of a scanning acoustic microscopy scan of one or more control samples of the same type as the sample or using known labeled defects.

[0011] The method according to the invention is applicable to various types of samples, from simple material samples to complex semiconductor samples, and is based on the fact that with a scanning acoustic microscope corresponding to the current state of the art, the acoustic signal can be detected, processed and stored for each scanning point. These temporal sequences are also called A-scan signals. Thereby, instead of the previous image-based analysis, a signal-based or volume-based analysis of the complete temporal signal sequence can be performed.

[0012] In principle, the inevitable data reduction problem in previous image-based analysis, which occurs when the detected complex ultrasonic signal is reduced to a single value, such as the maximum value of the amplitude in the considered time frame, during the generation of the image for each pixel, i.e. each raster point, is avoided here. This reduction makes the comprehensive information contained in the ultrasonic signal unavailable for the analysis. This especially leads to the fact that, particularly in the case of a high amplification factor, the noise of a single grayscale value reaches the limit value compared to the threshold value required in conventional image analysis, which has an adverse effect on the certainty of defect determination.

[0013] The ultrasonic signals of complex semiconductor samples often contain not only the signal part directly corresponding to the coupled compression wave, but also the signal parts generated by other wave modes. In high-speed materials, so-called mode conversion may occur. In this case, the coupled compression wave generates both a compression wave part and a shear wave part at the interface, and since both move at different propagation speeds within the sample, a time-shifted echo signal is generated. When two interfaces with large reflection coefficients face each other, multiple echoes are formed as the sound wave reflects back and forth multiple times. A transducer with a large opening angle may also generate Rayleigh waves and Lamb waves inside the solid. All of these signals can be used to determine the presence or absence of defects. Therefore, weakly developed defects generate characteristic signatures at many different positions in the signal. Since these signatures can use different parts of the signal for analysis, they provide a defect discrimination database that is superior to a single grayscale value.

[0014] Using an artificial neural network that can evaluate and classify the overall signal of the interface enhances the reliability of the network's determination in data analysis and surpasses image-based methods. In the case of labeling image data required for learning an artificial neural network for image data analysis, the availability of data is often restricted or a great deal of human labor is required, whereas this method focuses on the classification of individual A-scan signals. For this purpose, it is sufficient to train an artificial neural network with multiple A-scan signals from representative relevant regions. Even if the number of such signals is in the four digits, the ratio is small compared to the total data volume for a complete C-scan image.

[0015] The method according to the present invention is directed to training a network with signals from a related area, whereby the network can derive information and verify the usefulness of the complete information of the signals for the learning of a deep learning network. Deep learning networks are very efficient in automated feature extraction and often achieve better results than can be achieved in the case of manual selection. With correct optimization and continuous learning, deep learning networks can draw more conclusions than human operators. The 3D data detected by a scanning acoustic microscope is used as raw data for training a statistical model. These data make it possible to reconstruct all individual gate settings such as A-scans, C-scans, B-scans, slices and 3D scans (including amplitude and time-of-flight data), and to display the sample in individual layers.

[0016] Conventional methods rely in their applicability on a clean signal of an ultrasonic transducer and become inapplicable in the case of broadband ultrasonic transducers due to many signals of different origins. In an embodiment of the method according to the present invention, the ultrasonic transducer is designed as a broadband ultrasonic transducer having a bandwidth of at least 10%, in particular at least 20%, and the ultrasonic transducer is designed in particular to record signals generated due to mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or the intrinsic properties of the transducer and transfer them to a receiver.

[0017] The method according to the present invention uses a neural network trained by supervised learning or unsupervised learning. In the first case, one reference sample or at least one defective reference sample is scanned. The defect is visible to an expert or is known from other inspections. Next, the expert marks one or more regions of the reference sample without defects and one or more defective regions. By learning the neural network, it is prepared to assign the corresponding ultrasonic signal to one of the categories. In the case of deep learning, which signal structure is involved is not preset, and the neural network finds the most meaningful signal structure by itself.

[0018] In the case of unsupervised learning, a reference sample without defects is scanned. The ultrasonic signal of this reference sample indicates to the neural network trained there what the ultrasonic signal should ideally be like. During the learning phase, the neural network learns the entire range of signals allowed for different structures of the reference sample. This can be a number of different structures and the corresponding signal sequences that could not be imaged by conventional methods. Then, when a defective sample is scanned, the ultrasonic signals at the positions of the defects are not within the spectrum of the allowed signals and are different from them. By using criteria that describe the deviation from the ideal signal image, the presence or absence of defects can be evaluated.

[0019] In this context, it is advisable to train the neural network with ultrasonic signals recorded with the same parameters as the signals to be classified later. This relates, inter alia, to the bandwidth and transmission characteristics of the ultrasonic transducer and the signal frequency of the ultrasonic pulse.

[0020] In an embodiment, the ultrasonic transducer is designed as a wideband ultrasonic transducer having a bandwidth of at least 10%, particularly at least 20%, and the ultrasonic transducer is designed to record signals generated particularly due to mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or the inherent characteristics of the transducer and transfer them to a receiver. The ultrasonic signal transmitted by the wideband ultrasonic transducer contains much more information than the ultrasonic transducers that are normally used and have a small bandwidth or operate with a reduced bandwidth due to its high bandwidth. When using a wideband ultrasonic transducer, due to this information density, it has not been practical so far to apply a corresponding image analysis based on a neural network to the data generated using the wideband ultrasonic transducer. On the other hand, in the method according to the present invention, the additional available information is utilized to improve the separation degree of classification. Also, due to the high information density of the wideband signal, the neural network can be trained with one or a very small number of scans.

[0021] In an embodiment, the neural network is a convolutional neural network (CNN), particularly a 1D-resnet architecture having a one-dimensional convolutional block, and particularly includes automated feature extraction by a deep learning algorithm. Alternatively, the neural network is a recurrent neural network (RNN) having an architecture based particularly on LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit), and particularly the deep learning algorithm includes adaptation of the feed-forward stage of the RNN. The neural network may be designed as a hybrid of CNN and RNN.

[0022] In an embodiment, the ultrasonic signal is smoothed by a deterministic algorithm, in particular, by wavelet filtering, in particular, based on a Daubechies wavelet or a Mexican hat wavelet, prior to the analysis. Also, the ultrasonic signals used for the learning of the neural network should also be subjected to the same preprocessing in this case. The smoothing may help to remove signal artifacts that may occur due to the design of the device.

[0023] In an embodiment, the time sequence of the ultrasonic signal is analyzed as a discrete time series in the form of {signal length × 1}. By digitization, the discrete time series includes the length of thousands of data points and becomes a vector of the same length. In this case, each raster point or each pixel is individually analyzed as to whether it belongs to a defect. Additionally or alternatively, a volumetric analysis may be performed in which a plurality of ultrasonic signals of adjacent raster points are combined and analyzed as a discrete time series of raster points, in particular, as a data set in the form of {signal length × n × m}, arranged within a rectangle with side lengths n × m, where at least one of n and m is 2 or more, in particular n = m, and in particular n and m are odd numbers. This includes combinations row by row or column by column of a plurality of adjacent pixels or combinations of pixels in a rectangular or square arrangement. For example, in the case of a square arrangement where an odd number of pixels is selected as the side length, the center of the square coincides with the central pixel. In the case of a combination of adjacent pixels, regardless of whether it is linear or planar, the central pixel is weighted more than the surrounding pixels, and the analysis or learning of the neural network can be adapted.

[0024] In an embodiment, the processing performance is improved when the data set of the discretized ultrasonic signal is two-dimensional, the discretized ultrasonic signal is in the first dimension, and 1 time step is in the second dimension. The actual value of the time step depends on the digitization rate of the analog ultrasonic signal.

[0025] In an embodiment, the classification includes one or more classes for the absence of defects, in particular one or more classes related to one or more structures, and one or more classes for defects, in particular classes of different types of defects.

[0026] In an embodiment, during the classification, a confidence level for each classification is determined, in particular expressed as an intensity value for each raster point.

[0027] In particular, for a systematic analysis of defects in the manufacture of a sample, in an embodiment, for raster points having a defect classification, the classification and in particular the confidence level therefor are also superimposed on a C-scan image as color or grayscale values. In particular, the confidence level is integrated as a luminance parameter, a color parameter, or a transparency parameter.

[0028] In an embodiment, before the processing by the neural network, the ultrasonic signal is converted by regression, in particular by interpolation, or by a learned statistical model, in particular without loss of information about the sample, into a signal of a predetermined length and / or a predetermined time increment, and in particular noise, signal artifacts, and / or interference signals and distortions due to the design are reduced. According to the present invention, the loss of information is understood to be the loss of information indicating the sample, and the interference signals and distortions are inherent in the system and do not include any information about the sample itself. Ideally, the training data of the reference sample is recorded under the same circumstances and parameters as the ultrasonic signal of the sample to be tested later. However, in practice, since all test parameters cannot be repeated precisely, first, the system responses for different situations are recorded and analyzed, and it may be useful to remove parts of the system responses known during the preprocessing of the ultrasonic signal before learning and analysis, such as noise, signal artifacts, interference signals, and distortions.

[0029] Additionally or alternatively, the ultrasonic signal may be corrected for anomalies by an encoder-decoder architecture, where the ultrasonic signal is decomposed into multiple components, reconstructed based on learned domain knowledge, and the reconstruction error is derived from the difference between the input signal and the reconstructed signal. The encoder-decoder architecture converts the received ultrasonic signal into a good ultrasonic signal and is an example of regression. For this purpose, first, a statistical model, such as a neural network, is trained to convert a good signal into a good signal. This is a case of unsupervised learning, generating the aforementioned domain knowledge and incorporating it into the neural network. An abnormal signal deviating from the good signal due to the presence of defects is sequentially converted by the encoder-decoder architecture into the previously learned good signal, thereby correcting the anomalies. The difference between the output signal and the input signal forms the basis for defect classification and can be represented, for example, by a threshold of the degree of deviation.

[0030] In an embodiment, the reconstructed 3D dataset of the sample volume of the sample is generated by a learned statistical model that takes into account the characteristics of ultrasonic propagation within the sample, corrected for the effects of defocus, multiple echoes, mode conversion signals, and transducer-specific signals, enabling the generation of cross-sectional images along any plane. Since these effects can be modeled and back-calculated from the ultrasonic signal, they can be removed.

[0031] In an embodiment, as a basis for the initial learning, an unlearned neural network or a neural network pre-trained using at least one control sample of at least one foreign sample type among at least one foreign sample type is used. In the former case, since the neural network is trained from an initial state, learning takes a somewhat longer time. However, in the latter case, there is already available information, and it is only necessary to adapt it to the current control sample. Thereby, depending on the similarity to the previously learned sample type, the learning converges relatively quickly. After the learning of the current sample type is completed, the current version of the neural network can be saved for later use with the same sample type, thereby setting up a database for different prototype neural networks without having to train the neural network anew each time.

[0032] In an embodiment, for the initial learning of the neural network, the defect and other part labeling of the control sample are performed manually, particularly on the C-scan cross-section. In particular, the manually marked area occupies less than 50%, particularly less than 20%, and particularly less than 10% of the scanning area of the control sample. For this reason, in the method according to the present invention, since the complete information content of the ultrasonic signal of each pixel is used, the starting material can be significantly reduced. Therefore, the limitation to a relatively small area of the control sample is sufficient.

[0033] The object of the present invention is also achieved by a software program product comprising program code means for causing the scanning acoustic microscope to execute the method according to the present invention described above when executed on a data processing device of the scanning acoustic microscope. Alternatively, the learning of the neural network and the analysis of the ultrasonic signal may be performed on another data processing device.

[0034] Furthermore, the object of the present invention is also achieved by a scanning acoustic microscope comprising a positioning system with a holder for a sample, at least one ultrasonic transducer, in particular a broadband ultrasonic transducer, a pulse generation and reception unit connected or connectable to the at least one ultrasonic transducer, a data processing device, and an analog-to-digital conversion unit, which is designed and configured to implement the above-described method according to the present invention by means of the above-described software program product executable and stored in the data processing device.

[0035] The software program product and the scanning acoustic microscope achieve the same advantages, characteristics and features as the above-described method according to the present invention.

[0036] Further features of the present invention will become apparent from the description of embodiments according to the present invention, the claims and the accompanying drawings. Embodiments according to the present invention may involve individual features or combinations of a plurality of features.

[0037] In the context of the present invention, features specified as "in particular" or "preferably" should be understood as optional features.

Brief Description of the Drawings

[0038] Hereinafter, exemplary embodiments will be described with reference to the drawings without limiting the general inventive concept, and any details regarding the invention not further described in the text will be explicitly referred to in the drawings. They are shown below.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0039] In the drawings, since the same or similar types of elements and / or components are assigned the same reference numerals, the repeated description of each corresponding location is omitted.

[0040] FIG. 1 shows an overview of signal formation in scanning acoustic microscopy. As shown in the right half of the figure, an ultrasonic pulse 30, which typically contains frequencies between 10 MHz and 2000 MHz and is generated by a pulse generator (not shown), for example, a piezoelectric crystal, is irradiated onto a sample 20 to be inspected by an ultrasonic transducer 12. At the front surface 22 of the sample 20, a part of the pulse 30 is reflected as an echo 32, and another part of the pulse is reflected as an echo 34 at the back surface 24. In between, another part of the pulse 30 may be reflected as an echo 36 by the internal structure and / or defects of the internal volume 26 of the sample. The return echoes 32, 34, 36 are sequentially guided by the ultrasonic transducer 12 to a receiver (not shown). This may be the same piezoelectric crystal that generated the pulse 30. The corresponding scanning acoustic microscope is known.

[0041] In the left half of the figure, the time sequence of the amplitudes of the reflected signals is shown, which is a so-called A-scan. Corresponding to a short propagation time, first the echo 32 returns from the front surface. Also, this echo has the maximum amplitude. Next, a considerably weak echo 36 from inside the sample follows, and finally the echo 34 of the pulse returns from the back surface 26. This signal sequence is also shown schematically in the vertical direction corresponding to the depth cross-section of the sample 20.

[0042] In the analysis of the sample regarding the defect, the middle part of the signal is the part that attracts attention. Therefore, a time gate 40 is applied to exclude the echoes 32, 34 of the pulses 30 from the front surface 22 and the back surface 24. The signals located within the gate 40 contain various information regarding the internal state of the sample 20 at the current raster point or pixel.

[0043] In FIG. 2, a one-dimensional CNN architecture of a convolutional neural network (CNN) is schematically shown. The discrete time sequence of the amplitude signal at the gate 40 (“input time series”) is introduced into the input layer 52 and functions as an input signal. This is followed in order by a convolutional plane 54, which is a so-called convolution base, an averaging layer 56 (“global average pooling”), and a classifier 58 having one or more fully connected layers. The classifier 58 outputs a number of K initial classifications 60 indicating whether the sample 20 contains a defect at the current position and, if so, what kind of defect or what kind of structure it is.

[0044] FIG. 3 shows an alternative to the recurrent neural network 70, including a series of LSTM layers 72 on the input 71 side, followed by a fully connected layer 74. The output of the fully connected layer 74 is processed, for example, by a softmax function 76 to derive a prediction 78 corresponding to the classification of the CNN.

[0045] In FIG. 4, an example of unsupervised deep learning is schematically shown. The scanning acoustic microscope 10 performs a scan of a defect-free control sample 20 at each raster point (“single A-scan approach”). Since there are no defects in the control sample 20, the temporal sequence serves as the gold standard. Next, the neural network is trained with all the data of the A-scans by a deep learning model. In this way, the neural network learns the characteristics of the sequence of A-scans of all the structures included in the control sample and is set to optimally recognize these signals. Since the corresponding procedure is known, the trained neural network has knowledge about what signals are predicted for a proper sample. Subsequently, therefore, the A-scan signals from the locations with defects can be noticed and are capable of being identified and marked. This can be confirmed in the partial image on the right, where a defect is shown on the flip-chip sample.

[0046] In FIG. 5, a different method from that in FIG. 4 is shown, where the A-scan signals are pre-processed, for example, by wavelet filtering or upscaling, before learning or analysis.

[0047] Supervised learning may be performed instead of unsupervised learning. In this case, instead of a defect-free control sample, a control sample with defects is scanned. Next, experts or other systems used for defect recognition then continue with a labeling step of marking, where applicable, characteristic regions of different defects and, on the other hand, characteristic regions of different defect-free structures of the sample on the control sample. Subsequently, the neural network is trained with the aim of reproducing the corresponding defect or structure classification as accurately as possible. The procedure for supervised learning is also known.

[0048] In FIG. 6, for some of the aforementioned classifications, the temporal ultrasonic signal sequences within gate 40 are shown. In this example, the sample is a defective flip chip and corresponds to the respective right-hand representations in FIGS. 4 and 5. The gate start is indicated by a vertical line in each case. From top to bottom, these are the signals from the defective bright solder joint, the signals from the defective dark solder joint, the signals from the normal solder joint, and the signals from other structural regions. To the naked eye, hardly any differences can be recognized. The neural network learns during training which signal components or structures are characteristic of the various classifications.

[0049] FIG. 7 shows an overview of a general workflow in data analysis by machine learning. The scanning acoustic microscope 10 records the scan of the sample, i.e., the temporal sequence of ultrasonic signals at each scanning point on the sample. Subsequently, a procedure selection 110 is performed and an analysis procedure 112 is selected. For example, each pixel can be analyzed individually or, for example, a neighborhood evaluation of a square region having a side length of, for example, 3 pixels or 5 pixels can be performed. A large number of adjacent pixels in a row or column or a non-square continuous pixel region may, if applicable, be evaluated together with a weighting that slopes from the center of the region towards the edge.

[0050] After the selection of procedure 112, the amount of the recorded signals is subjected to evaluation. This may be done online in real time, but also downstream and, if applicable, offline. In this case, the recorded ultrasonic signals may be subjected to preprocessing 116 such as, for example, upsampling, downsampling and / or filtering, in particular wavelet filtering. Regression 118 may be performed, for example, using an autoencoder 119. As an example of this, there is a neural network trained by unsupervised learning. Classification 120 may also be performed using a CNN 50 or an RNN 70 trained by supervised learning.

[0051] Figure 8 shows a schematic diagram of the components of the method according to the present invention. Referring to the remaining figures, particularly FIG. 1, the scanning acoustic microscope 10 generates a pulse 30, which is switched by the ultrasonic transducer 12 and returns in the form of various echoes 32, 34, 36 and is digitized and analyzed. The distribution of functions, which may but is not limited to this, is a single-signal-based or context-based function. For example, regression functions such as noise suppression (denoising) and upsampling are usually, but not always, applied to individual ultrasonic signals. The individual ultrasonic signals may already be classified by the corresponding trained neural network.

[0052] Context-based functions relate to the comprehensive evaluation of the ultrasonic signals of adjacent scanning points or pixels. These may be classified by the corresponding trained neural network. Since the input data increases, the neural network grows accordingly, but the increase in the number of available signals also increases the possibility of obtaining a higher discrimination ability than in the case of individual signal analysis. By combining the signals of adjacent raster points, uncorrelated signal noise is suppressed and a sharper reconstructed image is generated. Therefore, comprehensively evaluating these more complex signals is also very suitable for two-dimensional or three-dimensional reconstruction of the sample. Of course, reconstruction may also be performed using individual signals.

[0053] All the features cited, including those obtained only from the drawings, and the individual features disclosed in combination with other features, are considered essential to the invention, either alone or in combination. Embodiments according to the present invention can be realized by individual features or combinations of a plurality of features.

Explanation of Reference Numerals

[0054] 10 Scanning acoustic microscope 12 Ultrasonic transducer 20 Sample 22 Front surface 24 Rear surface 26 Internal volume 30 Ultrasonic pulse 32 Echo from the front 34 Echo from the back 36 Echo from the internal volume 40 Gate 50 CNN 52 Input layer 54 Folding layer 56 Global average pooling 58 Classifier 60 K-output classification 70 RNN 71 Input 72 LSTM layer 74 Fully connected layer 76 Softmax evaluation 78 Prediction 110 Procedure selection 112 Selected procedure 114 A-scan 116 Preprocessing 118 Regression 119 Autoencoder 120 Classification

Claims

Claim 1 A method for automatically classifying defects in a sample by scanning acoustic microscopy, particularly in a frequency range from 10 MHz to 2000 MHz, comprising scanning a sample with a scanning acoustic microscope equipped with one or more ultrasonic transducers, positioning the sample stepwise with respect to the ultrasonic transducer at raster points, generating one or more ultrasonic signals at each raster point, and recording after reflection and / or transmission in and / or on the sample, the method comprising: digitizing and analyzing the temporal sequence of the reflected and / or transmitted and recorded ultrasonic signals, and classifying with respect to the defects by at least one neural network pre-trained without a teacher in initial learning by a deep learning algorithm, particularly by a scanning acoustic microscopy scan of one or more control samples of the same type as the sample or using known labeled defects. Claim 2 The ultrasonic transducer is designed as a broadband ultrasonic transducer having a bandwidth of at least 10%, particularly at least 20%. The method according to claim 1, wherein the ultrasonic transducer is designed to record signals generated, in particular, due to mode conversion, multiple echoes, Rayleigh waves, Lamb waves and / or the inherent properties of the transducer and transfer them to a receiver. Claim 3 The neural network is a convolutional neural network (CNN), in particular a 1D-resnet architecture having a one-dimensional convolutional block, in particular, the deep learning algorithm includes automated feature extraction, or a recurrent neural network (RNN) having an architecture based on LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit), in particular, the deep learning algorithm includes adaptation of the feed-forward stage of the RNN, or a hybrid of CNN and RNN according to claim 1 or 2. Claim 4 The method according to any one of claims 1 to 3, wherein the ultrasonic signal is smoothed by a deterministic algorithm, in particular by wavelet filtering, in particular based on a Daubechies wavelet or a Mexican hat wavelet, prior to the analysis.

5. The method according to any one of claims 1 to 3, wherein the time sequence of the ultrasonic signal is analyzed as a discrete time series in the form of {signal length × 1}, and / or a plurality of ultrasonic signals of adjacent raster points are combined and volumetric analysis is performed by analyzing them as a discrete time series of raster points, in particular as a data set in the form of {signal length × n × m}, arranged within a rectangle with side lengths n × m, at least one of n and m being 2 or more, in particular n = m, and in particular n and m being odd numbers.

6. The method according to any one of claims 1 to 5, wherein the data set of the discretized ultrasonic signal is two-dimensional, the discretized ultrasonic signal is in the first dimension, and one time step is in the second dimension.

7. The method according to any one of claims 1 to 6, wherein the classification includes one or more classes for the absence of defects, in particular one or more classes for one or more structures, one or more classes for defects, in particular classes for different types of defects.

8. The method according to any one of claims 1 to 7, wherein during the classification, a confidence level for each classification is determined, in particular expressed as an intensity value for each raster point.

9. For raster points having defect classifications, the classification and in particular the confidence level therefor are also superimposed on a C-scan image as color or grayscale values, in particular the confidence level is integrated as a luminance parameter, a color parameter or a transparency parameter. The method according to any one of claims 1 to 8.

10. Before the processing by the neural network, the ultrasonic signal is converted into a signal of a predetermined length and / or a predetermined time increment by regression, in particular by interpolation, or by a learned statistical model, in particular without loss of information regarding the sample, and in particular noise, signal artifacts and / or interference signals and distortions due to the design are reduced. The method according to any one of claims 1 to 9.

11. The ultrasonic signal can be corrected for anomalies by an encoder-decoder architecture, the ultrasonic signal is decomposed into a plurality of components, reconstructed based on learned domain knowledge, and a reconstruction error is derived from the difference between the input signal and the reconstructed signal. The method according to any one of claims 1 to 10.

12. The reconstructed 3D data set of the sample volume of the sample is generated by a learned statistical model that takes into account the characteristics of ultrasonic propagation in the sample, corrected for the effects of defocus, multiple echoes, mode conversion signals, and transducer-specific signals, enabling the generation of cross-sectional images along any plane. The method according to any one of claims 1 to 11.

13. As a basis for the initial learning, an unlearned neural network or a neural network pre-learned using at least one control sample of at least one heterogeneous sample type is used. The method according to any one of claims 1 to 12.

14. For the initial learning of the neural network, the labeling of the defects and other parts of the control sample is performed manually, especially on the C-scan cross-section, and in particular, the manually marked areas occupy less than 50%, especially less than 20%, and particularly less than 10% of the scanning area of the control sample. The method according to any one of claims 1 to 13.

15. A software program product comprising program code means for causing a scanning acoustic microscope to execute the method according to any one of claims 1 to 14 when executed on a data processing device of the scanning acoustic microscope.

16. A scanning acoustic microscope comprising a positioning system with a holder for the sample, at least one ultrasonic transducer, in particular a broadband ultrasonic transducer, a pulse generation and reception unit connected or connectable to the at least one ultrasonic transducer, a data processing device, and an analog-digital conversion unit, and in particular designed and arranged to perform the method according to any one of claims 1 to 14 by a software program product executable and stored in the data processing device.

Images