4D-STEM material defect automatic detection method and system
By employing a 4D-STEM material defect automated detection method, utilizing a teacher-student neural network architecture and self-supervised learning, the challenges of manual dependence and high-dimensional data processing in crystal defect analysis are solved, achieving efficient and accurate defect detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2025-12-24
- Publication Date
- 2026-05-05
AI Technical Summary
Existing technologies rely on human experience in crystal defect analysis, which is inefficient. Furthermore, traditional physical algorithms struggle to handle high-dimensional data and have poor noise resistance, making it difficult to effectively identify complex defect regions.
An automated defect detection method for 4D-STEM materials is adopted, which utilizes a teacher-student neural network architecture. The student network learns the physical prior knowledge of the teacher network, and combines self-supervised learning and graph neural network to process lattice point sets and image contrast features to achieve automated defect detection.
It has achieved automation and objectification of crystal defect analysis, improved analysis efficiency, lowered the professional threshold for operators, enhanced the model's generalization ability and defect identification accuracy under different conditions, and overcome the limitations of traditional algorithms in high-dimensional data processing.
Smart Images
Figure CN121978145A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of defect detection technology, and in particular to an automated method and system for detecting defects in 4D-STEM materials. Background Technology
[0002] The acquisition and analysis of crystal defect information mainly rely on two existing technical solutions: The first is based on dual-beam imaging and manual analysis using traditional electron microscopy (TEM). A specific reciprocal lattice vector g is manually selected using TEM / STEM equipment to generate an image under dual-beam imaging conditions. Then, based on the g∙b=0 invisible criterion, where b is the Burgers vector of dislocations, the presence, type, and Burgers vector of dislocations are manually determined by comparing the imaging contrast changes under multiple g vectors. This solution relies on expert experience to conduct multiple experimental operations and image comparisons, and can only be analyzed under specific g vectors. It has disadvantages such as low efficiency, strong subjectivity, the need for a deep crystallography background, the time-consuming selection of g vectors and imaging conditions, the accuracy of the results depending on experience and data loss, the discarding of diffraction information at other angles after selecting dual-beam conditions, and the ambiguity caused by insufficient information in complex or high-density defect regions.
[0003] The second approach is strain analysis based on traditional geometric phase analysis (GPA) or central moment CoM. This method uses 4D-STEM or HRTEM data to obtain the strain and displacement fields through Fourier transform or by calculating the position shift of diffraction spots. Then, it analyzes the distortion characteristics of the strain field to indirectly locate the dislocation core. This approach is purely physical calculation, directly extracting low-dimensional physical quantities from the data and then locating them through post-processing. It suffers from insufficient accuracy and robustness, is sensitive to noise, sample thickness variations, and local tilt, is prone to producing spurious strains or artifacts near the dislocation core, lacks long-range tracking capability, focuses only on the local strain field, and has difficulty identifying and tracking continuous dislocation lines or stacking faults across the field of view. The segmentation results are prone to breakage and discontinuity, and computational resources are wasted. 4D-STEM data has high dimensionality, and using only traditional physical algorithms will ignore a large amount of implicit nonlinear defect feature information. Summary of the Invention
[0004] In view of this, the purpose of this invention is to propose an automated detection method and system for 4D-STEM material defects, so as to solve the problems of existing crystal defect analysis relying on human experience, low efficiency, great influence of human factors, and the difficulty of traditional physical algorithms in handling high-dimensional data and poor noise resistance.
[0005] To achieve the above objectives, this invention provides an automated method for detecting defects in 4D-STEM materials, comprising the following steps:
[0006] Step S1: Obtain 4D-STEM data from material sample scanning and perform structured processing to construct a four-dimensional data cube containing information on the correlation between scanning position and diffraction pattern. Based on the four-dimensional data cube, generate low-dimensional latent feature vectors input by the student network and privileged modal data input by the teacher network.
[0007] Step S2: Construct an asymmetric teacher-student neural network, wherein the teacher network processes the privileged modality data to obtain physical semantic features, and the student network processes the low-dimensional latent feature vector to obtain feature representations;
[0008] Step S3: Using the low-dimensional latent feature vectors and privileged modal data generated in step S1, train the student network with a composite loss function. During the training process, the teacher network parameters are updated synchronously through momentum updates. The teacher network does not backpropagate, so that the student network learns the physical prior knowledge of the teacher network and aligns with the feature space.
[0009] Step S4: Remove the teacher network, connect the decoder and segmentation head after the encoder of the trained student network, and use labeled data to perform supervised fine-tuning of the student network;
[0010] Step S5: Deploy the fine-tuned student network, input the low-dimensional latent feature vector obtained after preprocessing the 4D-STEM data to be detected, and output the material defect detection results.
[0011] Preferably, in step S1, the low-dimensional latent feature vector is obtained by encoding the original diffraction pattern using a pre-trained variational autoencoder with frozen weights.
[0012] Preferably, in step S1, the process of generating the privileged modal data input by the teacher network includes:
[0013] By computationally processing 4D-STEM data, a simulated virtual dark-field image under dual-beam conditions is generated;
[0014] Index analysis of the diffraction pattern is performed to extract the strongest N Bragg diffraction peaks to construct a Bragg point set. Each point in the Bragg point set contains 6 physical descriptors: reciprocal space coordinates qx, qy, Miller indices h, k, l, and intensity I.
[0015] Preferably, the teacher network processes the privileged modality data to obtain physical semantic features, including:
[0016] The extracted Bragg point set tensor is flattened and projected onto the model dimension through a linear layer to obtain the point set features;
[0017] The virtual dark field image is projected onto the model dimension through a convolutional layer to obtain image features;
[0018] The point set features are concatenated with the image features and fused through a linear layer and the GeLU activation function. The result is then fed into the teacher's Transformer encoder to obtain a feature representation containing rich physical semantics.
[0019] Preferably, the composite loss function includes distillation loss, difference entropy loss, and mask reconstruction loss;
[0020] The distillation loss is calculated to determine the cross-entropy loss between the CLS tokens output by the teacher network and the student network, aligning the feature distributions of the student network and the teacher network.
[0021] The differential entropy loss calculates and maximizes the differential entropy of the student network output features, which is used to maximize the distance between features of different samples within the same batch and prevent feature collapse.
[0022] The mask reconstruction loss is used to randomly mask the input of the student network, requiring the network to reconstruct the occluded diffraction pattern features.
[0023] Preferably, in step S3, the momentum update updates the teacher network parameters through an exponential moving average.
[0024] Preferably, in step S5, the output material defect detection results include dislocation segmentation results and Burgers vector type.
[0025] This invention also provides an automated 4D-STEM material defect detection system, comprising:
[0026] The data acquisition and preprocessing module is used to acquire 4D-STEM data obtained from material sample scanning and perform structured processing to construct a four-dimensional data cube containing information on the correlation between scanning position and diffraction pattern. Based on the four-dimensional data cube, low-dimensional latent feature vectors input by the student network and privileged modal data input by the teacher network are generated respectively.
[0027] The network construction module is used to construct an asymmetric teacher-student neural network, wherein the teacher network processes the privileged modality data to obtain physical semantic features, and the student network processes the low-dimensional latent feature vector to obtain feature representations;
[0028] The network training module uses the low-dimensional latent feature vectors and privileged modal data generated in step S1 to train the student network with a composite loss function. During the training process, the teacher network parameters are updated synchronously through momentum updates. The teacher network does not backpropagate, so that the student network learns the physical prior knowledge of the teacher network and aligns with the feature space.
[0029] The network fine-tuning module removes the teacher network and connects a decoder and segmentation head after the encoder of the trained student network, using labeled data to supervise and fine-tune the student network.
[0030] The deployment and detection module is used to deploy the fine-tuned student network, input the low-dimensional latent feature vector obtained after preprocessing the 4D-STEM data to be detected, and output the material defect detection results.
[0031] The beneficial effects of this invention are:
[0032] 1. This invention automates and objectifies crystal defect analysis, significantly improving analysis efficiency and lowering the professional threshold for operators. Existing technologies rely on manual operation of transmission electron microscopes (TEM) to select specific two-beam conditions and perform image-by-image comparative analysis based on experience. This invention employs a "Teacher-Student" knowledge distillation architecture, where the teacher network integrates expert experience (two-beam images) and physical laws, guiding the student network through soft labels. After training, the student network can directly input raw 4D-STEM data and output defect information. This design internalizes complex physical analysis logic into the model. In practical applications, users do not need a deep crystallography background or to repeatedly adjust the TEM optical path to find two-beam conditions; they only need to acquire 4D-STEM data once, and the model can automatically output expert-level defect segmentation results within seconds. This greatly shortens the analysis cycle, eliminates errors caused by subjective human judgment, and ensures the consistency and objectivity of the results.
[0033] 2. This invention enhances the model's generalization ability by utilizing unlabeled data, reducing its reliance on large amounts of high-quality labeled data. The invention incorporates an auxiliary reconstructing head and a corresponding reconstructing loss function in the student network, requiring the network to recover or reconstruct the original 4D-STEM data features while learning defect segmentation. A self-supervised learning mechanism is introduced. Even in the absence of extensive manual fine-grained annotations (Ground Truth), the model can gain a deep understanding of the data's intrinsic physical distribution and background noise patterns through the reconstruction task. This not only prevents overfitting on a small number of samples but also significantly improves the model's generalization ability under different experimental conditions (different accelerating voltages, different sample thicknesses).
[0034] 3. This invention overcomes the limitations of traditional algorithms in high-dimensional data processing, significantly improving the accuracy and robustness of defect identification in complex environments. Existing technologies (such as GPA or CoM) typically only utilize the dimensionality-reduced 2D strain field or centroid displacement for analysis, losing a large amount of high-dimensional diffraction information. This invention innovatively introduces a Graph Neural Network (GNN) encoder into the teacher network to process the lattice point set (x, y, h, k, l, i), and deeply fuses the extracted lattice topological features with image contrast features in the feature space. GNN can directly understand the geometric topological relationships (such as vector angles and distances) of diffraction spots in reciprocal space, enabling the model to keenly capture minute lattice distortions caused by dislocations. Compared to the traditional GPA method, which is easily affected by noise, this multimodal fusion feature has stronger robustness to noise and thickness variations, thus achieving higher accuracy in low signal-to-noise ratio or complex defect (such as dislocation pile-up) regions. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is a flowchart of the 4D-STEM material defect automated detection method according to an embodiment of the present invention;
[0037] Figure 2 This is a framework diagram of the 4D-STEM material defect automated detection system according to an embodiment of the present invention;
[0038] Figure 3 This is a reconstruction result diagram after feature encoding using the VAE model in an embodiment of the present invention;
[0039] Figure 4 This is a real-space reconstruction image of a trained VAE after feature encoding, according to an embodiment of the present invention.
[0040] Figure 5 Label diagrams for three types of defects in embodiments of the present invention;
[0041] Figure 6 This is a diagram illustrating the effect of an embodiment of the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.
[0043] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0044] Example 1:
[0045] like Figure 1 As shown, this embodiment proposes an automated defect detection method for 4D-STEM (four-dimensional scanning transmission electron microscopy) materials based on cross-modal knowledge distillation. This method constructs an asymmetric "teacher-student" neural network architecture, such as... Figure 2 As shown, the Teacher Network uses "privileged modes" (i.e., the Bragg diffraction point set resolved from the diffraction pattern and the generated double-beam virtual dark-field image) with clear physical meaning but difficult to obtain on a large scale as input to extract physical features; the Student Network only uses the latent features of the original 4D-STEM diffraction pattern as input. By introducing DINO (self-distillation) loss, KoLeo (differential entropy) loss, and mask reconstruction loss during the pre-training stage, the Student Network is forced to learn the physical prior knowledge in the Teacher Network and align with the physical feature space. During the inference stage, only the trained Student Network is needed to achieve high-throughput, high-precision detection and segmentation of defects such as dislocations. The specific steps are as follows:
[0046] Step S1: Use a 4D-STEM device to perform grating scanning on the material sample, collect two-dimensional diffraction patterns (CBED patterns) at each scanning position, perform structured processing on the 4D-STEM data obtained from the scanning, and construct a four-dimensional data cube containing information on the correlation between the scanning position and the diffraction pattern.
[0047] To reduce computational complexity, a pre-trained variational autoencoder (VAE) with frozen weights is used to encode the original 4D-STEM diffraction pattern, compressing it into a low-dimensional latent vector, which serves as the input to the student network.
[0048] The generation of privileged modal data for teachers involves two parts: First, by processing 4D-STEM data computationally, virtual dark field (VDF) images under simulated two-beam conditions are generated. These images can visually reflect dislocation morphology. Second, index analysis is performed on the diffraction patterns to extract the strongest N Bragg diffraction peaks and construct a Bragg point set. Each point in the Bragg point set contains six physical descriptors: reciprocal space coordinates (qx, qy), Miller indices (h, k, l), and intensity I.
[0049] Step S2: Construct an asymmetric teacher-student neural network. The teacher network processes the privileged modality data to obtain physical semantic features. Specifically, the teacher network includes three parts: a point set branch, an image branch, and feature fusion. The point set branch flattens the extracted Bragg point set tensor and projects it onto the model dimension through a linear layer to obtain point set features. The image branch projects the virtual dark field image onto the model dimension through a convolutional layer to obtain image features. The feature fusion part concatenates the point set features and image features and fuses them through a linear layer and the GeLU activation function. The fusion result is input into the teacher Transformer encoder to obtain a feature representation containing rich physical semantics.
[0050] The student network processes the low-dimensional latent feature vector to obtain feature representations, specifically by directly inputting the VAE-encoded 4D-STEM latent features into the student Transformer encoder. The structure of the student network is consistent with the backbone of the teacher network, but the input layer is different.
[0051] Step S3: Using the low-dimensional latent feature vectors and privileged modal data generated in Step S1, train the student network with a composite loss function. During training, the teacher network parameters are updated synchronously via momentum updates. The teacher network does not perform backpropagation, enabling the student network to learn the teacher network's prior physical knowledge and align with the feature space. This step aims to enable the student network to learn physical laws. A momentum update strategy is adopted, where the teacher network parameters are updated by the exponential moving average (EMA) of the student network parameters, without backpropagation.
[0052] The total loss function for training consists of three parts:
[0053] DINO distillation loss Calculate the cross-entropy loss between the [CLS] tokens output by the teacher network and the student network. To enhance learning performance, the output probability distribution is sharpened and centered, forcing the feature distribution of the student network to align with that of the teacher network.
[0054] KoLeo Difference Entropy Loss This function calculates and maximizes the differential entropy of the student network output features. It maximizes the distance between features of different samples within the same batch, preventing feature collapse and ensuring the model learns diverse defective features.
[0055] Mask reconstruction loss The input to the student network is randomly masked, requiring the network to reconstruct the occluded diffraction pattern features. This forces the student network to understand the underlying structural rules of the diffraction pattern.
[0056] Step S4: Remove the teacher network and connect a decoder and a segmentation head after the trained student network encoder. Use a small number of expert-annotated material (such as magnesium alloy) dislocation segmentation masks as ground truth to perform end-to-end supervised fine-tuning of the network. The optimization objective is to minimize the difference between the predicted segmentation map and the expert annotation, and optimize the network's dislocation localization and classification performance in specific material systems.
[0057] Step S5: In practical applications, only the finely tuned student network is deployed. The finely tuned student network is deployed, inputting the low-dimensional latent feature vector obtained from the 4D-STEM data to be detected after VAE encoding, and outputting the material defect detection results. 3. Since there is no need to generate VDF images or perform Bragg point indexing, and the model has internalized physical knowledge, the inference speed is significantly improved compared to traditional methods (reduced from several hours to less than 10 minutes).
[0058] Example 2:
[0059] This embodiment provides an automated 4D-STEM material defect detection system, including:
[0060] The data acquisition and preprocessing module is used to acquire 4D-STEM data obtained from material sample scanning and perform structured processing to construct a four-dimensional data cube containing information on the correlation between scanning position and diffraction pattern. Based on the four-dimensional data cube, low-dimensional latent feature vectors input by the student network and privileged modal data input by the teacher network are generated respectively.
[0061] The network construction module is used to construct an asymmetric teacher-student neural network, wherein the teacher network processes the privileged modality data to obtain physical semantic features, and the student network processes the low-dimensional latent feature vector to obtain feature representations;
[0062] The network training module uses the low-dimensional latent feature vectors and privileged modal data generated in step S1 to train the student network with a composite loss function. During the training process, the teacher network parameters are updated synchronously through momentum updates. The teacher network does not backpropagate, so that the student network learns the physical prior knowledge of the teacher network and aligns with the feature space.
[0063] The network fine-tuning module removes the teacher network and connects a decoder and segmentation head after the encoder of the trained student network, using labeled data to supervise and fine-tune the student network.
[0064] The deployment and detection module is used to deploy the fine-tuned student network, input the low-dimensional latent feature vector obtained after preprocessing the 4D-STEM data to be detected, and output the material defect detection results.
[0065] Example 3:
[0066] This embodiment provides a specific case of an automated 4D-STEM material defect detection method. The system configuration and software environment are as follows:
[0067] This embodiment is developed and trained based on a deep learning framework, with the following specific configuration:
[0068] Hardware environment: 1 NVIDIA 5090 GPU.
[0069] Software environment: Operating system is Windows, programming language is Python 3.13, deep learning framework is PyTorch 2.7.0+cu129.
[0070] The experimental data were obtained from material data acquired using a 4D-STEM device. This embodiment uses dislocation defects in Mg alloys as the detection target. The dataset contains 600*600 scan locations, each corresponding to a 192*192 pixel diffraction pattern. The small amount of supervision data required for downstream tasks (dislocation segmentation and Burgers vector classification) consisted of dislocation masks and type labels manually annotated by materials scientists using traditional two-beam analysis.
[0071] Then, data preprocessing and mode generation are performed:
[0072] For the student-side input (4D-STEM Latent Feature) part, a VAE (Variational Autoencoder) is first trained to perform unsupervised compression of the original 4D-STEM diffraction patterns. After training, the weights of the VAE are frozen, and all 4D-STEM diffraction patterns are encoded into a latent feature vector S with a dimension set to Dvae=192. This vector serves as the input to the student network. Figure 3 This is a reconstruction of the diffraction pattern by the VAE encoder obtained after pre-training on the 4d-stem dataset. Figure 4 It is a real-space reconstruction image after feature encoding by the VAE encoder. Similar colors indicate that they have similar features.
[0073] 2. The teacher-side privileged information (VDF Images + Bragg Points) section involves selecting regions near two different reciprocal lattice vectors g in reciprocal space to perform virtual dark field (VDF) imaging on the 4D-STEM data, generating a dual-channel VDF image I. Bragg peak detection and indexing are performed on each diffraction pattern, extracting the top N=8 diffraction points with the highest intensity to form a Bragg point set. Each point in the set contains 6-dimensional features C_p=6 (including qx, qy, h, k, l, I), forming the point set tensor P. Figure 5 It is a virtual dark field (VDF) imaging of 4D-STEM data. After contrast adjustment, three defect labels are generated (from top left to bottom right: dislocation 1, blank, twin, and dislocation 2). The white area is the feature location to be learned. Together, they serve as a modal input for the teacher.
[0074] An asymmetric teacher-student neural network, with the core network based on a Vision Transformer encoder. The model dimension (Dmodel) is uniformly set to 256. The Transformer encoder contains 6 layers of encoding blocks and has 8 self-attention heads.
[0075] The teacher network flattens the point set P and maps it to E through a linear layer. pts The VDF image I is mapped to E through a 1*1 convolution. img Then E pts and E img The features are concatenated along the feature dimension and input into a fusion block containing a linear layer and a GeLU activation function to obtain the final teacher feature sequence T. teacher The weights θ of the teacher network t Using student network weights θ sThe exponential moving average (EMA) is updated, and the momentum τ is set to 0.999 to ensure that the teacher network provides stable learning objectives.
[0076] The Student Network only receives 4D-STEM latent features S encoded by VAE.
[0077] The asymmetric teacher-student neural network was then trained, and the total loss function during training consisted of three parts: weights. , Used to balance various losses:
[0078]
[0079] in The [CLS] token feature vector output by the teacher network. The [CLS] token feature vector output by the student network.
[0080] Training parameters were set as follows: Batch Size: 64. Optimizer: AdamW optimizer. Learning Rate: Cosine Annealing scheduler with an initial learning rate of 10⁻⁴. Training Cycles: Total pre-training cycles were set to 300 epochs.
[0081] After training, the teacher network is discarded, and a lightweight decoder (e.g., a U-Net-style or simple convolutional decoder) is added after the student Transformer encoder for pixel-level dislocation segmentation. Supervised fine-tuning is performed using expert-annotated ground truth labels for Mg alloy dislocations, and the loss function is a combination of Dice Loss and Cross Entropy Loss.
[0082] Figure 6 The image shows the segmentation results from the trained student's end. The original label is used for the plotting, and the right image shows the segmentation results.
[0083] The trained model was applied to a new 4D-STEM dataset and compared with existing techniques:
[0084] Segmentation accuracy: The average intersection-to-union ratio (IoU) of the dislocation region reaches 0.91 or higher.
[0085] Classification accuracy: Burgers vector type (e.g.) <0002> The classification accuracy of (e.g., <11-20>) reached over 88%.
[0086] Computational efficiency: The time required to process a standard-sized 4D-STEM dataset is reduced from several hours using traditional methods to less than 25 minutes, enabling high-throughput automated detection.
[0087] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0088] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0090] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.
[0091] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the various method embodiments described above.
[0092] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. An automated method for detecting defects in 4D-STEM materials, characterized in that, The method includes the following steps: Step S1: Obtain 4D-STEM data from material sample scanning and perform structured processing to construct a four-dimensional data cube containing information on the correlation between scanning position and diffraction pattern. Based on the four-dimensional data cube, generate low-dimensional latent feature vectors input by the student network and privileged modal data input by the teacher network. Step S2: Construct an asymmetric teacher-student neural network, wherein the teacher network processes the privileged modality data to obtain physical semantic features, and the student network processes the low-dimensional latent feature vector to obtain feature representations; Step S3: Using the low-dimensional latent feature vectors and privileged modal data generated in step S1, train the student network with a composite loss function. During the training process, the teacher network parameters are updated synchronously through momentum updates. The teacher network does not backpropagate, so that the student network learns the physical prior knowledge of the teacher network and aligns with the feature space. Step S4: Remove the teacher network, connect the decoder and segmentation head after the encoder of the trained student network, and use labeled data to perform supervised fine-tuning of the student network; Step S5: Deploy the fine-tuned student network, input the low-dimensional latent feature vector obtained after preprocessing the 4D-STEM data to be detected, and output the material defect detection results.
2. The automated detection method for 4D-STEM material defects according to claim 1, characterized in that, In step S1, the low-dimensional latent feature vector is obtained by encoding the original diffraction pattern using a pre-trained variational autoencoder with frozen weights.
3. The automated detection method for 4D-STEM material defects according to claim 1, characterized in that, In step S1, the process of generating the privileged modal data input by the teacher network includes: By computationally processing 4D-STEM data, a simulated virtual dark-field image under dual-beam conditions is generated; Index analysis of the diffraction pattern is performed to extract the strongest N Bragg diffraction peaks to construct a Bragg point set. Each point in the Bragg point set contains 6 physical descriptors: reciprocal space coordinates qx, qy, Miller indices h, k, l, and intensity I.
4. The automated detection method for 4D-STEM material defects according to claim 3, characterized in that, The physical semantic features obtained by the teacher network processing the privileged modality data include: The extracted Bragg point set tensor is flattened and projected onto the model dimension through a linear layer to obtain the point set features; The virtual dark field image is projected onto the model dimension through a convolutional layer to obtain image features; The point set features are concatenated with the image features and fused through a linear layer and the GeLU activation function. The result is then fed into the teacher's Transformer encoder to obtain a feature representation containing rich physical semantics.
5. The automated detection method for 4D-STEM material defects according to claim 1, characterized in that, The composite loss function includes distillation loss, difference entropy loss, and mask reconstruction loss; The distillation loss is calculated to determine the cross-entropy loss between the CLS tokens output by the teacher network and the student network, aligning the feature distributions of the student network and the teacher network. The differential entropy loss calculates and maximizes the differential entropy of the student network output features, which is used to maximize the distance between features of different samples within the same batch and prevent feature collapse. The mask reconstruction loss is used to randomly mask the input of the student network, requiring the network to reconstruct the occluded diffraction pattern features.
6. The automated detection method for 4D-STEM material defects according to claim 1, characterized in that, In step S3, the momentum update updates the teacher network parameters through an exponential moving average.
7. The automated detection method for 4D-STEM material defects according to claim 1, characterized in that, In step S5, the output material defect detection results include dislocation segmentation results and Burgers vector type.
8. An automated 4D-STEM material defect detection system, characterized in that, include: The data acquisition and preprocessing module is used to acquire 4D-STEM data obtained from material sample scanning and perform structured processing to construct a four-dimensional data cube containing information on the correlation between scanning position and diffraction pattern. Based on the four-dimensional data cube, low-dimensional latent feature vectors input by the student network and privileged modal data input by the teacher network are generated respectively. The network construction module is used to construct an asymmetric teacher-student neural network, wherein the teacher network processes the privileged modality data to obtain physical semantic features, and the student network processes the low-dimensional latent feature vector to obtain feature representations; The network training module uses the low-dimensional latent feature vectors and privileged modal data generated in step S1 to train the student network with a composite loss function. During the training process, the teacher network parameters are updated synchronously through momentum updates. The teacher network does not backpropagate, so that the student network learns the physical prior knowledge of the teacher network and aligns with the feature space. The network fine-tuning module removes the teacher network and connects a decoder and segmentation head after the encoder of the trained student network, using labeled data to supervise and fine-tune the student network. The deployment and detection module is used to deploy the fine-tuned student network, input the low-dimensional latent feature vector obtained after preprocessing the 4D-STEM data to be detected, and output the material defect detection results.