A computer vision-based crystal electron diffraction pattern feature analysis method

Abstract

This invention discloses a method for analyzing the features of crystal electron diffraction patterns based on computer vision, belonging to the field of image processing technology. It solves the problems of low detection efficiency in existing technologies, lack of prior knowledge of crystallographic physics constraints, insufficient noise resistance, and lack of end-to-end quantitative analysis in deep learning methods for diffraction patterns. The method includes: constructing an electron diffraction pattern dataset and training a model; detecting local bounding boxes on sub-patterns of the electron diffraction pattern to be inferred according to a set block size and sliding step size, then superimposing pixel offsets to map back to the global coordinate system; removing redundant boxes through non-maximum suppression to generate an initial set of diffraction spot center coordinates; constructing a lattice filtering algorithm based on the random sampling consensus algorithm to eliminate noise interference and obtain a corrected coordinate set; identifying the local diffraction pattern symmetry based on the number and angular features of diffraction spots in the neighborhood and calculating local geometric parameters to generate quantitative characterization data.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and relates to deep learning object detection technology, random sampling consensus algorithm, and geometric shape constraint method. Specifically, it relates to a method for analyzing the features of crystal electron diffraction patterns based on computer vision. Background Technology

[0002] Electron diffraction crystallography is one of the core methods for resolving microstructures in modern materials science. It can reveal the crystal orientation, phase composition, and local strain distribution of materials, and is of great significance for understanding the relationship between the microstructure and physical properties of materials. Especially in advanced semiconductor processes and the development of novel alloys, selected area electron diffraction patterns generated by transmission electron microscopy contain nanoscale structural information. However, actual acquired electron diffraction patterns usually have a high dynamic range, with excessively high intensity of the central transmission spot and weak signals of higher-order diffraction spots, often accompanied by strong scattering background and random noise, making it difficult to clearly define the boundary between weak diffraction spots and the background. In addition, with the development of transmission electron microscopy technology, a large amount of ultra-high resolution (such as 4K, 8K) diffraction pattern data can be generated in a single experiment. Manually calibrating these data one by one is not only tedious and inefficient, but also highly susceptible to the influence of researchers' subjective experience, making it difficult to guarantee the consistency of the analysis results. Therefore, developing a fully automated, high-precision electron diffraction pattern analysis method with physical a priori robustness is currently a research hotspot in the interdisciplinary field of artificial intelligence and materials science (AI4Science).

[0003] Before the widespread adoption of deep learning technology, electron diffraction pattern analysis typically employed semi-automated methods, primarily relying on model-driven image processing techniques. These methods can be broadly categorized into three types: the first type, morphology-based methods (such as adaptive thresholding, watershed algorithms, and Hough transform), suffer from poor noise resistance due to utilizing only low-level grayscale features of the image; the second type, template matching-based methods (such as ACOM-TEM), depend on large-scale offline databases, resulting in low computational efficiency and difficulty adapting to the morphological diversity caused by lattice distortion; and the third type, function fitting-based methods, while offering high accuracy, are highly dependent on the selection of initial parameters, and their computational cost is extremely high during high-throughput processing, making them unsuitable for real-time analysis.

[0004] Compared to traditional image processing methods, deep learning methods can fully utilize the diversity of large training samples, making their feature extraction capabilities superior to traditional methods in complex imaging environments. Represented by the research of Ziletti et al., which used deep convolutional neural networks for end-to-end global analysis of electron diffraction patterns, demonstrating the advantages of deep learning in identifying crystal system symmetry and space groups, this type of method does not incorporate prior physical knowledge of the long-range order of the reciprocal lattice, resulting in limited noise resistance. Another type of method uses fully convolutional neural networks or U-Net architectures for semantic segmentation of diffraction spots. For example, while the U-Net structure proposed by Ronneberger et al. can achieve good pixel-level segmentation results, when faced with today's large field-of-view, ultra-high-resolution electron diffraction images, the fixed geometric receptive field of conventional convolutional kernels limits direct processing, leading to memory overflow or loss of small target features. Conversely, using conventional slicing easily results in the truncation of edge diffraction spots, causing serious missed detections.

[0005] Currently, mainstream deep learning methods still face three main challenges in electron diffraction analysis. First, there is a lack of efficient methods for processing large field-of-view, high-resolution spectra, resulting in low detection efficiency and easy omission of edge spots. Second, purely data-driven models fail to effectively utilize physical prior knowledge such as the long-range order of reciprocal lattices as constraints, making them prone to false detections when dealing with background halos and image noise. Finally, existing network models typically remain at the visual localization level, failing to achieve automated conversion from diffraction spot coordinates to deeper features such as local diffraction pattern symmetry, lattice distortion, and complex order factor, and lacking end-to-end visualization and quantitative analysis tools. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to provide a method for analyzing the features of crystal electron diffraction patterns based on computer vision, which solves the problems of low detection efficiency, lack of crystallographic physics prior knowledge constraints, insufficient noise resistance, and lack of end-to-end quantitative analysis in the existing technology.

[0007] The technical solution adopted in this invention is as follows: A method for analyzing the features of crystal electron diffraction patterns based on computer vision includes the following steps: Step 1: Input the electron diffraction pattern to be inferred into the trained YOLO11 model, and set the sub-plot height for the inference stage. ,width and high overlap rate Width overlap rate Calculate the sliding step size, traverse the sub-patch sequence to obtain the pixel offset of each sub-patch in the global coordinate system; superimpose the local bounding boxes detected by the sub-patch with the corresponding pixel offsets and map them back to the global coordinate system; remove redundant boxes through non-maximum suppression to generate an initial set of diffraction spot center coordinates. ; Step 2: Select the parameter set according to the electron diffraction pattern size to be inferred and construct the KD-Tree spatial index, and set the initial diffraction spot center coordinates. Each diffraction spot This constitutes the set of distance distributions across the entire graph. Obtain the range of vector magnitude of the adaptive reciprocal lattice. By randomly sampling and verifying geometric constraints, an iterative search is performed to find a reciprocal lattice coordinate system model containing the largest number of interior points; an objective function is constructed based on the set of interior points of the reciprocal lattice coordinate system model. The origin coordinates and vector parameters of the reciprocal lattice are globally optimized, and noise is removed to obtain the corrected coordinate set. ; Step 3: In the corrected coordinate set In the middle, the target diffraction spots Taking the origin as the origin, a local neighborhood is constructed based on the dynamic radius search algorithm and an adaptive search radius strategy. The symmetry of the local diffraction pattern is identified based on the number and angular characteristics of the diffraction spots in the neighborhood. The reciprocal vector modulus and bond angle characteristics are calculated, and a complex order factor is introduced to quantify the degree of symmetry of the diffraction pattern. The training of the YOLO11 model involves: constructing an electron diffraction pattern dataset, and sorting the original electron diffraction patterns according to height... ,width and high overlap rate Width overlap rate The model is divided into training set sub-plots, and a three-stage progressive model training strategy is adopted to train the YOLO11 model with pre-trained weights.

[0008] Furthermore, the training process of the YOLO11 model is as follows: Let the area of ​​the bounding box of the original diffraction spots in the electron diffraction pattern be... The area of ​​the bounding box of the labeled sub-plots of the training set obtained after slicing is Define the retention threshold When the conditions are met At that time, the truncated labels of the sub-patterns in the training set are retained to obtain an effectively labeled diffraction spot dataset for model training; The effective labeled diffraction spot dataset includes electron diffraction spots and light-blocking plate regions central_spot, which are randomly divided into training and validation sets according to proportions. A three-stage progressive model training strategy is adopted. First, arbitrary pre-trained weights from the YOLO11 series models are loaded; then, the backbone network is frozen. Layer parameters, using a stochastic gradient descent optimizer, with an initial learning rate set. ,momentum Weight decay And introduce the probability as Weak data augmentation; secondly, entering a strong augmentation training phase with all parameters unfrozen, increasing the initial learning rate to... And in conjunction with cosine annealing scheduling, a decay factor is set. Maximum number of rounds and the number of early stop wheels At the same time, apply full probability enhancement and the probability is Random erasure is used as a strong data augmentation technique; finally, an extremely low learning rate is utilized. This drives the model to converge on the distribution of sub-plots in the training set, reducing the training confidence threshold to [value missing]. And increase the maximum number of targets to be detected. At the same time, Mosaic enhancement is disabled and the random erase probability is reduced to .

[0009] Furthermore, the process of step 1 is as follows: Step 1.1: The sliding step size of the sub-pattern on the electron diffraction pattern. and The calculation is as follows: in, This represents the floor function; Step 1.2: Based on the sliding step size obtained in Step 1.1 and Generate a sequence of sub-tiles and record each sub-tile. The coordinates of the upper left corner in the electron diffraction pattern ,in and Representing sub-blocks The pixel offset of the top left corner relative to the origin of the top left corner of the electron diffraction pattern in the horizontal and vertical directions; Step 1.3, for sub-blocks The first detected in There are several targets, and their local bounding box coordinates are... ,in and These represent the x and y coordinates of the target's top-left corner in the local coordinate system of the sub-tile. and These represent the x and y coordinates of the bottom right corner of the bounding box of the target in the local coordinate system of the sub-tile; mapping them back to the global coordinate system yields the global bounding box coordinates. : Simultaneously, the center coordinates of the target are calculated based on the mapped global bounding box coordinates. The calculation formula is as follows: Step 1.4: Analyze all candidate bounding boxes in the global coordinate system of the electron diffraction pattern. Perform nonmaximum suppression operation for any two local bounding boxes. Calculate the intersection-union ratio: First, low-confidence bounding boxes are filtered out using a confidence threshold. Then, a non-maximum suppression threshold is set. The remaining bounding boxes are sorted from highest to lowest confidence, and the bounding box with the highest confidence is retained. Redundant boxes with an intersection-union ratio (IU) greater than the non-maximum suppression threshold are removed. Only bounding boxes with an IU less than the non-maximum suppression threshold are retained, generating an initial set of diffraction spot center coordinates. .

[0010] Furthermore, the process of step 2 is as follows: Step 2.1: Set the initial set of diffraction spot center coordinates. Each diffraction spot Using KD-Tree to search its first The distances of the nearest neighbors constitute the set of distance distributions across the entire graph. ,in The preset number of nearest neighbors to search; Minimum length :Pick The set of, where The preset distance distribution quantile threshold, which corresponds to the first nearest neighbor distance, is used to exclude nearby noise interference. Maximum length : Set as minimum length times, that is ,in This is the preset maximum length expansion factor, and To cover the first nearest neighbor layer and allow for a certain coordinate offset; Obtain the reciprocal lattice vector magnitude range ; Step 2.2: From the initial set of diffraction spot center coordinates A diffraction spot is randomly selected as the origin of the reciprocal lattice. At the origin of the reciprocal lattice Pointed Within the radius, a KD-Tree search is used to find the reciprocal lattice vector neighborhood set, and two vectors are selected from iterative searches within this set. and It is a candidate reciprocal lattice vector that simultaneously satisfies the following three geometric constraints: Length constraint, and ; Key corner constraint, key corner , ,in The preset bond angle tolerance threshold is used to avoid collinearity of reciprocal vectors; Aspect ratio constraints ,in The preset aspect ratio upper limit threshold; Using the reciprocal lattice coordinate system model Verify the initial set of diffraction spot center coordinates Determine the geometric conformity of each point and calculate the corresponding projection coefficients: in, For containing two components and The projection coefficient vector, For set The first in Coordinates of the diffraction spots It is a matrix composed of candidate reciprocal lattice vectors; Calculate reconstruction error : in, This represents the rounding function. This represents the L2 norm used to calculate Euclidean distance; like ,in If the preset interior point distance error threshold is used, then the determination is made. For interior points, count the number of interior points. ; If the ratio of the number of interior points ,in With a preset interior point ratio threshold, the reciprocal lattice coordinate system model containing the largest number of interior points has been found. The iteration was terminated prematurely; Step 2.3: Define the nonlinear least squares objective function. Let be the sum of squared reconstruction errors for all interior points, where in It is the integer Miller index determined during the random sampling consistency stage; For the nonlinear least squares objective function Perform iterative minimization to optimize the coordinates of the origin of the reciprocal lattice and the reciprocal lattice vector parameters; Using the optimized reciprocal lattice origin coordinates and reciprocal lattice vector parameters, the sum of squared reconstruction errors of all interior points is recalculated, and errors exceeding a certain threshold are discarded. Edge noise, obtain the corrected coordinate set .

[0011] Furthermore, step 3 is as follows: Step 3.1: Design a dynamic radius search algorithm and define the search radius multiplier factor. Target diffraction spots Using the origin as the reference point, calculate the distance to the nearest neighbor. The search radius is set through iterative judgment. ,exist Searching for diffraction spot set in the neighborhood If the number of diffraction spots in this neighborhood This indicates that the radius is too small or is on the boundary, in which case execution will be performed. Update the search radius and repeat the search until... Or find a sufficient number of diffraction spots, i.e. ,in This is a preset threshold for the minimum number of diffraction spots. The preset radius reduction step size, This is the preset lower limit of the multiplier factor; Step 3.2: Arrange the diffraction spot set described in Step 3.1 according to the target diffraction spot. polar angle Sort and calculate the difference between adjacent angles. If and only if the number of diffraction spots in that neighborhood... And all adjacent bond angles satisfy At that time, it was determined to be a six-neighbor class, among which The preset six-neighbor bond angle error threshold; When the six-neighbor class fails to determine the nearest neighbor, the nearest neighbor point is selected. The corresponding vector is Search for vectors among the remaining neighboring points. , making and ,in and These are the preset orthogonal dot product error threshold and modulus error threshold, respectively; if a basis vector pair is found, and the bond angle difference formed by these four points satisfies... ,in If the preset eight-neighbor bond angle error threshold is met, it is determined to be an eight-neighbor class; If neither of the above two criteria is met, then the number of diffraction spots in the current neighborhood is determined. ,like If so, the local region is downgraded to a low-symmetry polygon class, the principal azimuth angle is determined based on the nearest neighbor point, and existing neighboring points are connected sequentially to extract the basic bond length and bond angle features of the local region; if If the features do not meet the polygon formation conditions at all, they are directly marked as disordered or defective regions, and the symmetry determination of the current target spot is terminated. Step 3.3, Calculation of local geometric parameters Reciprocal vector modulus: used to calculate the target diffraction pattern. Distance to each neighboring point Its mean Corresponding reciprocal lattice parameters ; Bond angle features and average angle difference: based on all adjacent angle differences obtained in step 3.2. Calculate their average value as the average angle difference. Simultaneously calculate the variance of all adjacent angle differences. As a bond angle feature, it is used to quantitatively assess the degree of lattice distortion, and the calculation formula is as follows: Complex order factor calculation: based on target diffraction spots Using the origin as the reference point, select the nearest neighboring point. Establish a local coordinate system with the direction as the reference axis; calculate the value of each spot in the neighborhood of the target diffraction spot. Relative to target diffraction pattern vector And calculate the phase angle of the vector in the local coordinate system. Secondly, choose a symmetric order. If it is determined to be a six-neighbor class, then If it is determined to be an eight-neighbor class, then If it is a low-symmetry polygon class that triggers the fallback logic, then the number of neighboring points in the local neighborhood is directly taken as the symmetry order, i.e. If the region has been marked as disordered or defective, the calculation is terminated immediately and a complex order factor is assigned. For six-neighbor classes, eight-neighbor classes, or low-symmetry polygon classes, summation using complex numbers is used: in, The imaginary unit is the modulus. This indicates the degree of integrity of the translational symmetry. A value close to 1 indicates good symmetry in the local diffraction pattern. A value close to 0 typically indicates an amorphous or severely distorted region, with the argument changing from... The calculation yields the rotational orientation angle of the crystal lattice in the two-dimensional plane.

[0012] Furthermore, the height overlap rate and width overlap rate are set to be higher in the inference phase than in the training phase.

[0013] Furthermore, in step 2, the parameter set for selecting the electron diffraction pattern size based on the required inference is specifically as follows: Set size threshold as Pixels, if Then the interior point threshold Pixel, reciprocal lattice vector neighborhood ; like Then the interior point threshold Pixel, reciprocal lattice vector neighborhood ,in For the height of the map, This represents the width of the map.

[0014] Compared with the prior art, the present invention has the following advantages: This invention addresses the detection of weak diffraction spots in large field-of-view images using a deep learning detection model with slice-assisted reasoning. It employs a lattice filtering algorithm based on random sampling consistency, introducing long-range crystal order as a physical constraint to automatically eliminate noise-induced false detections. Finally, through local diffraction pattern structure analysis, it automatically identifies the symmetry of local diffraction patterns and calculates neighborhood geometric features and complex order factors. This invention significantly improves false detection suppression while maintaining detection accuracy, achieving end-to-end processing of reciprocal lattice analysis. The specific analysis is as follows: (1) The present invention introduces a fusion strategy of slice-assisted reasoning and non-maximum suppression, which can directly process images of 4K, 8K and even larger sizes, effectively suppressing the missed detection caused by diffraction spots at the edge of the slice, realizing high-precision detection in the entire field of view, and solving the problem of low detection efficiency of traditional methods in large field of view scenarios.

[0015] (2) This invention combines deep learning detection results with a lattice filtering method based on random sampling consensus algorithm. By utilizing the long-range order physical prior of the reciprocal lattice of crystal, it can automatically identify and eliminate false detection points caused by image noise and background halo, which significantly improves the reliability of detection results.

[0016] (3) The present invention proposes a method for analyzing local diffraction pattern structure, which can automatically identify the symmetry of local diffraction pattern and calculate local geometric parameters without human intervention, providing a visual and quantitative analysis tool for studying the lattice distortion, strain distribution and phase structure inside the material.

[0017] This invention can not only automatically obtain the center coordinates of diffraction spots, but also effectively enhance the ability to suppress false detections, providing objective and accurate data for the analysis of lattice distortion and strain of the microstructure of crystal materials. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the system architecture of the present invention; Figure 2 This is a schematic diagram illustrating the principle of the adaptive search radius of the present invention; Figure 3 This is a graph showing the evolution of loss and accuracy of the model under the three-stage progressive training method of this invention. Figure 4 This is a comparison chart of ablation test indicators of the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0020] To address the technical bottlenecks in existing electron diffraction pattern analysis, such as difficulties in identifying weak diffraction spots, severe background noise interference, lack of physical prior constraints, and low quantification and batch processing capabilities, this invention designs a method. First, an electron diffraction pattern dataset is constructed and used for model training. Second, a deep learning target detection model with slice-assisted reasoning is employed to detect diffraction spots in the input electron diffraction pattern using slice-assisted reasoning. Then, based on the initial set of diffraction spot center coordinates, a random sampling consensus lattice filtering algorithm is constructed to initially filter this coordinate set. Subsequently, using the filtered diffraction spot set, a local neighborhood is constructed centered on the target diffraction spot, and local geometric statistical analysis is performed. Finally, the diffraction spot detection results are correlated with the reciprocal lattice feature analysis results and output.

[0021] Construct an electron diffraction pattern dataset, arranging the original electron diffraction patterns according to height. ,width and high overlap rate Width overlap rate The model is divided into training set sub-plots, and a three-stage progressive model training strategy is used to train the YOLO11 model with pre-trained weights. After training, as shown... Figure 1 As shown, the method for analyzing the features of crystal electron diffraction patterns based on computer vision specifically includes the following three stages: The first-stage deep learning object detection model with slice-assisted inference divides the high-resolution electron diffraction pattern to be inferred into sub-patterns for independent inference. After mapping back to the global coordinate system using offsets, non-maximum suppression is performed to generate an initial set of diffraction spot center coordinates. .

[0022] The second stage involves global reciprocal lattice filtering based on the random sampling consensus algorithm: A parameter set is selected according to the electron diffraction pattern size inferred, and a KD-Tree spatial index is constructed to adjust the initial set of diffraction spot center coordinates. Each diffraction spot This constitutes the set of distance distributions across the entire graph. Obtain the range of vector magnitude of the adaptive reciprocal lattice. By randomly sampling and verifying geometric constraints, an iterative search is performed to find a reciprocal lattice coordinate system model containing the largest number of interior points; an objective function is constructed based on the set of interior points of the reciprocal lattice coordinate system model. The origin coordinates and vector parameters of the reciprocal lattice are globally optimized, and noise is removed to obtain the corrected coordinate set. .

[0023] Third-stage local diffraction pattern structure analysis: in the corrected coordinate set In the middle, the target diffraction spots Using the origin as the starting point, an adaptive search radius strategy is introduced to construct a local neighborhood based on the dynamic radius search algorithm. The symmetry of the local diffraction pattern is identified based on the number and angular characteristics of the diffraction spots in the neighborhood. The reciprocal vector magnitude and bond angle characteristics are calculated, and a complex order factor is introduced to quantify the degree of symmetry of the diffraction pattern.

[0024] Example (1) Source and preprocessing of electron diffraction pattern dataset: The electron diffraction dataset is derived from selected area electron diffraction patterns of an unknown crystal material acquired by transmission electron microscopy. It contains 60 4K high-resolution images. In order to solve the problems of memory overflow and loss of small target features caused by directly inputting high-resolution images into the neural network, and to improve the robustness of the model in detecting edge targets, this invention implements a strict slicing and cleaning strategy in the data preprocessing stage.

[0025] (a) Slice generation and overlap settings during training phase In the training phase of the deep learning model, in order to enhance sample diversity and avoid the target being truncated at the edge of the slice as much as possible, this invention adopts a slice generation strategy based on overlapping sliding windows.

[0026] ①Slice size definition: Set the height and width of the slice as... ; ② Overlap Rate Parameter: During the training phase, to balance computational efficiency and the integrity of edge targets, an overlap rate is set in both the height and width directions. This provides training samples with different truncation perspectives for the model.

[0027] (b) Label cleaning and screening strategy based on area ratio When generating labels for slices, direct truncation often results in incomplete labels that contain only a very small portion of the target. These incomplete labels can interfere with the convergence of the noise-affected model. To address this, this invention designs a rigorous label selection mechanism.

[0028] ① Let the area of ​​the bounding box of the original diffraction spots in the electron diffraction pattern be... ; ②The area of ​​the bounding box of the training set sub-plots obtained after slicing is ; ③ Define the retention threshold Only if the condition is met Only when this condition is met will the target be labeled as a valid training sample for the current slice, thus preserving the truncated annotation information. This strategy effectively filters out incomplete diffraction spots caused by slice boundary cutting, ensuring that the model learns relatively complete diffraction spot features.

[0029] (c) Classification and Dataset Segmentation The dataset annotations include two categories: spot (electron diffraction spots) and central_spot (light-blocking region). To verify the model's generalization ability, the processed slice dataset was randomly divided into training and validation sets in a 9:1 ratio. The training set was used for backpropagation updates of the model weights, and the validation set was used to evaluate model performance and implement an early stopping strategy after each epoch.

[0030] (2) The present invention uses a three-stage progressive model training process. A three-stage training strategy is adopted, and in this embodiment, lightweight pre-trained weights with version number n from the YOLO11 series model are selected. First, the backbone network is frozen... Layer parameters, using a stochastic gradient descent optimizer, initial learning rate ,momentum Weight decay Gradient updates are performed only on the neck and head, and probabilities are introduced. Weak augmentation. Next, we proceed to strong augmentation training with all parameters unfrozen, increasing the initial learning rate to... And in conjunction with cosine annealing scheduling, attenuation factor Set the maximum number of rounds Stop early At the same time, apply full probability enhancement Randomly erase Strong data augmentation techniques. Finally, using an extremely low learning rate. The driving model converges on the distribution of sub-plots in the training set, reducing the training confidence threshold to [value missing]. Increase the maximum number of targets to Meanwhile, disable Mosaic enhancement. And reduce the probability of random erasure to .

[0031] like Figure 3 As shown, the left figure records the changes in the bounding box loss of the training and validation sets as the number of training epochs increases. In Stage 1, the loss decreases rapidly; in Stage 2, it maintains a steady downward trend; and in Stage 3, the training set loss continues to decrease while the validation set loss stabilizes, demonstrating good convergence and generalization ability. The right figure shows the evolution of mAP@0.5 and mAP@0.5:0.95. Both metrics continuously improve as training progresses, and the brief fluctuations during stage transitions do not affect the overall upward trend, ultimately reaching a high level at the end of training. This proves that the three-stage training strategy can effectively improve the model's detection performance. Overall, the three-stage training strategy achieves steady loss convergence and continuous accuracy improvement, verifying the effectiveness of progressive training in object detection tasks.

[0032] (3) The slice-assisted reasoning and result fusion process proposed in this invention In the reasoning stage, directly... Scaling an image to the model input size can lead to the loss of tiny diffraction spots (typically only a few pixels). Therefore, this invention introduces a slice-assisted inference strategy.

[0033] (a) Subplot partitioning strategy ① The electron diffraction pattern that needs to be deduced Divide into a series of overlapping sub-tiles, and set the tile size to [value missing]. ; ② Overlap Rate in Inference Stage: To ensure that targets located at the edge of a slice are in the central region of at least one slice and thus accurately detected, the overlap rate in the inference stage... It is higher than the training phase.

[0034] (b) Calculation of sliding window coordinates Slice sliding step on the image and The calculation is as follows: Based on the sliding step and Generate a sequence of sub-tiles and record each sub-tile. The coordinates of the upper left corner in the electron diffraction pattern ,in and Representing sub-blocks The pixel offset of the top left corner relative to the origin of the top left corner of the electron diffraction pattern in both the horizontal and vertical directions; if the image size is not an integer multiple of the slice size, then background completion is performed on the edge region.

[0035] (c) Subplot reasoning and global coordinate mapping Each sub-plot is independently input into the trained YOLO model for inference, with the inference confidence threshold set as follows: For sub-plots The first detected in There are several targets, and their local bounding box coordinates are... Map it back to the global coordinate system: Simultaneously, the center coordinates of the target are calculated based on the mapped global bounding box coordinates. The calculation formula is as follows: (d) Non-maximum suppression (NMS) based on IoU Due to overlapping slices, the same target may be detected multiple times. This involves the set of all detection results for the entire image. Perform the NMS operation. For any two detection boxes... Calculate the intersection-union ratio: First, filter out low-confidence bounding boxes using a confidence threshold, then set a non-maximum suppression threshold. 5. If the detection boxes have 50% overlap, sort the remaining bounding boxes from highest to lowest confidence level, retain the bounding box with the highest confidence level, and remove redundant boxes with an intersection-union ratio (IU) greater than the non-maximum suppression threshold. Only retain bounding boxes with an IU less than the non-maximum suppression threshold to generate the initial set of diffraction spot center coordinates. .

[0036] (4) The process of the lattice filtering algorithm based on random sampling consensus proposed in this invention Since deep learning models are based solely on visual features, they output an initial set of diffraction spot center coordinates. The data may contain noise (such as background stray points) or be missed. This invention utilizes the physical prior of crystallography, namely the long-range order of the reciprocal lattice, and uses a random sampling consensus algorithm to screen out interior points that conform to the two-dimensional periodic lattice distribution.

[0037] (a) Parameter selection and spatial index construction for different image sizes To accommodate images with different resolutions and crystal structures, a parameter set is first selected based on the electron diffraction pattern size required for inference, and a size threshold is set. Pixel; like Using "small graph parameters": interior point threshold Pixel, reciprocal lattice vector neighborhood .

[0038] like Using "Large Image Parameters": Inner Point Threshold Pixel, reciprocal lattice vector neighborhood .

[0039] KD-Tree Construction: Utilizing KD-Tree to... Build a spatial index to accelerate nearest neighbor search.

[0040] (b) Calculation of the adaptive reciprocal vector length range For the initial set of diffraction spot center coordinates Each diffraction spot Using KD-Tree to search its first The distances of the nearest neighbors constitute the set of distance distributions across the entire graph. ,in The preset number of nearest neighbors is preferably used in this embodiment. ; Minimum length :Pick The set of, where The preset distance distribution quantile threshold is preferably, in this embodiment, set as the threshold value. This quantile corresponds to the first nearest neighbor distance, excluding nearby noise interference: Maximum length : Set as minimum length times, that is ,in This is the preset maximum length expansion factor, and In this embodiment, it is preferred that To cover the first nearest neighbor layer and allow for a certain degree of lattice distortion; This yields the range of the reciprocal lattice vector length. .

[0041] (c) Iterative fitting of reciprocal lattice vector model Set the maximum number of iterations Or higher, perform the following steps in each iteration: ① Random sampling: from A diffraction spot is randomly selected as the assumed origin of the reciprocal lattice. ; ② Generation of candidate reciprocal lattice vectors: at the origin of the reciprocal lattice of Within the radius, a KD-Tree search is used to find neighboring points, and two vectors are selected from the search results. and As a candidate reciprocal vector; ③ Strict geometric constraint verification: To ensure that the selected reciprocal lattice vector is physically reasonable, the following three geometric constraints must be satisfied simultaneously: Length constraint, and ; Key corner constraint, key corner , ,in The preset bond angle tolerance threshold is preferably, in this embodiment, set as the threshold value. To avoid collinearity of reciprocal vectors; Aspect ratio constraints ,in In this embodiment, the preset aspect ratio upper limit threshold is preferably [value missing]. ; ④ Interior point determination and statistics: Reciprocal easy lattice coordinate system Verify the initial set of diffraction spot center coordinates Determine the degree of conformity of all points and calculate the projection coefficient: in, For containing two components and The projection coefficient vector, For set The first in Coordinates of the diffraction spots It is a matrix composed of candidate reciprocal lattice vectors; Calculate reconstruction error : in, This represents the rounding function. This represents the L2 norm used to calculate Euclidean distance; like ,in If the preset interior point distance error threshold is used, then the determination is made. For interior points, count the number of interior points. ; ⑤ Early Stop Strategy: If the proportion of inner points ,in The preset optimal in-model point ratio threshold is preferably, in this embodiment, set as the threshold value. If the optimal model has been found, then the reciprocal lattice coordinate system model containing the largest number of interior points has been found. The iteration was terminated prematurely.

[0042] (d) Nonlinear least squares global optimization The model obtained through random sampling consensus is determined based on only three points (origin + two base points), and its accuracy is limited by the positional errors of local points. Therefore, in order to obtain higher global accuracy, this invention constructs a corresponding optimization problem.

[0043] ① Objective function: Define a nonlinear least squares objective function. Let be the sum of squared reconstruction errors for all interior points, where .

[0044] in It is the integer Miller index determined during the random sampling consistency stage; ②Solution: Use the Levenberg-Marquardt algorithm to solve the problem. Perform iterative minimization to find the solution; ③ Final Filtering: Using the optimized reciprocal lattice origin coordinates and reciprocal lattice vector parameters, the sum of squared reconstruction errors of all interior points is recalculated, and errors exceeding the limit are discarded. Edge noise, obtain the corrected coordinate set .

[0045] (5) The method for analyzing local diffraction pattern structure and the calculation process of complex order factor proposed in this invention To obtain a pure set of diffraction spots Subsequently, the local diffraction pattern structure is quantitatively characterized, with the core being the automatic identification of local symmetry and the calculation of physical order parameters.

[0046] (a) Local Neighborhood Adaptive Search Strategy To accurately locate nearest-neighbor diffraction spots in the presence of lattice distortion, this invention designs a dynamic radius search algorithm. For example... Figure 2 As shown, the search radius multiplier factor is first defined. Target diffraction spots (The central spot in the image) is the origin, i.e., the central spot. Calculate the nearest neighbor distance. The search radius is set through iterative judgment. ,exist Searching for diffraction spot set in the neighborhood ;like This indicates that the radius is too small or is at the boundary; execute. Repeat the search until... Or find a sufficient number of diffraction spots.

[0047] This strategy ensures the adaptability of the neighborhood search. In this embodiment, the threshold for the number of minimum diffraction spots... Preferably, the radius multiplier decreases by a step size of 4. Preferably, the lower limit of the rate factor is 0.1. The preferred value is 1.5.

[0048] (b) Geometric construction method for determining the symmetry of local diffraction patterns This invention does not rely on template matching, but rather on automatic classification based on geometric features. First, preprocessing is performed, classifying diffraction spots in the neighborhood centered on the target point according to their relative position to the target point. polar angle Sort and calculate the difference between adjacent angles. If and only if the number of diffraction spots And all adjacent bond angles satisfy It was determined to be a six-neighbor class, among which In this embodiment, the preset six-neighbor bond angle error threshold is preferably... ; If the six-neighbor class decision fails, try the eight-neighbor class decision, that is, select the nearest neighbor. The corresponding vector is Search for vectors among the remaining neighboring points. , making and ,in and These are preset orthogonal dot product error thresholds and modulus error thresholds, respectively. In this embodiment, they are preferably... and If the basis vector pair is found, and the bond angle difference formed by these four points satisfies ,in In this embodiment, the preset eight-neighbor bond angle error threshold is preferably... If so, it is determined to be an eight-neighbor class; If neither of the above two criteria is met, then the number of diffraction spots in the current neighborhood is determined. ,like If so, the local region is downgraded to a low-symmetry polygon class, the principal azimuth angle is determined based on the nearest neighbor point, and existing neighboring points are connected sequentially to extract the basic bond length and bond angle features of the local region; if If the features do not meet the polygon formation conditions at all, they are directly marked as disordered or defective regions, and the symmetry determination of the current target spot is terminated.

[0049] (c) Calculation of local geometric parameters ① Reciprocal vector modulus: Calculates the distance from the target diffraction spot to each neighboring point. Its mean Corresponding reciprocal lattice parameters ; ② Bond angle features and average angle difference: based on the difference between all adjacent angles Calculate their average value as the average angle difference. Simultaneously calculate the variance of all adjacent angle differences. As a bond angle feature, it is used to quantitatively assess the degree of lattice distortion. The calculation formula is as follows: ③ Calculation of complex order factor: based on the target diffraction spots Using the origin as the reference point, select the nearest neighboring point. Establish a local coordinate system with the direction as the reference axis; calculate each neighboring point. Vector relative to the target blob And calculate the phase angle of the vector in the local coordinate system. Secondly, choose a symmetric order. If it is determined to be a six-neighbor class, then If it is determined to be an eight-neighbor class, then If the class is a low-symmetry polygon that triggers the fallback logic, then the number of diffraction spots in the local neighborhood is directly taken as the symmetry order, i.e. If the region has been marked as disordered or defective, the calculation is terminated immediately and a complex order factor is assigned. Finally, for the non-unordered region, we use complex summation: in The imaginary unit is the modulus. This indicates the degree of integrity of the translational symmetry. This indicates that the local diffraction pattern has good symmetry. Typically indicates amorphous or severely distorted regions, with the argument ranging from... The calculation yields the rotational orientation angle of the crystal lattice in the two-dimensional plane.

[0050] (6) Verification of experimental results To verify the effectiveness of the proposed method for analyzing the characteristics of crystal electron diffraction patterns, a complete verification was performed on the PyTorch 2.8 framework and the NVIDIA GeForce RTX 3090 GPU platform. Figure 4 As shown, the experimental results indicate that by introducing slice-assisted inference, the recall rate of small target detection increased from 76.2% with direct inference of the whole image to 92.8%; and after combining random sampling consistency global filtering, the precision rate increased from 79.1% to 95.4%, effectively eliminating background noise.

[0051] In summary, this invention can be widely applied in the field of microstructure characterization of crystal materials, laying the foundation for subsequent crystal orientation analysis and phase structure identification. It also provides new research ideas and technical means for a deeper understanding of the relationship between the microstructure and physical properties of novel crystal materials.

Claims

1. A method for analyzing the features of crystal electron diffraction patterns based on computer vision, characterized in that, Includes the following steps: Step 1: Input the electron diffraction pattern to be inferred into the trained YOLO11 model, and set the sub-plot height for the inference stage. ,width and high overlap rate Width overlap rate Calculate the sliding step size, traverse the sub-patch sequence to obtain the pixel offset of each sub-patch in the global coordinate system; superimpose the local bounding boxes detected by the sub-patch with the corresponding pixel offsets and map them back to the global coordinate system; remove redundant boxes through non-maximum suppression to generate an initial set of diffraction spot center coordinates. ; Step 2: Select the parameter set according to the electron diffraction pattern size to be inferred and construct the KD-Tree spatial index, and set the initial diffraction spot center coordinates. Each diffraction spot This constitutes the set of distance distributions across the entire graph. Obtain the range of vector magnitude of the adaptive reciprocal lattice. ; By randomly sampling and verifying geometric constraints, an iterative search is performed to find a reciprocal lattice coordinate system model containing the largest number of interior points; an objective function is then constructed based on the set of interior points of the reciprocal lattice coordinate system model. The origin coordinates and vector parameters of the reciprocal lattice are globally optimized, and noise is removed to obtain the corrected coordinate set. ; Step 3: In the corrected coordinate set In the middle, the target diffraction spots Taking the origin as the origin, a local neighborhood is constructed based on the dynamic radius search algorithm and an adaptive search radius strategy is introduced. The symmetry of the local diffraction pattern is identified based on the number and angular characteristics of the diffraction spots in the neighborhood. The reciprocal vector magnitude and bond angle characteristics are calculated, and a complex order factor is introduced to quantify the degree of symmetry of the diffraction pattern. The training of the YOLO11 model involves: constructing an electron diffraction pattern dataset, and sorting the original electron diffraction patterns according to their height... ,width and high overlap rate Width overlap rate The model is divided into training set sub-plots, and a three-stage progressive model training strategy is adopted to train the YOLO11 model with pre-trained weights.

2. The method for analyzing the characteristics of crystal electron diffraction patterns based on computer vision according to claim 1, characterized in that, The training process of the YOLO11 model is as follows: Let the area of ​​the bounding box of the original diffraction spots in the electron diffraction pattern be... The area of ​​the bounding box of the labeled sub-plots of the training set obtained after slicing is Define the retention threshold When the conditions are met At that time, the truncated labels of the sub-patterns in the training set are retained to obtain an effectively labeled diffraction spot dataset for model training; The effective labeled diffraction spot dataset includes electron diffraction spots and light-blocking plate regions central_spot, which are randomly divided into training and validation sets according to proportions. A three-stage progressive model training strategy is adopted. First, arbitrary pre-trained weights from the YOLO11 series models are loaded; then, the backbone network is frozen. Layer parameters, using a stochastic gradient descent optimizer, with an initial learning rate set. ,momentum Weight decay And introduce the probability as Weak data augmentation; Secondly, we enter the fully unfrozen, heavily augmented training phase, increasing the initial learning rate to [value missing]. And in conjunction with cosine annealing scheduling, a decay factor is set. Maximum number of rounds and the number of early stop wheels At the same time, apply full probability enhancement and the probability is Random erasure is used as a strong data augmentation technique; finally, an extremely low learning rate is utilized. This drives the model to converge on the distribution of sub-plots in the training set, reducing the training confidence threshold to [value missing]. And increase the maximum number of targets to be detected. At the same time, Mosaic enhancement is disabled and the random erase probability is reduced to .

3. The method for analyzing the characteristics of crystal electron diffraction patterns based on computer vision according to claim 1, characterized in that, The process of step 1 is as follows: Step 1.1: The sliding step size of the sub-pattern on the electron diffraction pattern. and The calculation is as follows: in, This represents the floor function; Step 1.2: Based on the sliding step size obtained in Step 1.1 and Generate a sequence of sub-tiles and record each sub-tile. The coordinates of the upper left corner in the electron diffraction pattern ,in and Representing sub-blocks The pixel offset of the top left corner relative to the origin of the top left corner of the electron diffraction pattern in the horizontal and vertical directions; Step 1.3, for sub-blocks The first detected in There are several targets, and their local bounding box coordinates are... ,in and These represent the x and y coordinates of the target's top-left corner in the local coordinate system of the sub-tile. and These represent the x and y coordinates of the bottom right corner of the bounding box of the target in the local coordinate system of the sub-tile; mapping them back to the global coordinate system yields the global bounding box coordinates. : Simultaneously, the center coordinates of the target are calculated based on the mapped global bounding box coordinates. The calculation formula is as follows: Step 1.4: Analyze all candidate bounding boxes in the global coordinate system of the electron diffraction pattern. Perform nonmaximum suppression operation for any two local bounding boxes. Calculate the intersection-union ratio: First, low-confidence bounding boxes are filtered out using a confidence threshold. Then, a non-maximum suppression threshold is set. The remaining bounding boxes are sorted from highest to lowest confidence, and the bounding box with the highest confidence is retained. Redundant boxes with an intersection-union ratio (IU) greater than the non-maximum suppression threshold are removed. Only bounding boxes with an IU less than the non-maximum suppression threshold are retained, generating an initial set of diffraction spot center coordinates. .

4. The method for analyzing the characteristics of crystal electron diffraction patterns based on computer vision according to claim 1, characterized in that, The process of step 2 is as follows: Step 2.1: Set the initial set of diffraction spot center coordinates. Each diffraction spot Using KD-Tree to search its first The distances of the nearest neighbors constitute the set of distance distributions across the entire graph. ,in The preset number of nearest neighbors to search; Minimum length :Pick The set of, where The preset distance distribution quantile threshold, which corresponds to the first nearest neighbor distance, is used to exclude nearby noise interference. Maximum length : Set as minimum length times, that is ,in This is the preset maximum length expansion factor, and To cover the first nearest neighbor layer and allow for a certain coordinate offset; Obtain the reciprocal lattice vector magnitude range ; Step 2.2: From the initial set of diffraction spot center coordinates A diffraction spot is randomly selected as the origin of the reciprocal lattice. At the origin of the reciprocal lattice Pointed Within the radius, a KD-Tree search is used to find the reciprocal lattice vector neighborhood set, and two vectors are selected from iterative searches within this set. and It is a candidate reciprocal lattice vector that simultaneously satisfies the following three geometric constraints: Length constraint, and ; Key corner constraint, key corner , ,in The preset bond angle tolerance threshold is used to avoid collinearity of reciprocal vectors; Aspect ratio constraints ,in The preset aspect ratio upper limit threshold; Using the reciprocal lattice coordinate system model Verify the initial set of diffraction spot center coordinates Determine the geometric conformity of each point and calculate the corresponding projection coefficients: in, For containing two components and The projection coefficient vector, For set The first in Coordinates of the diffraction spots It is a matrix composed of candidate reciprocal lattice vectors; Calculate reconstruction error : in, This represents the rounding function. This represents the L2 norm used to calculate Euclidean distance; like ,in If the preset interior point distance error threshold is used, then the determination is made. For interior points, count the number of interior points. ; If the ratio of the number of interior points ,in With a preset interior point ratio threshold, the reciprocal lattice coordinate system model containing the largest number of interior points has been found. The iteration was terminated prematurely; Step 2.3: Define the nonlinear least squares objective function. Let be the sum of squared reconstruction errors for all interior points, where in It is the integer Miller index determined during the random sampling consistency stage; For the nonlinear least squares objective function Perform iterative minimization to optimize the coordinates of the origin of the reciprocal lattice and the reciprocal lattice vector parameters; Using the optimized reciprocal lattice origin coordinates and reciprocal lattice vector parameters, the sum of squared reconstruction errors of all interior points is recalculated, and errors exceeding a certain threshold are discarded. Edge noise, obtain the corrected coordinate set .

5. The method for analyzing the characteristics of crystal electron diffraction patterns based on computer vision according to claim 1, characterized in that, The process of step 3 is as follows: Step 3.1: Design a dynamic radius search algorithm and define the search radius multiplier factor. Target diffraction spots Using the origin as the reference point, calculate the distance to the nearest neighbor. The search radius is set through iterative judgment. ,exist Searching for diffraction spot set in the neighborhood If the number of diffraction spots in this neighborhood This indicates that the radius is too small or is on the boundary, in which case execution will be performed. Update the search radius and repeat the search until... Or find a sufficient number of diffraction spots, i.e. ,in This is a preset threshold for the minimum number of diffraction spots. The preset radius reduction step size, This is the preset lower limit of the multiplier factor; Step 3.2: Arrange the diffraction spot set described in Step 3.1 according to the target diffraction spot. polar angle Sort and calculate the difference between adjacent angles. If and only if the number of diffraction spots in that neighborhood... And all adjacent bond angles satisfy At that time, it was determined to be a six-neighbor class, among which The preset six-neighbor bond angle error threshold; When the six-neighbor class fails to determine the nearest neighbor, the nearest neighbor point is selected. The corresponding vector is Search for vectors among the remaining neighboring points. , making and ,in and These are the preset orthogonal dot product error threshold and modulus error threshold, respectively; if a basis vector pair is found, and the bond angle difference formed by these four points satisfies... ,in If the preset eight-neighbor bond angle error threshold is met, it is determined to be an eight-neighbor class; If neither of the above two criteria is met, then the number of diffraction spots in the current neighborhood is determined. ,like If so, the local region is downgraded to a low-symmetry polygon class, the principal azimuth angle is determined based on the nearest neighbor point, and existing neighboring points are connected sequentially to extract the basic bond length and bond angle features of the local region; if If the features do not meet the polygon formation conditions at all, they are directly marked as disordered or defective regions, and the symmetry determination of the current target spot is terminated. Step 3.3, Calculation of local geometric parameters Reciprocal vector modulus: used to calculate the target diffraction pattern. Distance to each neighboring point Its mean Corresponding reciprocal lattice parameters ; Bond angle features and average angle difference: based on all adjacent angle differences obtained in step 3.

2. Calculate their average value as the average angle difference. Simultaneously calculate the variance of all adjacent angle differences. As a bond angle feature, it is used to quantitatively assess the degree of lattice distortion, and the calculation formula is as follows: Complex order factor calculation: based on target diffraction spots Using the origin as the reference point, select the nearest neighboring point. Establish a local coordinate system with the direction as the reference axis; Calculate the value of each spot in the neighborhood of the target diffraction spot. Relative to target diffraction pattern vector And calculate the phase angle of the vector in the local coordinate system. Secondly, choose a symmetric order. If it is determined to be a six-neighbor class, then ; If it is determined to be an eight-neighbor class, then If it is a low-symmetry polygon class that triggers the fallback logic, then the number of neighboring points in the local neighborhood is directly taken as the symmetry order, i.e. If the region has been marked as disordered or defective, the calculation is terminated immediately and a complex order factor is assigned. For six-neighbor classes, eight-neighbor classes, or low-symmetry polygon classes, summation using complex numbers is used: in, The imaginary unit is the modulus. This indicates the degree of integrity of the translational symmetry. A value close to 1 indicates good symmetry in the local diffraction pattern. A value close to 0 typically indicates an amorphous or severely distorted region, with the argument changing from... The calculation yields the rotational orientation angle of the crystal lattice in the two-dimensional plane.

6. The method for analyzing the characteristics of crystal electron diffraction patterns based on computer vision according to claim 1, characterized in that, The height overlap rate and width overlap rate are set to be higher in the inference phase than in the training phase.

7. The method for analyzing the characteristics of crystal electron diffraction patterns based on computer vision according to claim 1, characterized in that, In step 2, the parameter set for selecting the electron diffraction pattern size based on the required inference is specifically as follows: Set size threshold as Pixels, if Then the interior point threshold Pixel, reciprocal lattice vector neighborhood ; like Then the interior point threshold Pixel, reciprocal lattice vector neighborhood ,in For the height of the map, This represents the width of the map.

Images