Dislocation defect recognition method and system based on image recognition and deep learning network

Abstract

This invention discloses a method and system for dislocation defect identification based on image recognition and deep learning networks, belonging to the fields of image recognition and machine learning. The method first calibrates the dimensions of a high-resolution electron micrograph, identifies atomic coordinates using a deep learning object detection model, and calculates the local atomic environment vector for each atom. Second, using an atomic model dataset covering various materials, dislocation types, and deformation states, a deep learning network is trained capable of predicting the distance between atoms and dislocations based on the local environment vector. Finally, the image is meshed, and the neighboring atomic information of each mesh node is extracted. Combined with the dislocation distance information output by the trained deep learning network, unsupervised clustering is used to identify the dislocation location. This method achieves automated and high-precision identification of dislocation defects in face-centered cubic materials, while also enabling dynamic characterization of crystal defect deformation behavior.

Description

Technical Field

[0001] This invention belongs to the field of image recognition and machine learning, and specifically relates to a method and system for identifying dislocation defects based on image recognition and deep learning networks. Background Technology

[0002] Dislocations are defects arising from atoms or ions within a crystal deviating from their equilibrium positions, and are the primary carriers of plastic deformation in materials. Dislocations and interfaces containing dislocations are the main defects in crystalline materials, and their type, density, mobility, and interactions directly determine key mechanical properties such as strength, plasticity, and toughness. Controlling material properties by regulating the mechanical behavior of dislocations is a fundamental means of strengthening and toughening materials such as metals and ceramics. Therefore, to achieve precise control of material mechanical properties, reliable prediction of service life, and high-precision monitoring of structural evolution during ultra-precision machining, it is essential to achieve quantitative and rapid characterization of the mechanical behaviors of dislocations and dislocation-containing defects, including their type, location, and migration, at the atomic scale.

[0003] Currently, atomic-scale characterization of dislocation types, locations, and evolution processes still faces numerous challenges. The main reason is that existing atomic-scale dislocation analysis techniques heavily rely on the three-dimensional spatial coordinates of atoms, while experimental characterization methods typically only provide two-dimensional atomic coordinate information. For example, the most widely used dislocation analysis (DXA) algorithm identifies atomic position shifts by comparing the crystal atomic structure before and after deformation with an ideal lattice template, and then converts these shifts into lattice distortions to identify dislocation lines and their Burgers vectors; three-dimensional atomic coordinates are essential information for this. Although these coordinates can be obtained through atomic-scale computational simulations, current mainstream experimental methods used for characterizing dislocation deformation in actual materials, such as in-situ high-resolution transmission electron microscopy and scanning transmission electron microscopy, can only obtain the two-dimensional structure of dislocations within a certain projection plane, and cannot provide the out-of-plane atomic coordinate information required by existing dislocation analysis methods. This makes the dislocation analysis of experimental images and videos highly dependent on human experience, resulting in limited accuracy and low efficiency. On the other hand, while transmission electron microscopy (TEM) with a rotatable sample holder can roughly reveal the three-dimensional morphology of dislocation lines through contrast changes, its characterization accuracy is limited. Considering that the influence of the dislocation core on the arrangement of nearby atoms is usually limited to within a few lattice spaces, this technique is difficult to achieve atomic-scale characterization of its position and behavior. In addition, the emerging atomic electron tomography (AET) technique can reconstruct the three-dimensional atomic arrangement of nanoparticles at atomic resolution, but this technique has a long scanning time and limited temporal resolution, making it difficult to dynamically track and analyze the deformation behavior of dislocations. Currently, it is only applied to the static structure and composition analysis of nanoparticles.

[0004] In summary, the problems with current technologies can be categorized as follows: 1) Existing dislocation analysis techniques rely on three-dimensional atomic coordinates, making it difficult to analyze the core location and type of dislocations based on two-dimensional images. 2) Dislocation analysis based on existing high-resolution characterization techniques is highly dependent on human experience, making it difficult to perform on a large scale with high precision. 3) Existing machine learning characterization methods require the construction of specific training data, resulting in high application barriers and poor generalization ability across different material systems. Currently, there is a lack of methods and systems capable of dynamically characterizing dislocations at the atomic scale based on two-dimensional atomic coordinates. Summary of the Invention

[0005] The purpose of this invention is to solve the above-mentioned problems in the prior art and to provide a method and system for identifying dislocation defects based on image recognition and deep learning networks.

[0006] The specific technical solution adopted in this invention is as follows:

[0007] In a first aspect, the present invention provides a method for identifying dislocation defects based on image recognition and deep learning networks, comprising:

[0008] S1. After calibrating the size of each frame of electron microscopy image in the high-resolution electron microscopy video of crystal defects of face-centered cubic material, the atomic coordinates are identified by a deep learning target detection model. Then, the planar distance between each atom and multiple nearest neighbor atoms in the calibrated image is calculated and used as the local atomic environment vector of that atom.

[0009] S2. Obtain a generalized training sample set of atomic models containing dislocation defects covering different materials, dislocation types and deformation states, and train a deep learning network for each type of dislocation, so that it can predict and output the nearest plane distance between the atom and the single type of dislocation, using the local atomic environment vector of each atom in the dislocation model as input.

[0010] S3. For each frame of calibrated image, perform grid division. For each grid node, extract the planar distance between the grid node and multiple nearest neighbor atoms and use it as the first distance vector. At the same time, for each dislocation type, input the local atomic environment vectors of all the nearest neighbor atoms in the calibrated image into the deep learning network trained for the current dislocation type. Construct the second distance vector by the nearest planar distance between all the nearest neighbor atoms and the dislocation of the current type. Then, perform unsupervised clustering analysis on all grid nodes based on the Euclidean distance between the second vector and the first vector. Use the clusters obtained by clustering as the dislocations identified under the current dislocation type.

[0011] As a preferred embodiment of the first aspect above, the high-resolution electron microscopy video of crystal defects is an electron microscopy video of face-centered cubic metal or alloy material obtained by high-resolution transmission electron microscopy, scanning transmission electron microscopy or electron tomography imaging equipment.

[0012] As a preferred embodiment of the first aspect, the local atomic environment vector consists of the planar distances of 10 to 30 nearest neighbor atoms, and the local atomic environment vector needs to be regularized and normalized before being input into the deep learning network.

[0013] As a preferred embodiment of the first aspect, the generalized training sample set is obtained through simulation calculation. Specifically, the method is as follows: a series of atomic-scale models covering different dislocation types are constructed, and different dislocation configurations are introduced into each atomic-scale model through molecular dynamics relaxation and molecular static energy minimization calculations, and then stress is applied to deform the model to obtain a series of dislocation models with different dislocation types and different deformation states; the dislocation types and dislocation core coordinates contained in each dislocation model are extracted, and training samples are further constructed and added to the training sample set; in each training sample, the local atomic environment vector of each atom in the dislocation model is used as the sample input, and the nearest plane distance between the atom and the single type of dislocation is used as the sample label.

[0014] As a preferred embodiment of the first aspect above, when constructing the generalized training sample set, the atomic-scale model covers two types: a bicrystalline quasi-two-dimensional model with an orientation difference ranging from 8° to 78° and a polycrystalline model with random orientation. The material includes at least two types: pure Au and pure Ni, and the dislocation types cover at least 1 / 2... <110> Total dislocation, 1 / 6 <112> Shockley's incomplete dislocation, 1 / 6 <110> Ladder rod misalignment, 1 / 3 <100> Hess dislocation and 1 / 3 <111> Frank dislocations are subjected to stresses of at least shear, uniaxial tension, multiaxial tension, and compression.

[0015] As a preferred embodiment of the first aspect above, for each type of dislocation, after training the deep learning network using the generalized training sample set, if the prediction error still exceeds the allowable error threshold, it is necessary to re-perform the simulation calculation, construct an incremental learning sample set for the actual element type and dislocation configuration corresponding to the high-resolution electron microscopy video of the crystal defect, and perform transfer learning fine-tuning on the trained deep learning network.

[0016] As a preferred embodiment of the first aspect, the deep learning network trained for each type of dislocation is in the form of an integrated neural network group, which contains multiple independently trained multilayer feedforward neural networks. The average of the outputs of all multilayer feedforward neural networks is used as the final output of the integrated neural network group.

[0017] As a preferred embodiment of the first aspect, when performing the unsupervised clustering analysis for each type of dislocation, all grid nodes with an Euclidean distance less than the upper limit of the threshold are first screened in the calibrated image. If there are grid nodes that meet the screening criteria, DBSCAN clustering is performed in batches or by merging according to the number of nodes. The clusters that exceed the minimum size obtained by clustering are taken as the dislocations identified under the current dislocation type. If no clusters exceeding the minimum size are obtained or no grid nodes that meet the screening criteria are found, it is directly determined that there are no dislocation defects of the current dislocation type in the calibrated image.

[0018] Secondly, the present invention provides a dislocation defect identification system based on image recognition and deep learning networks, comprising:

[0019] The data input module is used for users to input high-resolution electron microscopy videos of crystal defects collected from face-centered cubic materials;

[0020] The result output module, according to the dislocation defect identification method based on image recognition and deep learning network described in any of the first aspects above, identifies dislocations of different types for each frame of the high-resolution electron microscopy video of crystal defects, and then outputs the number, type and coordinate information of dislocations contained in each frame of the image in a structured manner according to the output mode specified by the user, or visualizes the displacement changes of dislocations in the time dimension to show the dynamic evolution behavior of dislocation defects.

[0021] Thirdly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0022] The memory is used to store computer programs;

[0023] The processor is configured to, when executing the computer program, implement the dislocation defect identification method based on image recognition and deep learning networks as described in any of the first aspects above.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] 1) Existing dislocation analysis methods heavily rely on the three-dimensional coordinates of each atom, while current high-resolution electron microscopy characterization techniques typically only provide the two-dimensional coordinates of atoms, resulting in limited and inefficient dislocation analysis capabilities on experimental images. To address this, this invention proposes a machine learning-based dislocation identification and dynamic evolution analysis model, capable of rapidly obtaining the core location of dislocations and determining the dislocation type using two-dimensional images as input, achieving 1 / 6 <112> 1 / 3 <001> and 1 / 6 <110> The characterization of major dislocations in face-centered cubic metals shows that the average error in their position prediction is less than 0.5 Å, achieving atomic-level precision.

[0026] 2) Traditional machine learning models have limited generalization capabilities across different material systems. Their training and deployment require targeted modeling and demand a high level of expertise from users. This invention constructs a generalization training sample set covering atomic models containing dislocation defects in different materials, dislocation types, and deformation states, and uses this set to train a generalization deep neural network. Using the same deep neural network group, atomic-scale dislocation characterization of most face-centered cubic metals can be achieved, demonstrating reliable generalization capabilities. For tasks requiring higher accuracy, this invention can further enhance dislocation characterization efficiency through transfer learning fine-tuning, achieving a prediction accuracy improvement of over 50%.

[0027] 3) This invention enables precise characterization of each dislocation type and location during the dynamic deformation process of defects containing multiple and various types of dislocations. Using this as basic information, dynamic characterization of the deformation behavior of crystal defects at larger scales, such as grain boundaries, can be achieved. This invention provides a rapid characterization tool with atomic-scale resolution for dislocation-dominated plastic processes in crystalline materials such as metals and ceramics, contributing to a deeper understanding of the deformation behavior of dislocation-dominated materials. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the steps of a dislocation defect identification method based on image recognition and deep learning networks.

[0029] Figure 2 A schematic diagram of the module composition of a dislocation defect identification system based on image recognition and deep learning networks;

[0030] Figure 3 This is a schematic diagram of the structure of a computer electronic device;

[0031] Figure 4 This is a flowchart illustrating the dislocation defect identification method in this embodiment of the invention.

[0032] Figure 5 This is a schematic diagram of an integrated neural network group with transfer learning function in an embodiment of the present invention;

[0033] Figure 6 Here is a flowchart of unsupervised clustering analysis in an embodiment of the present invention, where (a) is the overall clustering analysis process; and (b) is the DBSCAN clustering flowchart.

[0034] Figure 7 The above are the atomic-scale characterization results of the dislocation array in metallic copper (Cu) in the embodiments of the present invention, where (a) is 1 / 6 of the high-resolution image. <112> Dislocation core location identification effect; (b) is 1 / 6 <112> (c) shows the true location of the dislocation core; (c) compares the true and predicted values ​​of the dislocation core coordinates.

[0035] Figure 8 This is a dynamic characterization of the dislocation migration behavior of metallic copper (Cu) in an embodiment of the present invention, wherein (a) is a schematic diagram of cyclic shear loading; and (b) is a comparison between the predicted and actual values ​​of grain boundary displacement during cyclic deformation.

[0036] Figure 9 The above are the characterization results of various types of dislocations in metallic gold (Au) in the embodiments of the present invention, wherein (a) is 1 / 3 <001> Dislocation; (b) is 1 / 6 <110> Dislocation; (c) is the actual location of the dislocation. Detailed Implementation

[0037] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0038] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.

[0039] This invention provides a dislocation defect identification method based on image recognition and deep learning networks. Based on two-dimensional image recognition and a deep neural network trained by generalized sample data, this method achieves high-precision localization of dislocation cores and rapid classification of dislocation types. It can be directly used for dislocation analysis of common face-centered cubic metals, accurately and reliably extracting dislocation information from high-resolution electron micrographs of crystal defects, and realizing precise tracking and synchronous analysis of different dislocation types and positions during material dynamic deformation and complex defect evolution.

[0040] See Figure 1 As shown, in a preferred embodiment of the present invention, the specific steps of the above-mentioned dislocation defect identification method based on image recognition and deep learning network include S1 to S3. The specific implementation of each step is described in detail below.

[0041] S1. After calibrating the size of each frame of electron micrograph in the high-resolution electron microscopy video of crystal defects of face-centered cubic materials, the atomic coordinates are identified by a deep learning target detection model. Then, the planar distance between each atom and multiple nearest neighbor atoms in the calibrated image is calculated and used as the local atomic environment vector of that atom.

[0042] It should be noted that the face-centered cubic material in this invention can be any face-centered cubic metal or alloy, such as elemental metals or alloys of Au, Ni, Ag, Al, Cu, etc. For high-resolution electron microscopy of crystal defects in face-centered cubic materials, the target material containing crystal defects can first be prepared into a sample. Then, data can be acquired using any microscopic characterization equipment capable of obtaining atomic plane coordinates, such as an in-situ high-resolution transmission electron microscope, scanning transmission electron microscope, or electron tomography imaging device. In-situ high-resolution transmission electron microscopes are preferred. Furthermore, during high-resolution electron microscopy of crystal defects on the sample, an external load required for the experiment can be applied during the recording process, and high-resolution electron microscopic image frames of crystal defects at different times can be captured to facilitate subsequent dynamic identification of defect evolution behavior. In embodiments of this invention, the applied external load can be in the form of tension, compression, etc.

[0043] It should also be noted that the image size calibration in this invention requires ensuring that the lattice spacing within the calibrated image in the image coordinate system equals the theoretical or calculated value of the lattice spacing in the actual face-centered cubic material. During frame-by-frame size calibration, a mapping relationship between image pixel values ​​and real spatial distances can be established by calibrating the ratio between the spacing of different atomic rows in the image and the theoretical lattice spacing, and then the image is scaled. Specifically, since resolution, camera intrinsic and extrinsic parameters, and other factors affect the size of a single pixel during image capture, it is necessary to measure the lattice spacing of the image frames in the image coordinate system after obtaining the high-resolution electron microscopy video of crystal defects. Then, based on the measured lattice spacing value in the image coordinate system and the corresponding theoretically calculated lattice spacing value, the image scaling ratio is determined. Finally, each frame of the electron microscopy image in the high-resolution electron microscopy video of crystal defects is scaled according to this ratio. For ease of description, the scaled image is referred to as the calibrated image. Therefore, the image distance directly measured from the calibrated image corresponds to the physical size in the actual face-centered cubic material without further conversion.

[0044] It should also be noted that, theoretically, any deep learning object detection model can be used to identify the atomic coordinates of the calibrated image, such as the general YOLO model, Faster R-CNN model, RetinaNet, etc. However, due to the specific characteristics of the image targeted in this invention, it is recommended to use the specific model AtomSegNet, which is suitable for identifying atoms. AtomSegNet adopts an encoder-decoder structure based on the U-Net architecture, which can effectively remove background noise and preserve atomic-level details. The deep learning object detection model can extract the image coordinates of all atoms in the image in a two-dimensional Cartesian coordinate system by identifying the atoms in the calibrated image. Since each atom is a two-dimensional circular region in the calibrated image, its image coordinates can be the image coordinates of the center point of the circular region.

[0045] In embodiments of the present invention, when calculating the local atomic environment vector of each atom in the calibrated image, the number m of nearest neighbor atoms to be included in the calculation range can be optimized according to actual conditions, and m is preferably 10~30. Based on the preset m, with each atom as the central atom, the nearest neighbor atoms 1, 2, ..., are calculated sequentially. The planar distances between the atoms are denoted as follows: This allows us to obtain the local atomic environment vector of the atom. For each atom in each frame of the calibrated image, its local atomic environment vector needs to be calculated to facilitate defect prediction in the subsequent deep learning network. Following standard preprocessing methods for model input data, the local atomic environment vector needs to undergo regularization and normalization before being input into the deep learning network. During regularization, each... Each element in the vector needs to be divided by the last element of the vector. , conversion to obtain When performing normalization, Z-score normalization can be performed on a vector-by-vector dimension for the local atomic environment vectors of all atoms in all calibrated image frames.

[0046] S2. Obtain a generalized training sample set of atomic models containing dislocation defects covering different materials, dislocation types and deformation states, and train a deep learning network for each type of dislocation, so that it can predict and output the nearest plane distance between the atom and the single type of dislocation, using the local atomic environment vector of each atom in the dislocation model as input.

[0047] It should be noted that the generalized training sample set in this invention needs to be as generalized as possible, that is, it needs to cover sample data under various possible materials, dislocation types, and stress-deformation states. Theoretically, these training samples can be obtained through experimental measurements, but considering the large amount of data and the high difficulty of obtaining it, the embodiments of this invention construct the generalized training sample set through simulation calculations. The specific methods for obtaining the generalized training sample set are shown in 1) and 2) below:

[0048] 1) Using atomic-scale modeling software such as Atomsk, Matlab, and Ovito, a series of atomic-scale models covering different dislocation types are constructed. Different dislocation configurations are introduced into each atomic-scale model through molecular dynamics relaxation and molecular static energy minimization calculations. Then, stress is applied to the models after introducing dislocation configurations to induce deformation, thereby obtaining a series of dislocation models with different dislocation types and different deformation states (including the undeformed case). The materials, dislocation types, and deformation states in different dislocation models can all be different to enrich the sample types as much as possible.

[0049] In embodiments of the present invention, when constructing the generalized training sample set, the atomic-scale model covers two types: a bicrystalline quasi-two-dimensional model with an orientation difference ranging from 8° to 78° and a randomly oriented polycrystalline model. The material includes at least two types: pure Au and pure Ni, and the dislocation type covers at least 1 / 2. <110> Total dislocation, 1 / 6 <112> Shockley's incomplete dislocation, 1 / 6 <110> Ladder rod misalignment, 1 / 3 <100> Hess dislocation and 1 / 3 <111> Frank partial dislocations are subjected to stresses including at least shear, uniaxial tension, multiaxial tension, and compression. The resulting atomic-scale models can encompass atomic models containing dislocation defects in various materials, dislocation types, and deformation states. It's important to note that bicrystalline and polycrystalline models of pure Au and Ni are used for modeling and obtaining sample data because Au (lattice constant = 0.408 nm) and Ni (lattice constant = 0.352 nm) have a large range of lattice constants. Using only these two elements can broaden the generalization performance of deep learning models for metals with different lattice constants. However, theoretically, simulation data from other types of metallic elements (such as Ag, Al, Cu) and alloy models can be further incorporated as samples.

[0050] 2) After obtaining a series of dislocation models, dislocation types and dislocation core coordinates are analyzed. The dislocation types and dislocation core coordinates contained in each dislocation model are extracted. Then, training samples are constructed and various training samples are added to the training sample set to form the aforementioned generalized training sample set. Each training sample contains sample input and sample label (Groundtruth). The sample input is the local atomic environment vector of each atom in the dislocation model, and the sample label is the nearest plane distance between the atom and the single type of dislocation.

[0051] Specifically, for each atom in all calibrated image frames, theoretically, a training sample can be constructed. The sample label requires first calculating the image distance between that atom and each nearest type of dislocation. Assuming there are e types of dislocations, the image distance between that atom and each nearest type of dislocation is calculated (the image distance for the k-th type of dislocation is denoted as...). (k=1,2,……,e), and then these e image distances are constructed as atom-dislocation relative position vector in vector form. It is important to note that a single calibrated image frame may contain multiple dislocations of a certain type. In this case, it is necessary to calculate the image distances between the atom and all dislocations of that type, and then record the closest distance in the atom-dislocation relative position vector. Similarly, a single calibrated image frame may not contain all type e dislocations. Therefore, for dislocation types not covered in the calibrated image frame, the aforementioned image distance cannot actually be calculated. In this case, the image distance can be directly assigned a default value far exceeding the conventional distance level, for example, 99999. This is because in actual face-centered cubic materials, due to the limited sample size for electron microscopy characterization, the distance between atoms and dislocations cannot reach 99999 angstroms. For each atom in all calibrated image frames, e training samples corresponding to type e dislocations can be constructed, with their sample inputs being... The sample label corresponding to the training sample of the k-th type dislocation is... .

[0052] It should be noted that when performing dislocation type and dislocation core coordinate analysis on the dislocation model, it is preferable to use the dislocation analysis (DXA) algorithm of the Ovito open-source software. The dislocation core can be selected as the centroid of the dislocation, that is, the centroid coordinates of each dislocation are used as the dislocation core coordinates for analysis.

[0053] Once the generalized training sample set is obtained, the deep learning network can be trained to predict and output the nearest plane distance between the atom and a single type of dislocation (corresponding to the dislocation type applicable during the current deep learning network training), using the local atomic environment vector of each atom in the dislocation model as input. It is important to note that since this invention needs to identify multiple different dislocation types, a separate deep learning network suitable for each dislocation type needs to be trained. The type of deep learning network is not limited; in the embodiments of this invention, a fully connected feedforward neural network is preferred, but convolutional neural networks or other deep neural network models can also be used, as long as they have neurons capable of freezing and updating weights and biases.

[0054] To ensure the reliability of model predictions, the deep learning network in this embodiment is designed as an ensemble neural network. Specifically, the deep learning network trained for each dislocation type is not a single network, but rather an ensemble neural network composed of multiple independently trained multilayer feedforward neural networks. Each multilayer feedforward neural network in the ensemble neural network takes the local atomic environment vector of each atom in the dislocation model as input, performs predictions independently, and outputs the nearest plane distance between that atom and the dislocation of that single type. However, the nearest plane distance output by all multilayer feedforward neural networks in the ensemble neural network group is... The average needs to be taken and then used as the nearest plane distance between the atom and the single-type dislocation in the final output of the integrated neural network group.

[0055] The training process of deep learning networks is existing technology and will not be elaborated further. However, since each trained deep learning network needs to be differentiated based on the applicable dislocation type, the training sample data also needs to be pre-selected according to the dislocation type. A sub-sample dataset for each dislocation type can be used to train a deep learning network suitable for that dislocation type. The loss function for training deep learning networks can be set as average error loss or mean squared error loss, etc., with the goal of minimizing the loss value. The learnable parameters are continuously updated iteratively until the model converges or reaches the maximum number of iterations, at which point the process terminates.

[0056] In the embodiments of this invention, since the materials in the generalized training sample set actually only include pure Au and pure Ni, it may be difficult to achieve good prediction accuracy in some special cases. Therefore, for each dislocation type, if the prediction error still exceeds the allowable error threshold after training the deep learning network using the generalized training sample set, it is necessary to re-perform the simulation calculation. An incremental learning sample set is constructed based on the actual element type and dislocation configuration corresponding to the high-resolution electron microscopy video of crystal defects, and the trained deep learning network is fine-tuned through transfer learning. The method of constructing the incremental learning sample set is similar to that of the aforementioned generalized training sample set, but it mainly focuses on simulating the element type and dislocation configuration corresponding to the currently identified material sample to obtain targeted training samples. That is, if the training samples for the actual corresponding element type and dislocation configuration are not included or are few in number in the generalized training sample set, a new molecular dynamics simulation is carried out by modifying the lattice constant, the type of chemical element used in the simulation, etc., to construct a small incremental learning sample set (the sample size is usually less than 1000). This dataset is a personalized dataset for the current actual sample.

[0057] The aforementioned transfer learning fine-tuning is based on the deep learning network that has been pre-trained on a large-scale generalized training sample set. In order to enable the model to converge as soon as possible, the weights and biases of the shallowest hidden layers in the network can be frozen, and only the remaining hidden layers are allowed to update parameters to complete the transfer learning on the personalized dataset.

[0058] S3. For each frame of calibrated image, perform grid division. For each grid node, extract the planar distance between the grid node and multiple nearest neighbor atoms and use it as the first distance vector. At the same time, for each dislocation type, input the local atomic environment vectors of all the nearest neighbor atoms in the calibrated image into the deep learning network trained for the current dislocation type. Construct the second distance vector by the nearest planar distance between all the nearest neighbor atoms and the dislocation of the current type. Then, perform unsupervised clustering analysis on all grid nodes based on the Euclidean distance between the second vector and the first vector. Use the clusters obtained by clustering as the dislocations identified under the current dislocation type.

[0059] It should be noted that when dividing the calibrated image into a grid in this invention, the specific grid size can be adjusted according to the actual situation. Theoretically, the smaller the grid, the higher the accuracy, but the greater the computational load. Therefore, it is preferable to use an interval of 0.5 angstroms to divide the grid evenly in both the horizontal and vertical directions, obtaining uniformly distributed 0.5 angstrom × 0.5 angstrom grid nodes.

[0060] Additionally, it should be noted that when extracting the first distance vector for each grid node, the number of nearest neighbor atoms *n* can be optimized and adjusted based on actual conditions. For the grid node *N* in the *i*th row and *j*th column... ij In other words, it can be searched for with grid node N ij Find the 1st, 2nd, ..., nth nearest neighbor atoms, and then calculate the grid node N. ij The planar distances to these n nearest neighbor atoms are denoted as follows: ,…, Thus, the grid node N is obtained. ij First distance vector ,…, And grid node N ij The second distance vector is constructed separately for each dislocation type. For the k-th dislocation type, its grid nodes N ij The second distance vector is denoted as The second distance vector We need to obtain the local atomic environment vectors of the aforementioned n nearest neighbor atoms (the local atomic environment vectors of each atom in the calibrated image have already been obtained in step S1). Then, we input the local atomic environment vectors of these n nearest neighbor atoms into a deep learning network trained for the k-th dislocation type to obtain the nearest plane distance between each nearest neighbor atom and the k-th type of dislocation. The nearest plane distance of n nearest neighbor atoms Combined into a vector form, denoted as ,…, ,in Represents grid node N ij The The nearest plane distance between the nearest neighbor atom and the k-th type of dislocation .

[0061] Therefore, for the kth type of dislocation, for each grid node N ij The first distance vector can be calculated from both. Second distance vector Calculate the Euclidean distance between the two. This can be used to represent the grid node N. ijThe probability of a dislocation belonging to the k-th dislocation type. Based on the dislocation probabilities of all grid nodes in each frame of the calibrated image, a dislocation probability distribution map can be obtained for each frame of the calibrated image. Since dislocations in an image often exhibit spatial continuity, clustering the dislocation probabilities of all grid nodes in this dislocation probability distribution map can identify dislocation defects. However, the distance between a grid node and a nearby dislocation nucleus generally has an upper limit; therefore, grid nodes that do not meet the criteria can be pre-selected before clustering. Before implementing unsupervised clustering analysis, a Euclidean distance can be pre-defined. upper limit threshold At the same time, a minimum cluster size N representing dislocations is defined. cluster When performing cluster analysis for any k-th dislocation type, the Euclidean distance is first selected from the calibrated image. Less than the upper limit of the threshold If none of the grid nodes meet the filtering criteria (i.e., all grid nodes are greater than or equal to the upper threshold), then... If the k-th type of dislocation defect does not exist in the calibrated image, then it is directly determined that there is no dislocation defect in the calibrated image; if there are grid nodes that meet the screening criteria, that is, there is partial Euclidean distance. Less than the upper limit of the threshold For the grid nodes, unsupervised clustering (preferably DBSCAN clustering) is performed by selecting either batching or merging based on the number of nodes that meet the criteria, with the clustering exceeding the minimum size N. cluster The clusters are the dislocations identified under the k-th dislocation type. If the number of clusters does not exceed the preset minimum size N, then... cluster If the cluster is identified, then it is determined that there is no dislocation defect of the kth type in the calibrated image.

[0062] It should be noted that the clustering algorithm used in the unsupervised clustering analysis of this invention can be the DBSCAN algorithm, or density-based clustering methods such as OPTICS (Ordering Points To Identify the Clustering Structure), distribution-based Gaussian mixture model clustering algorithm, graph theory-based spectral clustering algorithm, and other clustering algorithms. In the embodiments of this invention, the DBSCAN algorithm is preferred because it is simple and efficient, and can complete the clustering task required by this invention with relatively low computational cost.

[0063] Additionally, it should be noted that when selecting batch or merging methods for unsupervised clustering based on the number of eligible grid nodes, the performance of the data processing device executing the method, such as memory resources, needs to be considered. For ordinary personal computers, if the number of eligible grid nodes is too large, batch clustering can be considered to reduce the resources required for a single clustering calculation. The batch clustering results can then be merged across batches to obtain the final clustering result. Conversely, if the number of eligible grid nodes is small, all grid nodes can be merged and clustered at once, provided that memory resources are sufficient.

[0064] In this invention, the clusters obtained from unsupervised clustering analysis must satisfy the condition that the number of grid nodes contained in the cluster exceeds a preset minimum size N. cluster Only when this is done can it be considered a dislocation defect, avoiding the incorrect identification of a large number of meaningless noise points. For a cluster identified as a dislocation defect, the average coordinates of all grid nodes in the calibrated image can be calculated and used as the dislocation core coordinates of the dislocation. At the same time, its dislocation type also needs to be recorded.

[0065] In embodiments of the present invention, for each calibrated image frame, the type and coordinate information of the dislocations identified in the image can be statistically analyzed and stored as a structured vector. One optional form is: dislocation number, dislocation core x-coordinate, dislocation core y-coordinate, and dislocation type. Furthermore, since high-resolution electron microscopy videos of crystal defects are recorded in time sequence, the dislocations identified in each calibrated image frame can actually reflect the dynamic evolution process and can be visualized in different ways based on the aforementioned structured information. For example, for a specific target dislocation defect, the displacement change curve of the dislocation defect can be plotted based on the average value of the dislocation core coordinates during the deformation process, thereby describing the overall evolution behavior of the dislocation defect.

[0066] It should be noted that the method steps shown in S1 to S3 above can essentially be implemented in the form of computer programs or software functional modules.

[0067] Therefore, based on the same inventive concept, such as Figure 2 As shown, the present invention also provides a dislocation defect identification system based on image recognition and deep learning networks, corresponding to the dislocation defect identification method based on image recognition and deep learning networks provided in the above embodiments, which includes:

[0068] The data input module is used for users to input high-resolution electron microscopy videos of crystal defects collected from face-centered cubic materials;

[0069] The results output module allows the user to identify dislocations of different types for each frame of the high-resolution electron microscopy video of crystal defects, using the dislocation defect identification method based on image recognition and deep learning networks as described above. Then, according to the output mode specified by the user, the module outputs the number, type, and coordinate information of dislocations in each frame in a structured manner, or visualizes the displacement changes of dislocations in the time dimension to show the dynamic evolution behavior of dislocation defects.

[0070] It should be noted that both the data input module and the result output module can use a GUI interface to provide intuitive user interaction, or they can use commands for input and output.

[0071] Furthermore, based on the same inventive concept, such as Figure 3 As shown, the present invention also provides a computer electronic device corresponding to the dislocation defect identification method based on image recognition and deep learning network provided in the above embodiments, which includes a memory and a processor;

[0072] The memory is used to store computer programs;

[0073] The processor is configured to implement the dislocation defect identification method based on image recognition and deep learning networks as described above when executing the computer program.

[0074] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0075] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium corresponding to the dislocation defect identification method based on image recognition and deep learning networks. The storage medium stores a computer program, which, when executed by a processor, can realize the dislocation defect identification method based on image recognition and deep learning networks as described above.

[0076] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can realize the dislocation defect identification method based on image recognition and deep learning networks as described above.

[0077] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S3.

[0078] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0079] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0080] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0081] The present invention will further demonstrate the detailed implementation process and technical effects of the dislocation defect identification method based on image recognition and deep learning networks shown in steps S1 to S3 on a specific dataset through a specific embodiment, so as to facilitate understanding of the essence of the present invention.

[0082] Example

[0083] The basic flow and principle of this embodiment are the same as the dislocation defect identification method based on image recognition and deep learning networks shown in steps S1 to S3 above, and will not be completely repeated here. This embodiment mainly demonstrates the specific implementation methods, some specific parameter settings, and implementation results of each step. For ease of description, the method shown in steps S1 to S3 will be referred to as the method of this invention. Figure 4 As shown, the dislocation defect identification method based on image recognition and deep learning networks in this embodiment has the following steps:

[0084] Step 1: Using copper (Cu) and gold (Au) as face-centered cubic metal materials, samples were prepared and high-resolution electron microscopy videos of crystal defects were obtained using in-situ high-resolution transmission electron microscopy. Then, the dimensions were calibrated frame by frame to determine the ratio between the spacing of different atomic rows in the image and the theoretical lattice spacing. The mapping relationship between image pixel values ​​and real spatial distances was established. Then, the image frames in the video were scaled so that the lattice spacing in each calibrated image in the image coordinate system was equal to the theoretically calculated value of the lattice spacing in the actual face-centered cubic material.

[0085] The second step is to use the deep learning object detection model AtomSegNet to identify atoms in each frame of the calibrated image and extract the coordinates of all atoms in the calibrated image in a two-dimensional Cartesian coordinate system (based on the image coordinates of the atom center).

[0086] Step 3: For each calibrated image frame, calculate the local atomic environment vector of each atom in the image. The calculation process is as follows: Taking each atom as the central atom, calculate the local atomic environment vector of the atom and its nearest neighbors in sequence, i.e., the 1st, 2nd, ..., ... The planar distance between m atoms is the local atomic environment vector of that atom, which is composed of the planar distances between its m nearest neighbors. Among them, positive integers The truncation factor represents the number of nearest-neighbor atoms that need to be included in the distance calculation. Testing has shown that the truncation factor in this embodiment... The optimal value m is set to 20, meaning that in this embodiment, for each central atom, the planar distances between the 1st to 20th nearest neighbor atoms are considered. Then, for each... Perform regular expression processing, dividing each dimension by The local atomic environment vector is then converted to a regularized form. ,

[0087] Step 4: Once the local atomic environment vector after the above regularization process has been extracted from each atom in all calibrated images... Then, normalization preprocessing can be performed on each of the m vector dimensions to obtain the regularized and normalized local atomic environment vectors. The normalization method uses Z-score normalization, and its calculation formula is as follows: , in Each piece of data Normalized value It is the mean of all the data. This is the standard deviation of all data. It's important to note that... When performing normalization, the m vector dimensions need to be independent, meaning that for any vector dimension, the formula above... and It is necessary to calculate the local atomic environment vectors of all atoms in all calibrated images. The mean and standard deviation of the elements in this vector dimension.

[0088] Step 5: Using atomic-scale modeling software such as Atomsk, Matlab, and Ovito, construct a structure containing 1 / 2 <110> Total dislocation, 1 / 6 <112> Shockley's incomplete dislocation, 1 / 6 <110> Ladder rod misalignment, 1 / 3 <100> Hess dislocation and 1 / 3 <111> This study presents atomic-scale models of the main types of dislocations in face-centered cubic metals, including Frank partial dislocations (hereinafter referred to as Type 1 to Type e dislocations, where e is actually equal to 5). Specifically, using atomic-scale modeling software, a series of bicrystalline quasi-two-dimensional models of pure Au and pure Ni elements (one of the two elements is used for each model) with orientation differences ranging from 8° to 78° are constructed. Then, LAMMPS software is used to perform molecular dynamics relaxation (relaxation time between 10–50 ps) and molecular static energy minimization calculations at 300 K to introduce the aforementioned different types of dislocation structures (i.e., dislocation configurations). On the other hand, polycrystalline models of randomly oriented pure Au and pure Ni elements (one of the two elements is used for each model) are constructed based on the Thiessen polygon method. Molecular dynamics relaxation and energy minimization calculations are then performed to obtain more diverse dislocation structures (i.e., dislocation configurations). Different grains in the aforementioned bicrystalline quasi-two-dimensional and polycrystalline models should have the same tilt axis to ensure that the dislocation lines are as straight as possible in the direction perpendicular to the paper. After obtaining the above dislocation structure using different atomic-scale models, it is possible to achieve the following results in 10-1 9 Dynamic loading calculations were performed under different stress modes, including shear, uniaxial tension, multiaxial tension, and compression, to obtain dislocation models of the dislocation structure under different deformation states. This allows for the simulation of dislocation models covering different materials, dislocation types, and deformation states.

[0089] Step 6: Based on the dislocation analysis (DXA) algorithm of the Ovito open-source software, perform dislocation type and dislocation core coordinate analysis on the above dislocation model, outputting the type of each dislocation and its average position projection in the two-dimensional plane, forming an atom-dislocation model containing atomic coordinates and dislocation core coordinates. This atom-dislocation model is an atomic model containing dislocation defects, from which information can be further extracted to construct the sample data required for training.

[0090] Step 7: Calculate the relative positions of atoms and dislocations in the atom-dislocation model obtained in Step 6. Specifically, for each atom, calculate its nearest planar distance to the dislocations of types 1 to e from Step 5, and store it in an atom-dislocation relative position vector with e elements. That is, for any atom, its atom-dislocation relative position vector... The atom and the nearest 1 / 2 were stored separately. <110> Total dislocation, 1 / 6 <112> Shockley's incomplete dislocation, 1 / 6 <110> Ladder rod misalignment, 1 / 3 <100> Hess dislocation and 1 / 3 <111> The planar distance of Frank partial dislocations, where if a certain type of dislocation does not exist, then the above planar distance is its distance in... The element value in the array is set to 99999, and when multiple dislocations of a certain type exist, the closest planar distance is recorded. .

[0091] Simultaneously, for each atom, it is necessary to calculate, with that atom as the central atom, the first, second, ..., next-neighbor dislocations in the atom-dislocation model. The planar distance between atoms is obtained. The vector is then regularized and normalized according to steps three and four above to obtain the regularized and normalized local atomic environment vector. However, it should be noted that the regularization and normalized local atomic environment vectors here... It is extracted for the atom-dislocation model, and is different from the extraction from the calibrated image in steps three and four. Although the data sources are different, the definitions are the same, and the extraction and calculation methods are also the same.

[0092] Step 8: Based on the data obtained in Step 7, for each atom in the atom-dislocation model, calculate the local atomic environment vector corresponding to each atom. and the relative position vector of atoms and dislocations It can The e-dimensional elements in the data are used as sample labels and (As input samples) Pair them up to construct e training samples. Once training samples covering all atoms in all atom-dislocation models across different materials, dislocation types, and deformation states are obtained, a generalization training sample set can be constructed. The generalization training sample set can be further divided into a series of sub-sample sets according to dislocation type, which can be used for subsequent training of deep learning networks suitable for different dislocation types.

[0093] Step 9: Based on the generalized training sample set from Step 8, train an ensemble neural network based on deep learning for each type of dislocation. For example... Figure 5 As shown, the integrated neural network group of this invention includes M deep learning networks. Specifically, the deep learning networks employ multi-layer feedforward neural networks with the ability to freeze the weights and biases of specific layers. Their topology is 20-20-40-40-20-1, meaning they contain one input layer with 20 nodes, four hidden layers with 20, 40, 40, and 20 nodes respectively, and ultimately output individual elements representing the relative positions of atoms and dislocations. During training, each deep learning network uses the mean-square error (MSE) as the loss function, with an initial learning rate of 0.001, and updates the network parameters using Levenberg-Marquardt. The default training steps are 1000. To prevent overfitting, an early stopping strategy is employed: training ends prematurely when the loss function on the validation set stops decreasing after more than 20 steps. During training, 70% of the training data is randomly divided into a training set, 15% into a validation set, and 15% into a test set. For each dislocation type, M independent training iterations are performed based on a subset of samples of that dislocation type. Each training iteration uses the same network topology, differing only in the random initialization of the training data and hyperparameters. After each training iteration, a multilayer feedforward neural network suitable for that dislocation type is obtained. These M multilayer feedforward neural networks form an ensemble neural network group for that dislocation type. The final prediction result of the ensemble neural network group is determined by the average of the outputs of the M multilayer feedforward neural networks, thus reducing the impact of training randomness on prediction error.

[0094] Step 10: Perform performance testing on the deep ensemble neural network group obtained in Step 9 for each type of dislocation on the test set, obtain the atom-dislocation relative position vector for each atom, and calculate the prediction error based on the truth label.

[0095] Step 11: Initially assess the prediction performance to decide whether to perform transfer learning fine-tuning. Specifically, set the machine learning prediction error threshold based on the lattice spacing of the material system. In this embodiment, the default setting is Root Mean Square Error (RMSE) = 1 Å, because the error corresponding to the predicted dislocation core position is generally within 1 / 3 of the lattice constant, which is sufficient to meet the characterization error requirements for most scenarios. If the prediction error RMSE is greater than the prediction error threshold, then perform step 12 on the integrated neural network group; otherwise, directly perform step 13.

[0096] Step 12: Preliminary assessment is made to determine whether the actual elemental type and major dislocation structure of the face-centered cubic metallic material sample corresponding to the high-resolution electron microscopy video of crystal defects from Step 1 are included in the generalization training sample set. If not, or if the sample size is too small, new molecular dynamics simulations are conducted by modifying the lattice constant and the types of chemical elements used in the simulation. This constructs a small number of training samples (usually less than 1000 data points) targeting these elemental types and major dislocation structures, forming an incremental learning sample set. Based on this incremental learning sample set, the deep ensemble neural network obtained in Step 9 is fine-tuned to meet the required prediction performance. During fine-tuning, the weights and biases of the first and second hidden layers in the multilayer feedforward neural network are frozen, allowing only the remaining hidden layers to update their parameters. The training parameters, training set partitioning method, and normalization mode of the fine-tuned model are the same as those in the generalization training dataset from Step 9. Thus, the generalized deep learning model is transferred to a personalized prediction scenario for the current sample, achieving a significant improvement in prediction accuracy.

[0097] Step 13: After the ensemble neural network groups for all dislocation types have met the required prediction performance, traverse each frame of the calibrated image sequentially, and divide the coordinate system uniformly in the horizontal and vertical directions at 0.5 angstroms intervals to obtain a uniformly distributed grid of nodes N. ij , where i and j are the horizontal and vertical indices of the grid node in the calibrated image, respectively. For grid node N in the calibrated image... ij Perform planar distance calculations on each pair of atoms to calculate the mesh node N. ij The planar distance to all nearest neighbor atoms is taken as the distance between grid nodes N. ij The nearest n nearest neighbor atoms (denoted as grid node N) ij The first distance vector is constructed using the planar distances of the n nearest neighbor atoms. ,…, ,in ,…, Representing grid nodes N respectively ij The planar distance to its 1st, 2nd, ..., nth nearest neighbor atom. The number of nearest neighbor atoms considered. After testing, its reasonable value range is [2, 8].

[0098] Simultaneously, for any k-th dislocation type, and reading the data from step three for grid node N... ij The local atomic environment vector is calculated from the n nearest neighbor atoms (the local atomic environment vector here needs to be regularized and normalized). ), invoke an ensemble neural network trained for this dislocation type, and convert the local atomic environment vectors of the n nearest neighbor atoms into their respective local atomic environment vectors. Inputting the data into an ensemble neural network trained for the k-th dislocation type, we obtain the nearest-neighbor distance between each nearest-neighbor atom and the k-th type dislocation. The nearest plane distances of the n nearest neighbor atoms are calculated. Combined into a vector form, denoted as the second distance vector. ,…, The subscript k represents the dislocation type corresponding to the predicted value (k=1~e, respectively, being 1 / 2). <110> Total dislocation, 1 / 6 <112> Shockley's incomplete dislocation, 1 / 6 <110> Ladder rod misalignment, 1 / 3 <100> Hess dislocation and 1 / 3 <111> Frank's non-perfect dislocation.

[0099] Step 14: For each grid node N in each frame of the calibrated image ij Calculate a first distance vector Second distance vector The Euclidean distance between them, which is a scalar. This represents the node-dislocation matching index, which is the index for all grid nodes. This can be used to construct a dislocation probability distribution map.

[0100] Step 15: For each grid node N in each frame of the calibrated image ij Based on its Scalar values ​​are used for unsupervised cluster analysis. For example... Figure 6 As shown in (a), the workflow of unsupervised cluster analysis is as follows: First, set... Scalar threshold upper limit The value is 1 (its physical meaning is) Nodes within the threshold range are no more than 1 angstrom away from nearby dislocation cores. After screening and calibration, the image must meet the screening criteria (grid nodes N). ij of Less than the upper limit of this threshold The algorithm iterates through the grid nodes. If no grid node meets the filtering criteria, it indicates that the k-th type of dislocation may not exist in the calibrated image and returns an empty result. If grid nodes meet the filtering criteria, its number is first determined. If the number is greater than 50,000, batch clustering calculations are performed followed by cross-batch cluster merging. If the number is less than 50,000, all nodes are merged at once and then standard clustering calculations are performed directly. The purpose of determining the number of nodes is to save computational memory resources and ensure that this invention can be used on conventional personal computers with more than 4GB of memory. The clustering algorithm used in this embodiment is the DBSCAN algorithm, such as... Figure 6 As shown in (b), the clustering process of this method is as follows: First, set the neighborhood radius epsilon and the minimum cluster size MinPts parameters, and then traverse all data points (which are grid nodes in this embodiment). For each unvisited point, the number of neighboring points within its epsilon neighborhood is checked. If the number of neighboring points reaches the MinPts threshold, a new cluster is created, and connected points are grouped into the same cluster through density expansion; if the number of neighboring points is less than MinPts, they are marked as noise points. This process is repeated until all points have been processed, and finally, all identified clusters and noise points are output, completing the clustering analysis of the dataset.

[0101] Step 16: If no clusters are found after performing the DBSCAN algorithm on the calibrated image, an empty result is returned. If clusters exist, each cluster is considered a dislocation defect. The average coordinate of all grid nodes in each cluster is calculated and used as the core coordinate of the dislocation. The types and coordinate information of dislocations in the image are statistically analyzed and stored as a structured vector in the form of: dislocation number, dislocation core x-coordinate, dislocation core y-coordinate, and dislocation type.

[0102] Step 17: After identifying and structurally outputting dislocation information for all calibrated images in the high-resolution electron microscopy video of crystal defects, for any grain boundary defect containing dislocations, the coordinates of the dislocation core during its deformation process can be statistically analyzed, and the displacement change curve of the defect can be plotted to describe the overall evolution behavior of the defect.

[0103] In this embodiment, samples were prepared using copper (Cu) and gold (Au) as face-centered cubic metal materials, respectively. Then, high-resolution electron microscopy videos of crystal defects were obtained by in-situ high-resolution transmission electron microscopy, and the above-described method was executed.

[0104] like Figure 7As shown, partial results of atomic-scale characterization of dislocation arrays in Cu samples are presented, where (a) is 1 / 6 of the result based on a high-resolution image. <112> Dislocation core location identification effect; (b) is 1 / 6 <112> The true location of the dislocation core is shown in (c), which compares the true and predicted values ​​of the dislocation core coordinates. For this sample, the deep integrated neural network obtained in step nine was further combined with the approach in step thirteen. Transfer learning fine-tuning was performed using incremental learning training samples constructed from a small amount of Cu 7.5° grain boundary bicrystalline model data. Finally, the deep integrated neural network after transfer learning was obtained, and several 1 / 6 [units of measurement] at the Cu 13.5° grain boundary were [processed / adjusted]. <112> Dislocations were identified and characterized. The final results showed that the represented structure represented a total of 1 / 6 of the actual structure. <112> Dislocations were accurately classified and located, and the dislocation locations highly matched the results shown by dislocation analysis (DXA) based on three-dimensional atomic coordinates. The root mean square error (RMSE) of the predicted dislocation locations was approximately 0.34 Å, indicating that the characterization accuracy of dislocations in this embodiment is close to one-tenth of the Cu lattice constant, demonstrating high precision and high reliability. Furthermore, as... Figure 8 As shown, the dynamic characterization results of dislocation interface migration behavior in Cu samples are presented, where (a) is a schematic diagram of cyclic shear loading; and (b) is a comparison of the predicted and actual values ​​of grain boundary displacement during cyclic deformation. To achieve dynamic characterization of interface migration behavior, the Cu samples were subjected to cyclic shear loading at a rate of 10... 8 The strain rate is adjusted to 6% shear strain, then the shear direction is changed to no shear strain, and then the strain is adjusted back to 6% shear strain, repeating this process ten times. Record all 1 / 6 of the strain during this process. <112> The average vertical displacement of dislocations is taken as the displacement of the grain boundary. Based on the predictions of this invention and the actual results based on three-dimensional dislocation analysis (DXA), a grain boundary displacement-loading step curve was plotted, and a high degree of agreement was found between the two.

[0105] like Figure 9 As shown, the characterization results of various types of dislocations in the Au sample are presented, where (a) is 1 / 3 <001> Dislocation; (b) is 1 / 6 <110> Dislocations; (c) shows the actual location of the dislocations. For this sample, the deep integrated neural network obtained in step nine already meets the prediction performance requirements, and no further transfer learning fine-tuning is needed in step thirteen. Therefore, the deep integrated neural network obtained in step nine can directly predict the 1 / 3 of the Au 14° grain boundary. <001> and 1 / 6 <110> Dislocations can be characterized (or 1 / 6) <112> (Characterizing dislocations). The final results show that the representation structure represents 1 / 3 of the entire structure. <001> and 1 / 6 <110> All dislocations were accurately classified and located, with no omissions or misjudgments.

[0106] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for identifying dislocation defects based on image recognition and deep learning networks, characterized in that, include: S1. After calibrating the size of each frame of electron microscopy image in the high-resolution electron microscopy video of crystal defects of face-centered cubic material, the atomic coordinates are identified by a deep learning target detection model. Then, the planar distance between each atom and multiple nearest neighbor atoms in the calibrated image is calculated and used as the local atomic environment vector of that atom. S2. Obtain a generalized training sample set of atomic models containing dislocation defects covering different materials, dislocation types and deformation states, and train a deep learning network for each type of dislocation, so that it can predict and output the nearest plane distance between the atom and the single type of dislocation, using the local atomic environment vector of each atom in the dislocation model as input. S3. For each frame of calibrated image, perform grid division. For each grid node, extract the planar distance between the grid node and multiple nearest neighbor atoms and use it as the first distance vector. At the same time, for each dislocation type, input the local atomic environment vectors of all the nearest neighbor atoms in the calibrated image into the deep learning network trained for the current dislocation type. Construct the second distance vector by the nearest planar distance between all the nearest neighbor atoms and the dislocation of the current type. Then, perform unsupervised clustering analysis on all grid nodes based on the Euclidean distance between the second distance vector and the first distance vector. Use the clusters obtained by clustering as the dislocations identified under the current dislocation type.

2. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 1, characterized in that, The high-resolution electron microscopy video of crystal defects is an electron microscopy video of face-centered cubic metal or alloy materials obtained by high-resolution transmission electron microscopy, scanning transmission electron microscopy or electron tomography imaging equipment.

3. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 1, characterized in that, The local atomic environment vector consists of the planar distances of 10 to 30 nearest neighbor atoms, and the local atomic environment vector needs to be regularized and normalized before being input into the deep learning network.

4. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 1, characterized in that, The generalization training sample set is obtained through simulation calculation. Specifically, a series of atomic-scale models covering different dislocation types are constructed. Different dislocation configurations are introduced into each atomic-scale model through molecular dynamics relaxation and molecular static energy minimization calculations, and then stress is applied to deform the model, thereby obtaining a series of dislocation models with different dislocation types and different deformation states. The dislocation types and dislocation core coordinates contained in each dislocation model are extracted, and training samples are further constructed and added to the training sample set. In each training sample, the local atomic environment vector of each atom in the dislocation model is used as the sample input, and the nearest plane distance between the atom and the single type of dislocation is used as the sample label.

5. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 4, characterized in that, When constructing the generalization training sample set, the atomic-scale model covers two categories: bicrystalline quasi-two-dimensional models with orientation differences ranging from 8° to 78° and polycrystalline models with random orientations. The materials include at least pure Au and pure Ni, and the dislocation types cover at least 1 / 2 <110> Total dislocation, 1 / 6 <112> Shockley's incomplete dislocation, 1 / 6 <110> Ladder rod misalignment, 1 / 3 <100> Hess dislocation and 1 / 3 <111> Frank dislocations are subjected to stresses of at least shear, uniaxial tension, multiaxial tension, and compression.

6. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 5, characterized in that, For each type of dislocation, after training the deep learning network using the generalized training sample set, if the prediction error still exceeds the allowable error threshold, the simulation calculation needs to be performed again. An incremental learning sample set is constructed for the actual element type and dislocation configuration corresponding to the high-resolution electron microscopy video of the crystal defect, and the deep learning network that has been trained is fine-tuned by transfer learning.

7. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 1, characterized in that, The deep learning network trained for each type of dislocation is in the form of an ensemble neural network group, which contains multiple independently trained multilayer feedforward neural networks. The average of the outputs of all multilayer feedforward neural networks is taken as the final output of the ensemble neural network group.

8. The dislocation defect identification method based on image recognition and deep learning networks as described in claim 1, characterized in that, For each dislocation type, when performing the unsupervised clustering analysis, firstly, all grid nodes in the calibrated image whose Euclidean distance is less than the upper limit of the threshold are selected. If there are grid nodes that meet the selection criteria, DBSCAN clustering is performed in batches or by merging according to the number of nodes. The clusters that exceed the minimum size obtained by clustering are taken as the dislocations identified under the current dislocation type. If no clusters exceeding the minimum size are obtained or no grid nodes that meet the selection criteria are found, it is directly determined that there are no dislocation defects of the current dislocation type in the calibrated image.

9. A dislocation defect identification system based on image recognition and deep learning networks, characterized in that, include: The data input module is used for users to input high-resolution electron microscopy videos of crystal defects collected from face-centered cubic materials; The result output module, according to the dislocation defect identification method based on image recognition and deep learning network as described in any one of claims 1 to 8, identifies dislocations of different types for each frame of the high-resolution electron microscopy video of crystal defects, and then outputs the number, type and coordinate information of dislocations contained in each frame of the image in a structured manner according to the output mode specified by the user, or visualizes the displacement changes of dislocations in the time dimension to show the dynamic evolution behavior of dislocation defects.

10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the dislocation defect identification method based on image recognition and deep learning networks as described in any one of claims 1 to 8.

Images