A crystal defect type identification method, device, electronic device and storage medium

Abstract

This application discloses a method, device, electronic device, and storage medium for identifying crystal defect types, belonging to the field of crystal structure analysis technology. The method includes: obtaining a transmission microscopic image of a defect of a dolomite crystal; adding a defect type label to the defect transmission microscopic image based on a mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image to generate a labeled defect microscopic image; performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; constructing an initial RetinaNet model, using the defect microscopic image as input and the defect type as output, iteratively training the initial RetinaNet model to generate a target crystal defect type identification model; and identifying the type of defect in the crystal to be tested based on the transmission microscopic image of the crystal to be tested. The present invention can solve the technical problem that existing technical solutions cannot accurately locate and identify dolomite crystal defects and defect types.

Description

Technical Field

[0001] The present invention relates to the technical field of crystal structure analysis, and in particular to a method, device, electronic device and storage medium for identifying crystal defect types. Background Art

[0002] Dolomite has a wide spatial and temporal distribution in the ancient geological record. It is a common mineral in ancient rocks and a thermodynamically stable carbonate phase in modern seawater. Its crystal structure preserves evidence of the crystal formation environment, crystallization rate, crystal growth and change characteristics, and fluid characteristics. Crystal structure analysis offers unique advantages in studying the origin of dolomite. It can provide detailed atomic-level structural information and record the microscopic changes in minerals during diagenesis and alteration, thus overcoming the limitations of traditional petrological and geochemical methods. Therefore, dolomite crystal structure analysis is currently considered an effective means of identifying the formation mechanism of dolomite. Crystal structure analysis primarily studies crystal structural parameters such as the degree of order, unit cell parameters, interplanar spacing, and crystal defects. Crystal defects in dolomite can reflect the crystal formation environment and nucleation and growth processes, and can be used to analyze the formation mechanism of dolomite. Crystal defects generally refer to imperfect regions within the crystal structure where the atomic arrangement deviates from the ideal structure and locations where the integrity of the crystal structure is disrupted. Dolomite crystals form different types of defects due to the influence of changes in temperature, pressure, and medium component concentrations in the complex formation environment and nucleation-growth process. The analysis and identification of dolomite crystal defects is a key step in identifying the dolomite formation mechanism and also provides new ideas for studying the origin of dolomite. Therefore, the analysis, positioning and identification of dolomite crystal defects is of vital importance.

[0003] Dolomite crystal defect detection aims to identify and locate abnormal defect areas within an object. Traditional methods for crystal defect detection include visual inspection, corrosion and metallographic microscopy, X-ray diffraction, and electron microscopy. These methods are not only time-consuming and labor-intensive, but also require a high level of technical expertise from the inspector and are susceptible to subjective influences. Due to the diverse types of defects, noise in captured images, and the variety of abnormal areas on the target surface, image processing methods are prone to difficulties in feature modeling and migration, as well as inaccurate feature descriptions, when marking and detecting defects, impacting actual inspection results.

[0004] Therefore, there is an urgent need for a crystal defect type identification method that can quickly and accurately locate crystal defects and identify defect types by processing and analyzing crystal microscopic images. Summary of the Invention

[0005] In view of this, it is necessary to provide a crystal defect type identification method, device, electronic device and storage medium, which can quickly and accurately locate crystal defects and identify defect types by processing and analyzing crystal microscopic images.

[0006] In order to solve the above technical problems, on the one hand, the present invention provides a method for identifying crystal defect types, comprising:

[0007] Obtain transmission microscopic images of defects in dolomite crystals;

[0008] adding a defect type label to the defect transmission microscopic image according to a mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image to generate a labeled defect microscopic image;

[0009] Performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set;

[0010] Constructing an initial RetinaNet model, taking the defect microscopic image as input and the defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set to generate a target crystal defect type recognition model;

[0011] The transmission microscopic image of the crystal to be measured is identified based on the target crystal defect type identification model to obtain the defect location and defect type of the crystal to be measured.

[0012] In a possible implementation, crystal defects are classified according to their spatial dimensions to generate first-order defect types, which include zero-dimensional defects, one-dimensional defects, two-dimensional defects, and three-dimensional defects.

[0013] Each defect type in the first-order defect types is classified according to the imaging characteristics of the transmission microscopy image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type.

[0014] In one possible implementation, adding a defect type label to the defect transmission microscopic image to generate a labeled defect microscopic image includes:

[0015] assigning different defect type labels to each of the first-order defect type and the second-order defect type in turn;

[0016] extracting imaging features of a defect transmission microscopic image of the dolomite crystal;

[0017] determining the dimension of the crystal defect according to the imaging characteristics;

[0018] Based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image, and according to the dimension and imaging characteristics of the crystal defect, determining the first-order defect type and the second-order defect type of the crystal defect;

[0019] According to the first-order defect type and the second-order defect type of the crystal defect, a corresponding defect type label is added to each of the crystal defects to generate a labeled defect microscopic image.

[0020] In one possible implementation, performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set includes:

[0021] Performing an affine transformation on the labeled defect microscopic image based on a generative adversarial network to generate a first enhanced training set;

[0022] Determine the characteristic center area of ​​each defect in the first enhanced training set, perform random sliding sampling around the characteristic center area, and generate a second enhanced training set;

[0023] The unclear images in the first enhanced training set and the second enhanced training set are sharpened to generate a label defect training set.

[0024] In one possible implementation, building the initial RetinaNet model includes:

[0025] Build the original RetinaNet model;

[0026] The ResNet network structure in the original RetinaNet model is replaced with the MobileNetv2 network structure, and the CBAM attention mechanism is added to the original RetinaNet model. The original RetinaNet model is optimized and adjusted to generate the initial RetinaNet model.

[0027] In one possible implementation, iteratively training the initial RetinaNet model based on the label defect training set includes:

[0028] Based on the initial RetinaNet model, feature extraction is performed on defects in the labeled defect training set to generate defect features;

[0029] Outputting a predicted defect category according to the defect characteristics;

[0030] Constructing a defect recognition loss function, wherein the defect recognition loss function includes a defect classification loss function and a defect location loss function;

[0031] Calculating the loss function value between the predicted defect category and the label information according to the defect recognition loss function;

[0032] The initial RetinaNet model is iteratively trained according to the loss function value.

[0033] In one possible implementation, the calculation formula of the defect recognition loss function is:

[0034] ,

[0035] in, is the defect classification loss function, is the defect localization loss function, For all positive and negative samples, For all positive samples, is the number of positive samples, For the The classification loss value corresponding to the sample, that is, The loss value between the predicted result of the sample and the actual label result, For the The regression loss value corresponding to the sample, that is, The regression loss value is the difference between the predicted bounding box coordinates and the true bounding box coordinates of samples.

[0036] In a second aspect, the present invention further provides a crystal defect type identification device, comprising:

[0037] An image acquisition module, used to obtain a transmission microscopic image of defects in dolomite crystals;

[0038] a label image generation module, configured to add a defect type label to the defect transmission microscopic image according to a mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, thereby generating a labeled defect microscopic image;

[0039] A training set generation module, configured to perform image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set;

[0040] a recognition model generation module for constructing an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model based on the labeled defect training set to generate a target crystal defect type recognition model;

[0041] The defect recognition module is used to identify the transmission microscopic image of the crystal to be tested based on the target crystal defect type recognition model to obtain the defect location and defect type of the crystal to be tested.

[0042] In a third aspect, the present invention also provides an electronic device comprising a memory and a processor, wherein the memory is used to store programs and data; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the crystal defect type identification method as described above, and / or to implement the crystal defect type identification as described above.

[0043] In a fourth aspect, the present invention further provides a computer storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the crystal defect type identification method as described above.

[0044] The beneficial effects of the present invention are as follows: first, a defect transmission microscopic image of a dolomite crystal is obtained; based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, a defect type label is added to the defect transmission microscopic image to generate a labeled defect microscopic image; then, the labeled defect microscopic image is image pre-enhanced to generate a labeled defect training set; then, an initial RetinaNet model is constructed, with the defect microscopic image as input and the defect type as output, and the initial RetinaNet model is iteratively trained according to the labeled defect training set to generate a target crystal defect type recognition model; finally, based on the target crystal defect type recognition model, the transmission microscopic image of the crystal to be tested is recognized to obtain the defect location and defect type of the crystal to be tested. The present invention analyzes the transmission microscopic image of the dolomite crystal, analyzes the mapping relationship between the dolomite crystal defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and based on this mapping relationship, adds a defect label to each defect on the transmission microscopic image, and performs image enhancement on the labeled image to generate a training set, and iteratively trains the optimized RetinaNet model according to the training set to generate a final target crystal defect type recognition model. The present invention constructs a RetinaNet model by analyzing the mapping relationship between the defect types of dolomite crystals and the spatial dimensions and imaging characteristics of transmission microscopic images, thereby achieving rapid and accurate positioning of dolomite crystal defects and identification of defect types. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A schematic flow chart of an embodiment of the method for identifying crystal defect types provided by the present invention;

[0047] Figure 2An illustration of the crystal defect types provided by the present invention;

[0048] Figure 3 The present invention provides Figure 1 A flow chart of an embodiment of step S102;

[0049] Figure 4 The present invention provides Figure 1 A flow chart of an embodiment of step S103;

[0050] Figure 5 A schematic diagram comparing the original image and enhanced image of the vacancy defect type in the zero-dimensional defect and the grain boundary defect type in the two-dimensional defect provided by the present invention;

[0051] Figure 6 A comparison diagram of the original defect transmission microscopic image and the image after clarity processing provided by the present invention;

[0052] Figure 7 The present invention provides Figure 1 Flow chart of the first embodiment of step S104;

[0053] Figure 8 This is a schematic diagram of the optimized RetinaNet model structure provided by the present invention;

[0054] Figure 9 The present invention provides Figure 1 Flowchart of the second embodiment of step S104

[0055] Figure 10 A schematic structural diagram of an embodiment of an electronic device for identifying crystal defect types provided by the present invention;

[0056] Figure 11 This is a schematic structural diagram of an embodiment of the crystal defect type identification storage medium provided by the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] In the description of the embodiments of the present invention, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0059] The terms "first," "second," and so on, used in the embodiments of the present invention are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature designated as "first" or "second" may explicitly or implicitly include at least one such feature.

[0060] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] Before describing the embodiments, the following definitions are given for the relevant terms:

[0062] The RetinaNet model is an object detection model consisting of two main components: a backbone network and a feature pyramid network. The backbone network uses a pre-trained convolutional neural network. The feature pyramid network consists of multiple feature layers that contain different spatial and scale information.

[0063] The MobileNetv2 network is a lightweight convolutional neural network architecture designed to improve the efficiency of deep learning models on mobile and embedded devices, especially in resource-constrained environments.

[0064] The CBAM attention mechanism is an attention mechanism module used to enhance the performance of convolutional neural networks. The main goal of CBAM is to improve the perception ability of the model by introducing channel attention and spatial attention, thereby improving performance without increasing network complexity.

[0065] The present invention provides a crystal defect type identification method, device, electronic device and storage medium, which are described below respectively.

[0066] Figure 1 A schematic flow chart of an embodiment of the method for identifying crystal defect types provided by the present invention is shown as follows: Figure 1 As shown, the crystal defect type identification method includes:

[0067] S101, obtaining a transmission microscopic image of defects in a dolomite crystal;

[0068] S102, adding a defect type label to the defect transmission microscopic image according to a mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, to generate a labeled defect microscopic image;

[0069] S103, performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set;

[0070] S104, constructing an initial RetinaNet model, taking the defect microscopic image as input and the defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set to generate a target crystal defect type recognition model;

[0071] S105 , identifying the transmission microscopic image of the crystal to be measured based on the target crystal defect type identification model to obtain the defect location and defect type of the crystal to be measured.

[0072] Specifically, we first collected microscopic images of dolomite crystal defects from different regions and periods, and classified the crystal defects into different defect types according to their different spatial dimensions and microscopic imaging characteristics. We then analyzed the transmission microscopic images of dolomite crystals, and performed defect resolution and defect type resolution on the transmission microscopic images based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic images. We added a label corresponding to the defect type for each defect. On this basis, we used a generative adversarial network (GAN) to perform super-resolution conversion on the labeled transmission microscopic images and expand the dataset to obtain an enhanced training dataset. The training dataset was divided into two parts, 80% for the training set and 20% for the validation set. We then iteratively trained the RetinaNet model, and verified and evaluated the accuracy of the trained RetinaNet model to obtain the final crystal defect type recognition model.

[0073] This embodiment analyzes transmission microscopic images of dolomite crystals and analyzes the mapping relationship between dolomite crystal defect types and the spatial dimensions and imaging characteristics of the transmission microscopic images. Based on this mapping relationship, a defect label is added to each defect in the transmission microscopic image, and image enhancement is performed on the labeled image to generate a training set. The optimized RetinaNet model is iteratively trained based on the training set to generate the final target crystal defect type recognition model. By analyzing the mapping relationship between dolomite crystal defect types and the spatial dimensions and imaging characteristics of the transmission microscopic images, the present invention constructs a RetinaNet model, achieving rapid and accurate positioning of dolomite crystal defects and identification of defect types.

[0074] In some embodiments of the present invention, the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image includes:

[0075] Crystal defects are classified according to their spatial dimensions to generate first-order defect types, which include zero-dimensional defects, one-dimensional defects, two-dimensional defects, and three-dimensional defects.

[0076] Each defect type in the first-order defect types is classified according to imaging features of the transmission microscopy image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type.

[0077] Specifically, first-order defect types include zero-dimensional defects, one-dimensional defects, two-dimensional defects and three-dimensional defects; when the imaging feature of the defect is the position error of a single atom or ion, the spatial dimension is zero-dimensional, and the defect type is a zero-dimensional defect; when the imaging feature of the defect is extension along a specific direction in the crystal structure, the spatial dimension is one-dimensional, and the defect type is a one-dimensional defect; when the imaging feature of the defect in the crystal structure deviates from the periodic and regular arrangement in the form of a regional or layered structure, the spatial dimension is two-dimensional, and the defect type is a two-dimensional defect; when the imaging feature of the defect is the collective absence and aggregation of a large number of atoms or ions, the spatial dimension is three-dimensional, and the defect type is a three-dimensional defect.

[0078] Furthermore, for each first-order defect type, due to different defect characteristics, it is divided into multiple second-order defect types. Each first-order defect type includes one or more second-order defect types. The number of second-order defect types can be increased or decreased according to the defect imaging characteristics. In this embodiment, based on the imaging characteristics of the transmission microscopy image, the zero-dimensional defect is subdivided into three second-order defect types, including vacancies, interstitial ions and impurity atoms. Among them, the imaging characteristic of the vacancy is the existence of a vacant position in the atomic crystal structure; the imaging characteristic of the interstitial ion is the appearance of an ion in the crystal lattice gap at a position where there is no atom in the crystal structure; the imaging characteristic of the impurity atom is the appearance of a foreign atom at the crystal lattice position. One-dimensional defects are further subdivided into three second-order defect types: knife-edge dislocations, screw dislocations, and mixed dislocations. The imaging characteristic of a knife-edge dislocation is a line defect formed by the movement of atoms perpendicular to the crystal surface, specifically inserting a half-atomic plane into an ideal complete crystal structure. A knife-edge dislocation is the edge of a half-atomic plane. A screw dislocation is a line defect formed by the movement of atoms along a spiral path, with the dislocation line parallel to the slip direction. A mixed dislocation is a combination of knife-edge and screw dislocations. Two-dimensional defects are further subdivided into three second-order defect types: grain boundaries, twin boundaries, and stacking faults. The imaging characteristic of a grain boundary is that it appears between two crystal structures with completely different crystallization directions. The imaging characteristic of a twin boundary is that at certain stages during the crystal growth process, due to changes in environmental conditions or internal stress, part of the crystal structure rotates, forming twins. The imaging characteristic of a stacking fault is that at certain stages during the crystal growth process, due to changes in the atomic deposition order, part of the crystal structure undergoes stacking faults. Three-dimensional defects are subdivided into three second-order defect types, including voids, precipitates and fluid inclusions. Among them, the imaging characteristic of voids is the existence of space regions inside the crystal structure that are not occupied by atoms. The existence of voids will affect the mechanical and physical properties of the crystal and reduce the strength and density of the crystal; the imaging characteristic of precipitates is crystalline substances with different compositions or structures separated from the crystal matrix; the imaging characteristic of fluid inclusions is the fluid (such as gas, liquid or gas-liquid mixture) captured and enclosed inside the crystal during the growth process.

[0079] Furthermore, if Figure 2 As shown, Figure 2 The crystal defect type illustration provided by the present invention includes first-order defect types and their corresponding second-order defect types.

[0080] In some embodiments of the present invention, Figure 3 As shown, Figure 3 The present invention provides Figure 1 The flowchart of an embodiment of step S102 includes:

[0081] S301, assigning different defect type labels to each first-order defect type and second-order defect type in turn;

[0082] S302, extracting imaging features of a defect transmission microscopic image of a dolomite crystal;

[0083] S303, determining the dimension of the crystal defect according to the imaging characteristics;

[0084] S304, based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image, determining the first-order defect type and the second-order defect type of the crystal defect according to the dimension and imaging characteristics of the crystal defect;

[0085] S305 , adding a corresponding defect type label to each of the crystal defects according to the first-order defect type and the second-order defect type of the crystal defect, and generating a labeled defect microscopic image.

[0086] Specifically, different defect type labels are assigned to each first-order defect type and second-order defect type. For example, the labels of the four first-order defect types are 1, 2, 3 and 4, respectively. The label of the zero-dimensional defect is 1, and the labels of the three corresponding second-order defect types, vacancies, interstitial ions and impurity atoms, are 1-1, 1-2 and 1-3, respectively. Similarly, the label of the one-dimensional defect is 2, and the labels of the three corresponding second-order defect types, knife dislocations, screw dislocations and mixed dislocations, are 2-1, 2-2 and 2-3, respectively. The label of the two-dimensional defect is 3, and the labels of the three corresponding second-order defect types, grain boundaries, twin boundaries and stacking faults, are 3-1, 3-2 and 3-3, respectively. The label of the three-dimensional defect is 4, and the labels of the three corresponding second-order defect types, voids, precipitates and fluid inclusions, are 4-1, 4-2 and 4-3, respectively. Crystal defects are subdivided into multiple types according to their dimensions and imaging characteristics.

[0087] Furthermore, the imaging features of the defect transmission microscopy image of the dolomite crystal are extracted, the dimension of the crystal defect is judged according to the imaging features of the defect transmission microscopy image, the first-order defect type of the crystal defect is judged according to the dimension, and the second-order defect type of the crystal defect is further judged according to the imaging features. According to the first-order defect type and the second-order defect type, combined with the defect labels corresponding to the above defect types, corresponding first-order defect labels and second-order defect labels are added to each crystal defect.

[0088] This embodiment determines the first-order defect type of the crystal defect based on the dimensions of the defect transmission microscopic image of the dolomite crystal, and determines the second-order defect type of the crystal defect based on the imaging characteristics of the defect transmission microscopic image. The defects are subdivided into multiple types, which can more accurately and clearly analyze the dolomite crystal.

[0089] In some embodiments of the present invention, Figure 4 As shown, Figure 4 The present invention provides Figure 1 The flowchart of an embodiment of step S103 includes:

[0090] S401, performing affine transformation on the labeled defect microscopic image based on a generative adversarial network to generate a first enhanced training set;

[0091] S402, determining the characteristic center area of ​​each defect in the first enhanced training set, performing random sliding sampling around the characteristic center area, and generating a second enhanced training set;

[0092] S403 : performing a clearing process on the unclear images in the first enhanced training set and the second enhanced training set to generate a label defect training set.

[0093] It should be noted that due to the small number of transmission microscopy image samples and the unclearness of some collected images, overfitting and incorrect recognition are prone to occur, resulting in the generated defect images being too single. Therefore, image pre-enhancement is performed here. By using the generative adversarial network (GAN), a GAN model is constructed to perform super-resolution conversion on the defect transmission microscopy images of dolomite crystals and expand the data set.

[0094] Specifically, for the transmission microscopy images of training samples, the following three steps are performed on the transmission microscopy images of training samples through the adversarial generative network GAN. First, the image is affine transformed, and the training data is increased by random flipping and mirroring operations of the image; then, the characteristic center area of ​​the defect is determined and random sliding sampling is performed, such as Figure 5 As shown, Figure 5 The schematic diagram of the comparison between the original image and the enhanced image of the vacancy defect type in the zero-dimensional defect and the grain boundary defect type in the two-dimensional defect provided by the present invention. The first column of images in the figure is the original defect transmission microscopy image, and the second to fourth columns are the enhanced images after affine transformation and slip sampling; finally, the unclear images are clarified according to the super-resolution conversion algorithm, as shown in FIG. Figure 6 As shown, Figure 6 This is a comparison of the original defect transmission microscopic images provided by the present invention and the images after the clarity processing. The first row, A, B, and C, are defect transmission microscopic images of dolomite crystal defects, while the second row, A', B', and C', are defect transmission microscopic images after the clarity processing. In this example, a random affine transformation is first performed on the labeled defect microscopic image. The transformed image is 800×800, and the size of the defect feature in the image is approximately 400×400. The image acquisition window size is 600×600, and multiple random sliding cropping is performed around the center of the defect feature. Ultimately, approximately 1800 multidimensional crystal defect images are obtained, of which 1440 are used as training images and 360 as verification images.

[0095] Furthermore, it can be understood that this embodiment combines the generative adversarial network (GAN) and the optimized RetinaNet model to form a complete target crystal defect type recognition model. In actual use, for the transmission microscopic image of the dolomite crystal to be tested that needs to be identified, when processing it through the generative adversarial network (GAN), it is only necessary to sharpen the transmission microscopic image, and then use the sharpened image as the input of the RetinaNet model to locate defects and identify defect types in the dolomite crystal to be tested.

[0096] In some embodiments of the present invention, Figure 7 As shown, Figure 7 The present invention provides Figure 1 The flowchart of the first embodiment of step S104 in FIG. 1 includes:

[0097] S701, build the original RetinaNet model;

[0098] S702. Use the MobileNetv2 network structure to replace the ResNet network structure in the original RetinaNet model, add the CBAM attention mechanism to the original RetinaNet model, optimize and adjust the original RetinaNet model, and generate an initial RetinaNet model.

[0099] It should be noted that the ResNet network structure with a deep network structure and a large number of parameters in the RetinaNet model is replaced with a lightweight MobileNetv2 network structure as the backbone feature extraction network, thereby reducing computing resource costs and shortening training time, enhancing target detection effects, and eliminating the risk of overfitting; the CBAM attention mechanism is added to the RetinaNet model to improve the model's feature extraction ability and robustness under complex striped backgrounds. On this basis, the Feature Pyramid Network (FPN) is introduced to fuse feature maps of different levels in the backbone network to generate a set of pyramid-shaped feature maps with multi-scale features. This model can effectively perform feature extraction and detection training on images of various scales, thereby improving the accuracy of target detection, such as Figure 8 As shown, Figure 8 This is a schematic diagram of the optimized RetinaNet model structure provided by the present invention, in which the GAN adversarial network is integrated to form a complete optimized RetinaNet model.

[0100] This embodiment optimizes the RetinaNet model through the lightweight MobileNetv2 network structure and the CBAM attention mechanism, thereby improving the accuracy and robustness of the RetinaNet model.

[0101] In some embodiments of the present invention, Figure 9 As shown, Figure 9 The present invention provides Figure 1 The flowchart of the second embodiment of step S104 includes:

[0102] S901. Based on the initial RetinaNet model, extract features of defects in the labeled defect training set and generate defect features.

[0103] S902. Output the predicted defect category based on the defect characteristics;

[0104] S903: Construct a defect recognition loss function, which includes a defect classification loss function and a defect location loss function.

[0105] S904. Calculate the loss function value between the predicted defect category and the label information based on the defect recognition loss function;

[0106] S905: Iteratively train the initial RetinaNet model according to the loss function value.

[0107] It should be noted that the FocalLoss loss function is added to the RetinaNet model to fine-tune its attention, giving priority to details that are easily overlooked but crucial to overall recognition accuracy, and solving the problems of imbalanced number of positive and negative samples and difficulty in distinguishing feature types during the detection process. The formula of the FocalLoss loss function is:

[0108] ,

[0109] in, is the loss function value, is the estimated probability of the RetinaNet model, γ is the adjustable focus parameter, is the modulation factor.

[0110] It should be further explained that the classification loss function ( ) predicts the target category and uses the bounding box regression loss function ( ) to accurately locate the target position.

[0111] Specifically, this implementation iteratively trained the initial RetinaNet model using 1440 training images labeled with defect types. The loss function was calculated based on the RetinaNet model's output labeling and the actual defect labels. The model's hyperparameters were then adjusted and optimized based on the loss function to obtain the final target defect type detection model. Finally, defect location and defect type identification were performed on 360 crystal defect verification images. The accuracy and rate of crystal defect location and identification before and after model optimization were compared. The optimized model showed significant improvements in accuracy after training, with an average accuracy exceeding 88%, an increase of 10.4% in accuracy, and a rate increase of 2.8 f / s. This demonstrates that the optimized RetinaNet model is suitable for intelligent and effective identification of dolomite crystal defects. The specific comparison results are shown in Table 1.

[0112] Table 1 Comparison of test results

[0113]

[0114] This embodiment iteratively trains the optimized RetinaNet model using a labeled defect training set based on the defect classification loss function and the defect localization loss function, thereby improving the accuracy and efficiency of model recognition.

[0115] In some embodiments of the present invention, the calculation formula of the defect recognition loss function is:

[0116] ,

[0117] in, is the defect classification loss function, is the defect localization loss function, For all positive and negative samples, For all positive samples, is the number of positive samples, For the The classification loss value corresponding to the sample, that is, The loss value between the predicted result of the sample and the actual label result, For the The regression loss value corresponding to the sample, that is, The regression loss value is the difference between the predicted bounding box coordinates and the true bounding box coordinates of samples.

[0118] In order to better implement the crystal defect type identification method in the embodiment of the present invention, based on the crystal defect type identification method, correspondingly, Figure 10 As shown, an embodiment of the present invention further provides a crystal defect type identification device 1000 including:

[0119] Image acquisition module 1001, used to acquire a transmission microscopic image of defects in dolomite crystals;

[0120] The label image generation module 1002 is used to add a defect type label to the defect transmission microscopic image according to the relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and generate a labeled defect microscopic image;

[0121] The training set generation module 1003 is used to perform image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set;

[0122] The recognition model generation module 1004 is used to construct an initial RetinaNet model, using the defect microscopic image as input and the defect type as output, and iteratively train the initial RetinaNet model based on the labeled defect training set to generate a target crystal defect type recognition model;

[0123] The defect recognition module 1005 is configured to recognize the transmission microscopic image of the crystal to be measured based on the target crystal defect type recognition model, and obtain the defect location and defect type of the crystal to be measured.

[0124] The crystal defect type identification device 1000 provided in the above embodiment can implement the technical solution described in the above embodiment of the crystal defect type identification method. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the crystal defect type identification method, which will not be repeated here.

[0125] In the embodiments of the present invention, the crystal defect type identification device may be a standalone server, or a server network or server cluster composed of servers. For example, the crystal defect type identification device described in the embodiments of the present invention includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0126] like Figure 11 As shown, the present invention also provides a crystal defect type identification device 1100. The crystal defect type identification device 1100 includes a processor 1101, a memory 1102 and a display 1103. Figure 11 Only some components of the crystal defect type identification apparatus 1100 are shown, but it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0127] In some embodiments, the processor 1101 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 1102 , such as the crystal defect type identification method of the present invention.

[0128] In some embodiments of the present invention, processor 1101 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, processor 1101 may be local or remote. In some embodiments, processor 1101 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, multiple clouds, or any combination thereof.

[0129] In some embodiments, the memory 1102 may be an internal storage unit of the crystal defect type identification device 1100, such as a hard disk or memory of the crystal defect type identification device 1100. In other embodiments, the memory 1102 may also be an external storage device of the crystal defect type identification device 1100, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the crystal defect type identification device 1100.

[0130] Furthermore, the memory 1102 may include both an internal storage unit of the crystal defect type identification device 1100 and an external storage device. The memory 1102 is used to store application software for installing the crystal defect type identification device 1100 and various data.

[0131] In some embodiments, display 1103 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 1103 is used to display information about crystal defect type identification device 1100 and to display a visual user interface. Components 1101-1103 of crystal defect type identification device 1100 communicate with each other via a system bus.

[0132] In some embodiments of the present invention, when the processor 1101 executes the crystal defect type identification program in the memory 1102, the following steps may be implemented:

[0133] Obtain transmission microscopic images of defects in dolomite crystals;

[0134] adding a defect type label to the defect transmission microscopic image according to a mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image to generate a labeled defect microscopic image;

[0135] Performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set;

[0136] Constructing an initial RetinaNet model, taking the defect microscopic image as input and the defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set to generate a target crystal defect type recognition model;

[0137] The transmission microscopic image of the crystal to be measured is identified based on the target crystal defect type identification model to obtain the defect type of the crystal to be measured.

[0138] It should be understood that, when the processor 1101 executes the crystal defect type identification program in the memory 1102 , in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0139] Furthermore, the embodiments of the present invention do not specifically limit the type of crystal defect type identification device 1100 mentioned. The crystal defect type identification device 1100 may be a portable crystal defect type identification device such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable crystal defect type identification devices include, but are not limited to, portable crystal defect type identification devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable crystal defect type identification device may also be other portable crystal defect type identification devices. It should also be understood that in some other embodiments of the present invention, the crystal defect type identification device 1100 may not be a portable crystal defect type identification device, but may be a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0140] Accordingly, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the steps or functions of the crystal defect type identification method provided in the above-mentioned method embodiments.

[0141] Those skilled in the art will appreciate that all or part of the process flow of the above-described method embodiment can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0142] The above is a detailed introduction to the crystal defect type identification method, device, electronic device and storage device provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying crystal defect types, characterized in that: include: Obtain transmission microscopic images of defects in dolomite crystals; adding a defect type label to the defect transmission microscopic image according to a mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image to generate a labeled defect microscopic image; Performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; Constructing an initial RetinaNet model, taking the defect microscopic image as input and the defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set to generate a target crystal defect type recognition model; Identifying the transmission microscopic image of the crystal to be measured based on the target crystal defect type identification model to obtain the defect location and defect type of the crystal to be measured; The mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image includes: Classifying crystal defects according to their spatial dimensions to generate first-order defect types, wherein the first-order defect types include zero-dimensional defects, one-dimensional defects, two-dimensional defects, and three-dimensional defects; Classifying each defect type in the first-order defect types according to imaging features of the transmission microscopy image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type; Adding a defect type label to the defect transmission microscopic image to generate a labeled defect microscopic image includes: assigning different defect type labels to each of the first-order defect type and the second-order defect type in turn; extracting imaging features of a defect transmission microscopic image of the dolomite crystal; determining the dimension of the crystal defect according to the imaging characteristics; Based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image, and according to the dimension and imaging characteristics of the crystal defect, determining the first-order defect type and the second-order defect type of the crystal defect; According to the first-order defect type and the second-order defect type of the crystal defect, a corresponding defect type label is added to each of the crystal defects to generate a labeled defect microscopic image.

2. The crystal defect type identification method according to claim 1, characterized in that: The labeled defect microscopic image is pre-enhanced to generate a labeled defect training set, including: Performing an affine transformation on the labeled defect microscopic image based on a generative adversarial network to generate a first enhanced training set; Determine the characteristic center area of ​​each defect in the first enhanced training set, perform random sliding sampling around the characteristic center area, and generate a second enhanced training set; The unclear images in the first enhanced training set and the second enhanced training set are sharpened to generate a label defect training set.

3. The method for identifying crystal defect types according to claim 2, wherein: Build the initial RetinaNet model, including: Build the original RetinaNet model; The ResNet network structure in the original RetinaNet model is replaced with the MobileNetv2 network structure, and the CBAM attention mechanism is added to the original RetinaNet model. The original RetinaNet model is optimized and adjusted to generate the initial RetinaNet model.

4. The method for identifying crystal defect types according to claim 3, wherein: Iteratively training the initial RetinaNet model according to the labeled defect training set includes: Based on the initial RetinaNet model, feature extraction is performed on defects in the labeled defect training set to generate defect features; Outputting a predicted defect category according to the defect characteristics; Constructing a defect recognition loss function, wherein the defect recognition loss function includes a defect classification loss function and a defect location loss function; Calculating the loss function value between the predicted defect category and the label information according to the defect recognition loss function; The initial RetinaNet model is iteratively trained according to the loss function value.

5. The method for identifying crystal defect types according to claim 4, wherein: The calculation formula of the defect recognition loss function is: ， in, is the defect classification loss function, is the defect localization loss function, For all positive and negative samples, For all positive samples, is the number of positive samples, For the The classification loss value corresponding to the sample, that is, The loss value between the predicted result of the sample and the actual label result, For the The regression loss value corresponding to the sample, that is, The regression loss value is the difference between the predicted bounding box coordinates and the true bounding box coordinates of samples.

6. A crystal defect type identification device, characterized in that: include: An image acquisition module, used to obtain a transmission microscopic image of defects in dolomite crystals; a label image generation module, configured to add a defect type label to the defect transmission microscopic image according to a relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, thereby generating a labeled defect microscopic image; A training set generation module, configured to perform image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; a recognition model generation module for constructing an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model based on the labeled defect training set to generate a target crystal defect type recognition model; A defect recognition module is used to identify the transmission microscopic image of the crystal to be tested based on the target crystal defect type recognition model to obtain the defect location and defect type of the crystal to be tested; The mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image includes: Classifying crystal defects according to their spatial dimensions to generate first-order defect types, wherein the first-order defect types include zero-dimensional defects, one-dimensional defects, two-dimensional defects, and three-dimensional defects; Classifying each defect type in the first-order defect types according to imaging features of the transmission microscopy image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type; Adding a defect type label to the defect transmission microscopic image to generate a labeled defect microscopic image includes: assigning different defect type labels to each of the first-order defect type and the second-order defect type in turn; extracting imaging features of a defect transmission microscopic image of the dolomite crystal; determining the dimension of the crystal defect according to the imaging characteristics; Based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopy image, and according to the dimension and imaging characteristics of the crystal defect, determining the first-order defect type and the second-order defect type of the crystal defect; According to the first-order defect type and the second-order defect type of the crystal defect, a corresponding defect type label is added to each of the crystal defects to generate a labeled defect microscopic image.

7. An electronic device for identifying crystal defect types, characterized in that: comprising a memory and a processor, wherein, The memory is used to store programs; The processor is coupled to the memory and is configured to execute the program stored in the memory to implement the steps of the crystal defect type identification method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores computer program instructions, and when the computer program instructions are executed by a computer, the computer is caused to execute the crystal defect type identification method according to any one of claims 1 to 5.

Images