Crystal defect type identification method and device, electronic equipment and storage medium

Abstract

The invention discloses a crystal defect type identification method and device, electronic equipment and a storage medium, and belongs to the technical field of crystal structure analysis, and the method comprises the steps: obtaining a defect transmission microscopic image of a dolomite crystal; adding a defect type label for the defect transmission microscopic image according to the mapping relation between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and generating a label defect microscopic image; performing image pre-enhancement on the label defect microscopic image to generate a label defect training set; constructing an initial RetinaNet model, and performing iterative training on the initial RetinaNet model by taking the defect microscopic image as input and the defect type as output to generate a target crystal defect type identification model; and according to the transmission microscopic image of the to-be-detected crystal, identifying the type of the defect of the to-be-detected crystal. The technical problem that the defects and defect types of the dolomite crystals cannot be accurately positioned and identified in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of crystal structure analysis, and in particular to a crystal defect type identification method, device, electronic equipment and storage medium. Background Art

[0002] Dolomite has a wide temporal and spatial distribution in ancient geological records. It is a common mineral in ancient rocks and a thermodynamically stable carbonate phase in modern seawater. The crystal structure preserves evidence of the crystal formation environment, crystallization rate, crystal growth and change characteristics, and fluid characteristics. Crystal structure analysis has unique advantages in the study of the genesis of dolomite. It can provide detailed atomic-level structural information and record the microscopic changes of minerals during diagenetic alteration, making up for the limitations of traditional petrological and geochemical methods. Therefore, dolomite crystal structure analysis is considered to be an effective means to identify the formation mechanism of dolomite. Crystal structure analysis mainly studies the crystal structure parameters of dolomite, such as order, unit cell parameters, interplanar spacing, and crystal defects. Among them, the crystal defects of dolomite can reflect the crystal formation environment and nucleation and growth process, and the formation mechanism of dolomite can be analyzed based on the crystal defects of dolomite. Crystal defects usually refer to incomplete areas where the arrangement of atoms in the internal structure of the crystal deviates from the ideal structure and locations where the integrity of the crystal structure is destroyed. In the complex formation environment and nucleation-growth process, dolomite crystals will form different types of defects due to the influence of changes in temperature, pressure, medium component concentration, etc. The analysis and identification of dolomite crystal defects is a key step in identifying the formation mechanism of dolomite, and also provides a new idea for studying the origin of dolomite. Therefore, the analysis, positioning and identification of dolomite crystal defects is crucial.

[0003] Dolomite crystal defect detection aims to identify and locate the defect abnormal area of ​​the object. Traditional crystal defect detection methods include manual visual method, corrosion and metallographic microscope observation method, X-ray diffraction method and electron microscope method. These methods are not only time-consuming and labor-intensive, but also have very high requirements on the technical level of the inspectors and are easily affected by human subjective factors. Due to the variety of defect types, noise interference in the collected images, and the diversity of abnormal areas on the target surface, image processing methods are prone to difficulties in feature modeling and migration, inaccurate feature description, etc. when marking and detecting defects, which affects the actual detection effect.

[0004] Therefore, there is an urgent need for a crystal defect type identification method that can quickly and accurately locate crystal defects and identify defect types by processing and analyzing crystal microscopic images. Summary of the invention

[0005] In view of this, it is necessary to provide a crystal defect type identification method, device, electronic device and storage medium, which can quickly and accurately locate crystal defects and identify defect types by processing and analyzing crystal microscopic images.

[0006] In order to solve the above technical problems, on the one hand, the present invention provides a method for identifying crystal defect types, comprising: Obtain transmission microscopic images of defects in dolomite crystals; According to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, a defect type label is added to the defect transmission microscopic image to generate a labeled defect microscopic image; Performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; Constructing an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set, and generating a target crystal defect type recognition model; The transmission microscopic image of the crystal to be tested is identified based on the target crystal defect type identification model to obtain the defect location and defect type of the crystal to be tested.

[0007] In a possible implementation, crystal defects are classified according to the spatial dimensions of the defects to generate first-order defect types, which include zero-dimensional defects, one-dimensional defects, two-dimensional defects, and three-dimensional defects.

[0008] Each defect type in the first-order defect type is classified according to the imaging characteristics of the transmission microscopic image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type.

[0009] In a possible implementation, adding a defect type label to the defect transmission microscopic image to generate a labeled defect microscopic image includes: Assigning different defect type labels to each of the first-order defect type and the second-order defect type in turn; Extracting imaging features of defect transmission microscopic images of the dolomite crystal; determining the dimension of the crystal defect according to the imaging feature; Based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, the first-order defect type and the second-order defect type of the crystal defect are determined according to the dimension and imaging characteristics of the crystal defect; According to the first-order defect type and the second-order defect type of the crystal defect, a corresponding defect type label is added to each of the crystal defects to generate a labeled defect microscopic image.

[0010] In a possible implementation, the label defect microscopic image is pre-enhanced to generate a label defect training set, including: Performing an affine transformation on the labeled defect microscopic image based on a generative adversarial network to generate a first enhanced training set; Determine the characteristic center area of ​​each defect in the first enhanced training set, perform random sliding sampling with the characteristic center area as the center, and generate a second enhanced training set; The unclear images in the first enhanced training set and the second enhanced training set are cleared to generate a label defect training set.

[0011] In one possible implementation, building the initial RetinaNet model includes: Build the original RetinaNet model; The ResNet network structure in the original RetinaNet model is replaced by the MobileNetv2 network structure, and the CBAM attention mechanism is added to the original RetinaNet model. The original RetinaNet model is optimized and adjusted to generate the initial RetinaNet model.

[0012] In a possible implementation, iteratively training the initial RetinaNet model according to the label defect training set includes: Based on the initial RetinaNet model, feature extraction is performed on defects in the labeled defect training set to generate defect features; Outputting a predicted defect category according to the defect feature; Constructing a defect recognition loss function, wherein the defect recognition loss function includes a defect classification loss function and a defect location loss function; According to the defect recognition loss function, a loss function value between the predicted defect category and the label information is calculated; The initial RetinaNet model is iteratively trained according to the loss function value.

[0013] In a possible implementation, the calculation formula of the defect recognition loss function is: , in, is the defect classification loss function, is the defect localization loss function, For all positive and negative samples, For all positive samples, is the number of positive samples, For the The classification loss value corresponding to the sample, that is, The loss value between the predicted result of a sample and the actual label result, For the The regression loss value corresponding to the sample, that is, The regression loss value is the difference between the predicted bounding box coordinates and the true bounding box coordinates of samples.

[0014] In a second aspect, the present invention further provides a crystal defect type identification device, comprising: An image acquisition module, used to acquire a defect transmission microscopic image of dolomite crystals; A label image generation module is used to add a defect type label to the defect transmission microscopic image according to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, so as to generate a label defect microscopic image; A training set generation module, used for performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; A recognition model generation module is used to construct an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set, and generating a target crystal defect type recognition model; The defect recognition module is used to recognize the transmission microscopic image of the crystal to be tested based on the target crystal defect type recognition model to obtain the defect location and defect type of the crystal to be tested.

[0015] In a third aspect, the present invention also provides an electronic device comprising a memory and a processor, wherein the memory is used to store programs and data; the processor is coupled to the memory, and is used to execute the program stored in the memory, to implement the crystal defect type identification method as described above, and / or to implement the crystal defect type identification as described above.

[0016] In a fourth aspect, the present invention further provides a computer storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the crystal defect type identification method as described above.

[0017] The beneficial effects of the present invention are as follows: first, a defect transmission microscopic image of a dolomite crystal is obtained; according to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, a defect type label is added to the defect transmission microscopic image to generate a labeled defect microscopic image; then, the labeled defect microscopic image is image pre-enhanced to generate a labeled defect training set; then, an initial RetinaNet model is constructed, with the defect microscopic image as input and the defect type as output, and the initial RetinaNet model is iteratively trained according to the labeled defect training set to generate a target crystal defect type recognition model; finally, the transmission microscopic image of the crystal to be tested is identified based on the target crystal defect type recognition model to obtain the defect location and defect type of the crystal to be tested. The present invention analyzes the transmission microscopic image of the dolomite crystal, analyzes the mapping relationship between the dolomite crystal defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and according to this mapping relationship, adds a defect label to each defect on the transmission microscopic image, and performs image enhancement on the labeled image to generate a training set, and iteratively trains the optimized RetinaNet model according to the training set to generate a final target crystal defect type recognition model. The present invention constructs a RetinaNet model by analyzing the mapping relationship between the defect types of dolomite crystals and the spatial dimensions and imaging characteristics of transmission microscopic images, thereby achieving rapid and accurate positioning of dolomite crystal defects and identification of defect types. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic flow chart of an embodiment of a method for identifying crystal defect types provided by the present invention; Figure 2 An illustration of the crystal defect types provided by the present invention; Figure 3 The present invention provides Figure 1 A schematic diagram of a flow chart of an embodiment of step S102; Figure 4 The present invention provides Figure 1 A schematic flow chart of an embodiment of step S103; Figure 5 A schematic diagram comparing the original image and the enhanced image of the vacancy defect type in the zero-dimensional defect and the grain boundary defect type in the two-dimensional defect provided by the present invention; Figure 6A comparison diagram of the original defect transmission microscopic image provided by the present invention and the image after the clarity processing; Figure 7 The present invention provides Figure 1 A schematic diagram of a flow chart of a first embodiment of step S104; Figure 8 This is a schematic diagram of the optimized RetinaNet model structure provided by the present invention; Fig. 9 The present invention provides Figure 1 Flow chart of the second embodiment of step S104 Fig.10 A schematic diagram of the structure of an embodiment of an electronic device for identifying crystal defect types provided by the present invention; Fig.11 A schematic diagram of the structure of an embodiment of a crystal defect type identification storage medium provided by the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] In the description of the embodiments of the present invention, unless otherwise specified, "multiple" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone.

[0022] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0023] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0024] Before describing the embodiments, the following definitions are given for the relevant terms: The RetinaNet model is an object detection model that consists of two main parts: the backbone network and the feature pyramid network. The backbone network uses a pre-trained convolutional neural network. The feature pyramid network consists of multiple feature layers that contain different spatial and scale information; The MobileNetv2 network is a lightweight convolutional neural network architecture designed to improve the efficiency of deep learning models on mobile and embedded devices, especially in resource-constrained environments; The CBAM attention mechanism is an attention mechanism module used to enhance the performance of convolutional neural networks. The main goal of CBAM is to improve the perception ability of the model by introducing channel attention and spatial attention, thereby improving performance without increasing network complexity.

[0025] The present invention provides a crystal defect type identification method, device, electronic device and storage medium, which are respectively described below.

[0026] Figure 1 A schematic flow chart of an embodiment of the crystal defect type identification method provided by the present invention is as follows: Figure 1 As shown, the crystal defect type identification method includes: S101, obtaining a transmission microscopic image of defects in a dolomite crystal; S102, adding a defect type label to the defect transmission microscopic image according to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and generating a labeled defect microscopic image; S103, performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; S104, constructing an initial RetinaNet model, taking the defect microscopic image as input and the defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set, and generating a target crystal defect type recognition model; S105 , identifying the transmission microscopic image of the crystal to be tested based on the target crystal defect type identification model to obtain the defect location and defect type of the crystal to be tested.

[0027] Specifically, we first collect microscopic images of dolomite crystal defects from different regions and periods, and classify the crystal defects into different defect types according to their different spatial dimensions and their microscopic imaging characteristics. We analyze the transmission microscopic images of dolomite crystals, and perform defect resolution and defect type resolution on the transmission microscopic images according to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic images, and add a label corresponding to the defect type to each defect. On this basis, we use a generative adversarial network (GAN) to perform super-resolution conversion on the labeled transmission microscopic images and expand the data set to obtain an enhanced training data set. The training data set is divided into two parts, 80% of which is the training set and 20% is the validation set. We iteratively train the RetinaNet model, and verify and evaluate the accuracy of the trained RetinaNet model to obtain the final crystal defect type recognition model.

[0028] This embodiment analyzes the transmission microscopic image of dolomite crystals, analyzes the mapping relationship between the defect type of dolomite crystals and the spatial dimensions and imaging characteristics of the transmission microscopic image, adds a defect label to each defect on the transmission microscopic image according to this mapping relationship, and performs image enhancement on the labeled image to generate a training set, iteratively trains the optimized RetinaNet model according to the training set, and generates the final target crystal defect type recognition model. The present invention constructs a RetinaNet model by analyzing the mapping relationship between the defect type of dolomite crystals and the spatial dimensions and imaging characteristics of the transmission microscopic image, thereby realizing the rapid and accurate positioning of dolomite crystal defects and the identification of defect types.

[0029] In some embodiments of the present invention, the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image includes: Crystal defects are classified according to the spatial dimensions of the defects to generate first-order defect types, which include zero-dimensional defects, one-dimensional defects, two-dimensional defects and three-dimensional defects.

[0030] Each defect type in the first-order defect type is classified according to the imaging characteristics of the transmission microscopic image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type.

[0031] Specifically, first-order defect types include zero-dimensional defects, one-dimensional defects, two-dimensional defects and three-dimensional defects; when the imaging feature of the defect is the position error of a single atom or ion, the spatial dimension is zero-dimensional, and its defect type is a zero-dimensional defect; when the imaging feature of the defect is extension along a specific direction in the crystal structure, the spatial dimension is one-dimensional, and its defect type is a one-dimensional defect; when the imaging feature of the defect in the crystal structure is a regional or layered structure that deviates from the periodic and regular arrangement, the spatial dimension is two-dimensional, and its defect type is a two-dimensional defect; when the imaging feature of the defect is the collective absence and aggregation of a large number of atoms or ions, the spatial dimension is three-dimensional, and its defect type is a three-dimensional defect.

[0032] Furthermore, for each first-order defect type, due to different defect characteristics, it is divided into multiple second-order defect types, each first-order defect type includes one or more second-order defect types, and the number of second-order defect types can be increased or decreased according to the defect imaging characteristics. In this embodiment, according to the imaging characteristics of the transmission microscopy image, the zero-dimensional defect is subdivided into three second-order defect types, including vacancies, interstitial ions and impurity atoms, among which the imaging characteristic of vacancies is the existence of vacant positions in the atomic crystal structure; the imaging characteristic of interstitial ions is the appearance of ions in the gaps in the crystal lattice at positions where there are no atoms in the crystal structure; the imaging characteristic of impurity atoms is the appearance of foreign atoms at the crystal lattice positions. One-dimensional defects are subdivided into three second-order defect types, including knife dislocation, screw dislocation and mixed dislocation. The imaging feature of knife dislocation is a line defect formed by the movement of atoms in a direction perpendicular to the crystal surface, specifically inserting a half-atomic plane into an ideal complete crystal structure; the knife dislocation is the edge of a half-atomic plane; the imaging feature of screw dislocation is a line defect formed by the movement of atoms on a spiral path, and its dislocation line is parallel to the slip direction; the imaging feature of mixed dislocation is a combination of knife dislocation and screw dislocation. Two-dimensional defects are subdivided into three second-order defect types, including grain boundary, twin boundary and stacking fault. The imaging feature of grain boundary is that it appears between two crystal structures, and the crystallization directions of the two crystals are completely different; the imaging feature of twin boundary is that at certain stages of the crystal growth process, due to changes in environmental conditions or the action of internal stress, part of the crystal structure rotates to form twins; the imaging feature of stacking fault is that at certain stages of the crystal growth process, due to changes in the atomic deposition order, part of the crystal structure has stacking faults. Three-dimensional defects are subdivided into three second-order defect types, including voids, precipitates and fluid inclusions. The imaging characteristic of voids is the existence of space regions inside the crystal structure that are not occupied by atoms. The existence of voids will affect the mechanical and physical properties of the crystal and reduce the strength and density of the crystal; the imaging characteristic of precipitates is crystalline substances with different compositions or structures separated from the crystal matrix; the imaging characteristic of fluid inclusions is the fluid (such as gas, liquid or gas-liquid mixture) captured and enclosed inside the crystal during the growth process.

[0033] Furthermore, if Figure 2 As shown, Figure 2 The crystal defect type illustration diagram provided by the present invention includes first-order defect types and their corresponding second-order defect types.

[0034] In some embodiments of the present invention, Figure 3 As shown, Figure 3 The present invention provides Figure 1 The flowchart of an embodiment of step S102 in the embodiment includes: S301, assigning different defect type labels to each first-order defect type and second-order defect type in turn; S302, extracting imaging features of defect transmission microscopic images of dolomite crystals; S303, determining the dimension of the crystal defect according to the imaging characteristics; S304, based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, determining the first-order defect type and the second-order defect type of the crystal defect according to the dimension and imaging characteristics of the crystal defect; S305 , adding a corresponding defect type label to each of the crystal defects according to the first-order defect type and the second-order defect type of the crystal defect, and generating a labeled defect microscopic image.

[0035] Specifically, different defect type labels are assigned to each first-order defect type and second-order defect type. For example, the labels of the four first-order defect types are 1, 2, 3 and 4, respectively. The label of the zero-dimensional defect is 1, and the labels of the three corresponding second-order defect types, vacancies, interstitial ions and impurity atoms, are 1-1, 1-2 and 1-3, respectively. Similarly, the label of the one-dimensional defect is 2, and the labels of the three corresponding second-order defect types, knife dislocations, screw dislocations and mixed dislocations, are 2-1, 2-2 and 2-3, respectively. The label of the two-dimensional defect is 3, and the labels of the three corresponding second-order defect types, grain boundaries, twin boundaries and stacking faults, are 3-1, 3-2 and 3-3, respectively. The label of the three-dimensional defect is 4, and the labels of the three corresponding second-order defect types, voids, precipitates and fluid inclusions, are 4-1, 4-2 and 4-3, respectively. Crystal defects are subdivided into multiple types according to dimension and imaging characteristics.

[0036] Furthermore, the imaging features of the defect transmission microscopy image of the dolomite crystal are extracted, the dimensions of the crystal defects are determined according to the imaging features of the defect transmission microscopy image, the first-order defect types of the crystal defects are determined according to the dimensions, the second-order defect types of the crystal defects are further determined according to the imaging features, and the corresponding first-order defect labels and second-order defect labels are added to each crystal defect according to the first-order defect type and the second-order defect type combined with the defect labels corresponding to the above defect types.

[0037] This embodiment determines the first-order defect type of the crystal defect according to the dimension of the defect transmission microscopic image of the dolomite crystal, determines the second-order defect type of the crystal defect according to the imaging characteristics of the defect transmission microscopic image, and subdivides the defects into multiple types, which can analyze the dolomite crystal more accurately and clearly.

[0038] In some embodiments of the present invention, Figure 4 As shown, Figure 4 The present invention provides Figure 1 The flowchart of an embodiment of step S103 in the embodiment includes: S401, performing affine transformation on the labeled defect microscopic image based on a generative adversarial network to generate a first enhanced training set; S402, determining the characteristic center area of ​​each defect in the first enhanced training set, performing random sliding sampling with the characteristic center area as the center, and generating a second enhanced training set; S403 , performing clarity processing on unclear images in the first enhanced training set and the second enhanced training set to generate a label defect training set.

[0039] It should be noted that due to the small number of transmission microscopy image samples and the unclearness of some collected images, overfitting and incorrect recognition are likely to occur, resulting in the generated defect images being too single. Therefore, image pre-enhancement is performed here, and a GAN model is constructed through the generative adversarial network GAN to perform super-resolution conversion on the defect transmission microscopy images of dolomite crystals and expand the data set.

[0040] Specifically, for the transmission microscopic images of the training samples, the adversarial generative network GAN is used to process the transmission microscopic images of the training samples in the following three steps: first, the image is affine transformed, and the training data is increased by using random flipping, mirroring and other operations of the image; then, the characteristic center area of ​​the defect is determined for random sliding sampling, such as Figure 5 As shown, Figure 5 The figure is a schematic diagram of the comparison between the original image and the enhanced image of the vacancy defect type in the zero-dimensional defect and the grain boundary defect type in the two-dimensional defect provided by the present invention. The first column of images in the figure is the original defect transmission microscopic image, and the second to fourth columns are the enhanced images after affine transformation and slip sampling; finally, the unclear images are clarified according to the super-resolution conversion algorithm, such as Figure 6 As shown, Figure 6 The original defect transmission microscopic image provided by the present invention is compared with the image after the clarity processing, wherein the first row A, B and C are defect transmission microscopic images of dolomite crystal defects, and the second row A', B' and C' are defect transmission microscopic images after the clarity processing. In this embodiment, the label defect microscopic image is first subjected to a random affine transformation, the transformed image is 800×800, the size of the defect feature in the image is about 400×400, the window size of the image acquisition is 600×600, and multiple random sliding cropping is performed around the defect feature center, and finally about 1800 multi-dimensional crystal defect images are obtained, of which 1440 are set as training images and 360 are set as verification images.

[0041] Furthermore, it can be understood that this embodiment combines the generative adversarial network GAN and the optimized RetinaNet model to form a complete target crystal defect type recognition model. In actual use, for the transmission microscopic image of the dolomite crystal to be tested that needs to be identified, when it is processed by the generative adversarial network GAN, it is only necessary to clarify the transmission microscopic image, and then use the clarified image as the input of the RetinaNet model to locate the defects and identify the defect types of the dolomite crystal to be tested.

[0042] In some embodiments of the present invention, Figure 7 As shown, Figure 7 The present invention provides Figure 1 The flowchart of the first embodiment of step S104 in FIG. 1 includes: S701, build the original RetinaNet model; S702. Use the MobileNetv2 network structure to replace the ResNet network structure in the original RetinaNet model, add the CBAM attention mechanism to the original RetinaNet model, optimize and adjust the original RetinaNet model, and generate an initial RetinaNet model.

[0043] It should be noted that the ResNet network structure with a deep network structure and a large number of parameters in the RetinaNet model is replaced with a lightweight MobileNetv2 network structure as the backbone feature extraction network, thereby reducing the computing resource cost and shortening the training time, enhancing the target detection effect, and eliminating the risk of overfitting; the CBAM attention mechanism is added to the RetinaNet model to improve the model's feature extraction ability and robustness under complex striped background conditions. On this basis, the feature pyramid network (FPN) is introduced to fuse the feature maps of different levels in the backbone network to generate a set of pyramid-shaped feature maps with multi-scale features. This model can effectively perform feature extraction and detection training on images of various scales, thereby improving the accuracy of target detection, such as Figure 8 As shown, Figure 8 This is a schematic diagram of the optimized RetinaNet model structure provided by the present invention, in which the GAN adversarial network is combined to form a complete optimized RetinaNet model.

[0044] This embodiment optimizes the RetinaNet model through the lightweight MobileNetv2 network structure and the CBAM attention mechanism, thereby improving the accuracy and robustness of the RetinaNet model.

[0045] In some embodiments of the present invention, Fig. 9 As shown, Fig. 9 The present invention provides Figure 1 The flowchart of the second embodiment of step S104 in FIG. 1 includes: S901. Based on the initial RetinaNet model, feature extraction is performed on defects in the label defect training set to generate defect features; S902, outputting a predicted defect category according to defect characteristics; S903, constructing a defect recognition loss function, where the defect recognition loss function includes a defect classification loss function and a defect location loss function; S904, calculating the loss function value between the predicted defect category and the label information according to the defect recognition loss function; S905. Iteratively train the initial RetinaNet model according to the loss function value.

[0046] It should be noted that the FocalLoss loss function is added to the RetinaNet model to fine-tune its attention, give priority to those details that are easily overlooked but are crucial to the overall recognition accuracy, and solve the problem of imbalance in the number of positive and negative samples and difficulty in distinguishing feature types during the detection process. The formula of the FocalLoss loss function is: , in, is the loss function value, is the estimated probability of the RetinaNet model, γ is an adjustable focusing parameter, is the modulation factor.

[0047] It should be further explained that the classification loss function ( ) predicts the target category and uses the bounding box regression loss function ( ) to accurately locate the target position.

[0048] Specifically, this implementation uses 1440 training images with defect type labels to iteratively train the initial RetinaNet model, calculates the loss function value based on the output label results of the RetinaNet model and the actual label results of the defects, adjusts and optimizes the model's hyperparameters based on the loss function value, and obtains the final target defect type detection model. Finally, 360 crystal defect verification images are used to locate defects and identify defect types, and the accuracy and rate of crystal defect location and identification before and after model optimization are compared. It is found that the accuracy of the optimized model is significantly improved after training, and its accuracy value is above 88% on average, with an accuracy increase of 10.4% and a rate increase of 2.8f / s, indicating that the optimized RetinaNet model is suitable for intelligent and effective identification of dolomite crystal defects. The specific comparison results are shown in Table 1.

[0049] Table 1 Comparison of test results

[0050] This embodiment iteratively trains the optimized RetinaNet model using a labeled defect training set based on the defect classification loss function and the defect location loss function, thereby improving the accuracy and efficiency of model recognition.

[0051] In some embodiments of the present invention, the calculation formula of the defect recognition loss function is: , in, is the defect classification loss function, is the defect localization loss function, For all positive and negative samples, For all positive samples, is the number of positive samples, For the The classification loss value corresponding to the sample, that is, The loss value between the predicted result of a sample and the actual label result, For the The regression loss value corresponding to the sample, that is, The regression loss value is the difference between the predicted bounding box coordinates and the true bounding box coordinates of samples.

[0052] In order to better implement the crystal defect type identification method in the embodiment of the present invention, based on the crystal defect type identification method, correspondingly, Fig.10 As shown, the embodiment of the present invention further provides a crystal defect type identification device 1000 including: An image acquisition module 1001 is used to acquire a defect transmission microscopic image of a dolomite crystal; The label image generation module 1002 is used to add a defect type label to the defect transmission microscopic image according to the relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and generate a labeled defect microscopic image; The training set generation module 1003 is used to perform image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; The recognition model generation module 1004 is used to construct an initial RetinaNet model, with the defect microscopic image as input and the defect type as output, and iteratively train the initial RetinaNet model according to the labeled defect training set to generate a target crystal defect type recognition model; The defect recognition module 1005 is used to recognize the transmission microscopic image of the crystal to be tested based on the target crystal defect type recognition model to obtain the defect location and defect type of the crystal to be tested.

[0053] The crystal defect type identification device 1000 provided in the above embodiment can implement the technical solution described in the above crystal defect type identification method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above crystal defect type identification method embodiment, which will not be repeated here.

[0054] In the embodiment of the present invention, the crystal defect type identification device may be an independent server, or a server network or server cluster composed of servers. For example, the crystal defect type identification device described in the embodiment of the present invention includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0055] like Fig.11 As shown, the present invention also provides a crystal defect type identification device 1100. The crystal defect type identification device 1100 includes a processor 1101, a memory 1102 and a display 1103. Fig.11 Only some components of the crystal defect type identification device 1100 are shown, but it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0056] In some embodiments, the processor 1101 may be a central processing unit (CPU), a microprocessor or other data processing chip, and is used to run program codes or process data stored in the memory 1102, such as the crystal defect type identification method of the present invention.

[0057] In some embodiments of the present invention, the processor 1101 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 1101 may be local or remote. In some embodiments, the processor 1101 may be implemented in a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0058] In some embodiments, the memory 1102 may be an internal storage unit of the crystal defect type identification device 1100, such as a hard disk or memory of the crystal defect type identification device 1100. In other embodiments, the memory 1102 may also be an external storage device of the crystal defect type identification device 1100, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the crystal defect type identification device 1100.

[0059] Furthermore, the memory 1102 may include both an internal storage unit of the crystal defect type identification device 1100 and an external storage device. The memory 1102 is used to store application software for installing the crystal defect type identification device 1100 and various data.

[0060] In some embodiments, the display 1103 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 1103 is used to display information of the crystal defect type identification device 1100 and to display a visual user interface. The components 1101-1103 of the crystal defect type identification device 1100 communicate with each other via a system bus.

[0061] In some embodiments of the present invention, when the processor 1101 executes the crystal defect type identification program in the memory 1102, the following steps may be implemented: Obtain transmission microscopic images of defects in dolomite crystals; According to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, a defect type label is added to the defect transmission microscopic image to generate a labeled defect microscopic image; Performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; Constructing an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set, and generating a target crystal defect type recognition model; The transmission microscopic image of the crystal to be tested is identified based on the target crystal defect type identification model to obtain the defect type of the crystal to be tested.

[0062] It should be understood that: when the processor 1101 executes the crystal defect type identification program in the memory 1102, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0063] Furthermore, the embodiment of the present invention does not specifically limit the type of the crystal defect type identification device 1100 mentioned. The crystal defect type identification device 1100 may be a portable crystal defect type identification device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable crystal defect type identification devices include but are not limited to portable crystal defect type identification devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable crystal defect type identification device may also be other portable crystal defect type identification devices. It should also be understood that in some other embodiments of the present invention, the crystal defect type identification device 1100 may not be a portable crystal defect type identification device, but a desktop computer with a touch-sensitive surface (such as a touch panel).

[0064] Correspondingly, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the steps or functions of the crystal defect type identification method provided in the above-mentioned method embodiments.

[0065] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0066] The crystal defect type identification method, device, electronic device and storage device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying crystal defect types, characterized in that: include: Obtain transmission microscopic images of defects in dolomite crystals; According to the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, a defect type label is added to the defect transmission microscopic image to generate a labeled defect microscopic image; Performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; Constructing an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set, and generating a target crystal defect type recognition model; The transmission microscopic image of the crystal to be tested is identified based on the target crystal defect type identification model to obtain the defect location and defect type of the crystal to be tested.

2. The crystal defect type identification method according to claim 1, characterized in that: The mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image includes: Classifying crystal defects according to the spatial dimensions of the defects to generate first-order defect types, wherein the first-order defect types include zero-dimensional defects, one-dimensional defects, two-dimensional defects, and three-dimensional defects; Each defect type in the first-order defect type is classified according to the imaging characteristics of the transmission microscopic image to generate second-order defect types, wherein each first-order defect type includes at least one second-order defect type.

3. The crystal defect type identification method according to claim 2, characterized in that: Adding a defect type label to the defect transmission microscopic image to generate a labeled defect microscopic image includes: Assigning different defect type labels to each of the first-order defect type and the second-order defect type in turn; Extracting imaging features of defect transmission microscopic images of the dolomite crystal; determining the dimension of the crystal defect according to the imaging feature; Based on the mapping relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, the first-order defect type and the second-order defect type of the crystal defect are determined according to the dimension and imaging characteristics of the crystal defect; According to the first-order defect type and the second-order defect type of the crystal defect, a corresponding defect type label is added to each of the crystal defects to generate a labeled defect microscopic image.

4. The crystal defect type identification method according to claim 3, characterized in that: The labeled defect microscopic image is pre-enhanced to generate a labeled defect training set, including: Performing an affine transformation on the labeled defect microscopic image based on a generative adversarial network to generate a first enhanced training set; Determine the characteristic center area of ​​each defect in the first enhanced training set, perform random sliding sampling with the characteristic center area as the center, and generate a second enhanced training set; The unclear images in the first enhanced training set and the second enhanced training set are cleared to generate a label defect training set.

5. The crystal defect type identification method according to claim 4, characterized in that: Build the initial RetinaNet model, including: Build the original RetinaNet model; The ResNet network structure in the original RetinaNet model is replaced by the MobileNetv2 network structure, and the CBAM attention mechanism is added to the original RetinaNet model. The original RetinaNet model is optimized and adjusted to generate the initial RetinaNet model.

6. The crystal defect type identification method according to claim 5, characterized in that: Iteratively training the initial RetinaNet model according to the label defect training set includes: Based on the initial RetinaNet model, feature extraction is performed on defects in the labeled defect training set to generate defect features; Outputting a predicted defect category according to the defect feature; Constructing a defect recognition loss function, wherein the defect recognition loss function includes a defect classification loss function and a defect location loss function; According to the defect recognition loss function, a loss function value between the predicted defect category and the label information is calculated; The initial RetinaNet model is iteratively trained according to the loss function value.

7. The crystal defect type identification method according to claim 5, characterized in that: The calculation formula of the defect recognition loss function is: ， in, is the defect classification loss function, is the defect localization loss function, For all positive and negative samples, For all positive samples, is the number of positive samples, For the The classification loss value corresponding to the sample, that is, The loss value between the predicted result of the sample and the actual label result, For the The regression loss value corresponding to the sample, that is, The regression loss value is the difference between the predicted bounding box coordinates and the true bounding box coordinates of samples.

8. A crystal defect type identification device, characterized in that: include: An image acquisition module, used to acquire a defect transmission microscopic image of dolomite crystals; A label image generation module, used to add a defect type label to the defect transmission microscopic image according to the relationship between the defect type and the spatial dimension and imaging characteristics of the transmission microscopic image, and generate a labeled defect microscopic image; A training set generation module, used for performing image pre-enhancement on the labeled defect microscopic image to generate a labeled defect training set; A recognition model generation module is used to construct an initial RetinaNet model, taking a defect microscopic image as input and a defect type as output, iteratively training the initial RetinaNet model according to the labeled defect training set, and generating a target crystal defect type recognition model; The defect recognition module is used to recognize the transmission microscopic image of the crystal to be tested based on the target crystal defect type recognition model to obtain the defect location and defect type of the crystal to be tested.

9. An electronic device for identifying crystal defect types, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the crystal defect type identification method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores computer program instructions, and when the computer program instructions are executed by a computer, the computer is enabled to execute the crystal defect type identification method according to any one of claims 1 to 7.

Images