Training methods for EBSD image completion models and methods for completing EBSD images.

Abstract

This invention relates to the field of steel material performance analysis technology, specifically to a training method for an EBSD image completion model and an EBSD image completion method. The training method for the EBSD image completion model introduces scanning electron microscope (SEM) images from the same field of view to provide high-precision microstructure boundaries, and combines this with confidence assessment of EBSD data points to select high-confidence seed labels, effectively solving the problem of supervision signal defects caused by reliance on incomplete labels in traditional methods. Furthermore, by introducing a microstructure boundary mask strictly registered with the EBSD field of view, and utilizing its clear physical contour constraints combined with the contrast features of the BC image to construct a loss function, the neural network model is guided to perform reasonable inference and completion of crystallographic information within the defined physical boundaries, significantly improving the completeness and accuracy of the labeling of key microstructure regions in EBSD images.

Description

Technical Field

[0001] This invention relates to the field of steel material performance analysis technology, specifically to a training method for an EBSD image completion model and a method for completing EBSD images. Background Technology

[0002] In the research and industrial production of steel materials, microstructure is a core element connecting process parameters and final performance. For materials such as advanced high-strength steel, offshore platform steel, and high-performance pipeline steel, their strength, toughness, and fatigue performance largely depend on the size, distribution, and content of specific microstructures such as M / A islands, bainite, and pearlite. Therefore, achieving rapid and accurate quantitative analysis of these key microstructures has become an important requirement for optimizing composition design, controlling production processes, predicting product performance, and ensuring safe use.

[0003] Electron backscatter diffraction (EBSD) technology can provide unique crystallographic information such as crystal orientation and grain boundary type, thus effectively meeting the aforementioned requirements. However, it has limitations in identifying key microstructures in practical applications: due to the small size of microstructures such as Martensite / Austenite Islands and pearlite, the presence of numerous microstructures and potential microstrain, the diffraction pattern quality and signal-to-noise ratio are poor when EBSD acquires data in these regions. This significantly reduces the calibration rate of the EBSD system in these areas. When the material contains many inclusions, precipitates, or other second phases, the protrusions of these second phases after the sample stage is tilted by 70° will obscure the area behind them, resulting in a large number of "uncalibrated" points in the final output spectrum, making it impossible to generate complete and reliable distribution data.

[0004] The current solution uses the detected EBSD results as a standard, and trains a model to expand the image from solvable regions to unsuccessful regions. The fundamental flaw of this approach is that the "labels" it relies on for learning are inherently inaccurate or incomplete in the key regions (i.e., the tissue regions to be identified). Models trained under this flawed supervision have a pre-locked upper limit, unable to overcome the physical limitations of EBSD technology. They often only achieve limited filling in of data representation, failing to achieve substantial improvements in the physical accuracy of tissue identification. Summary of the Invention

[0005] This invention provides a training method for an EBSD image completion model and an EBSD image completion method to solve the problem of low calibration rate of EBSD in key tissue regions in related technologies.

[0006] In a first aspect, the present invention provides a training method for an EBSD image completion model, the method comprising the following steps: acquiring a training dataset, the training dataset comprising multiple sets of training samples, each set of training samples comprising a training EBSD image and a training scanning electron microscope image in the same field of view as the training EBSD image; for each set of training samples, performing the following preprocessing steps: extracting the contours of different tissues in the training scanning electron microscope image to obtain a training tissue boundary mask; extracting contrast data from the training EBSD image to obtain a training BC image, mapping the training tissue boundary mask to the coordinate system of the training BC image to obtain a training boundary mask registered with the training BC image; evaluating the credibility of EBSD data points in the training EBSD image to obtain a training seed label image; constructing a preset neural network model, using the training seed label image and the training BC image in the training dataset as input, and combining the corresponding training boundary mask to construct a loss function, training the neural network model to obtain an EBSD image completion model.

[0007] The training method for the EBSD image completion model provided by this invention provides high-precision tissue morphology boundaries by introducing scanning electron microscope images of the same field of view, and selects high-confidence seed labels by combining the confidence assessment of EBSD data points, effectively solving the problem of supervision signal defects caused by the reliance on incomplete labels of EBSD itself in traditional methods. By introducing a tissue boundary mask map that is strictly registered with the EBSD field of view, the loss function is constructed by using its clear physical contour constraints and the contrast features of the BC map, guiding the neural network model to perform reasonable inference and completion of crystallographic information within the determined physical boundaries. This not only significantly improves the completeness and accuracy of the calibration of key tissue regions in EBSD images, and realizes accurate identification and reliable reconstruction of small or occluded regions such as M / A islands and pearlites, but also fundamentally breaks through the calibration rate bottleneck caused by poor diffraction pattern quality and occlusion effect in traditional EBSD technology.

[0008] In some optional implementations, mapping the training tissue boundary mask to the coordinate system of the training BC image to obtain a training boundary mask registered with the training BC image includes: determining the matching points between the training SEM image and the training BC image; determining the affine transformation matrix using the matching points between the training SEM image and the training BC image; and mapping the training tissue boundary mask to the coordinate system of the training BC image according to the affine transformation matrix to obtain a training boundary mask registered with the training BC image.

[0009] This implementation achieves pixel-level precise registration between the scanning electron microscope image and the BC image through affine transformation, effectively eliminating coordinate deviations caused by sample drift or scanning distortion, and avoiding label noise caused by boundary mask misalignment.

[0010] In some optional implementations, the confidence assessment of EBSD data points in the training EBSD image to obtain a training seed label map includes: calculating the confidence score of each EBSD data point in the training EBSD image; determining whether each confidence score is greater than a preset confidence threshold; filtering EBSD data points with confidence scores greater than the confidence threshold, and combining them to generate a training seed label map.

[0011] This implementation method, through quantitative filtering of data points, can accurately eliminate low-quality noise, ensuring that high-confidence seed data serves as the core supervision signal for model training. This threshold-based filtering mechanism effectively avoids interference from erroneous labels on model learning, significantly improving the accuracy and reliability of model inference completion.

[0012] In some optional implementations, the training seed label map and training BC map from the training dataset are used as inputs, and a loss function is constructed by combining the corresponding training boundary mask map to train the neural network model to obtain the EBSD image completion model. This includes: inputting the training seed label map and training BC map from the training dataset into the neural network model and outputting the predicted crystallographic parameter map; calculating the classification loss between the predicted crystallographic parameter map and the training seed label map; calculating the region topology loss between the predicted crystallographic parameter map and the training boundary mask map; calculating the difference between the gradient magnitude map of the predicted crystallographic parameter map and the gradient magnitude map of the training BC map to obtain the edge constraint loss; and updating the parameters of the neural network model based on the classification loss, region topology loss, and edge constraint loss.

[0013] This implementation method constructs a multi-task loss function by fusing classification loss, region topology loss, and edge constraint loss. This function collaboratively guides the model to strictly adhere to the boundaries of the microstructure and microscopic details while preserving known crystallographic information. This joint optimization strategy effectively avoids the out-of-bounds diffusion and edge blurring of the completion results, and improves the consistency and realism of the phase distribution logic and geometric details of the crystallographic parameter map output by the model.

[0014] In some optional implementations, inputting the training seed label map and training BC map from the training dataset into the neural network model and outputting the predicted crystallographic parameter map includes: extracting multi-scale features from the training BC map and the training seed label map respectively to obtain BC features and seed features; concatenating the seed features and BC features through channels and processing them through convolutional layers and activation functions to generate an attention map; weighting the attention map and BC features to obtain fused features; and outputting the predicted crystallographic parameter map based on the fused features.

[0015] This implementation method, by fusing multi-scale features and introducing an attention mechanism, can guide the model to adaptively focus on key morphological features related to high-confidence seed regions in the BC map. This weighted fusion strategy effectively enhances the discriminativeness of feature representations, making the model more targeted when supplementing missing crystallographic information.

[0016] Secondly, the present invention also provides an EBSD image completion method, the method comprising the following steps: acquiring a scanning electron microscope image of the sample to be tested and EBSD signal data to be tested; extracting contrast data from the EBSD signal data to be tested to obtain a BC image to be tested; extracting the contours of different tissues in the scanning electron microscope image to be tested to obtain a boundary mask of the tissue to be tested; mapping the boundary mask of the tissue to be tested onto the coordinate system of the BC image to be tested to obtain a boundary mask registered with the BC image to be tested; evaluating the credibility of EBSD data points in the EBSD signal data to be tested to obtain a seed label image to be tested; inputting the BC image to be tested and the seed label image to be tested into an EBSD image completion model trained by the method of the first aspect or any corresponding embodiment to obtain a predicted crystallographic parameter map; and constructing a complete EBSD dataset based on the predicted crystallographic parameter map and the seed label image to be tested.

[0017] The EBSD image completion method provided by this invention, by fusing a high-precision tissue boundary mask and a high-confidence seed label, collaboratively guides the EBSD image completion model to make predictions, which can effectively ensure the accuracy of the completion results in terms of physical boundaries and crystallographic features.

[0018] In some optional implementations, a complete EBSD dataset is constructed based on the predicted crystallographic parameter map and the seed label map to be tested, including: replacing the data in the region of the seed label map to be tested with the data in the corresponding region of the predicted crystallographic parameter map, and retaining the original EBSD signal data in the region of the seed label map to be tested with the confidence level higher than or equal to the preset threshold, thereby generating a complete EBSD dataset.

[0019] This implementation method, based on a confidence threshold-based "optimal fusion strategy," can accurately retain high-confidence original measured data while filling low-confidence regions with high-quality data predicted by the model, effectively avoiding damage to the original reliable data. It not only maximizes the authenticity and physical consistency of the measured data but also significantly improves the completeness and reliability of the final EBSD dataset.

[0020] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the training method of the EBSD image completion model of the first aspect or any corresponding embodiment thereof and / or the EBSD image completion method of the second aspect or any corresponding embodiment thereof.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the training method of the EBSD image completion model of the first aspect or any corresponding embodiment thereof and / or the EBSD image completion method of the second aspect or any corresponding embodiment thereof.

[0022] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the training method of the EBSD image completion model of the first aspect or any corresponding embodiment thereof and / or the EBSD image completion method of the second aspect or any corresponding embodiment thereof. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a first flowchart of the EBSD image completion model training method according to an embodiment of the present invention; Figure 3 This is a second flowchart of the EBSD image completion model training method according to an embodiment of the present invention; Figure 4 This is a flowchart of the EBSD image completion method according to an embodiment of the present invention; Figure 5 These are SEM images of the samples taken when the sample stage is tilted to 0° according to an embodiment of the present invention; Figure 6 This is the BC plot in the EBSD results of the sample collected when the sample stage is tilted at 70° according to an embodiment of the present invention; Figure 7 This is an IPF plot in the EBSD results of a sample collected when the sample stage is tilted at 70° according to an embodiment of the present invention; Figure 8 This refers to the IPF-GB image in the completion result according to an embodiment of the present invention; Figure 9 This is a structural block diagram of an EBSD image completion model training device according to an embodiment of the present invention; Figure 10 This is a structural block diagram of an EBSD image completion device according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0027] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] As an optional application scenario of this invention, such as Figure 1 As shown, the system may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0029] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0030] Due to their small size, internal stress, and other microscopic features, as well as the presence of inclusions, precipitates, and other second-phase interference, the diffraction patterns generated by the electron beam in these regions exhibit poor quality and signal-to-noise ratio. Consequently, the calibration rate of the EBSD system drops sharply in these areas, resulting in a large number of uncalibrated points in the output. Using these flawed and erroneous EBSD data as standards for result completion and fitting essentially results in a mapping relationship containing incorrect information. This makes it impossible to avoid the original errors and omissions in EBSD when dealing with corresponding tissue regions in SEM images during the fitting process, ultimately compromising the accuracy and reliability of the output tissue segmentation maps and quantitative statistical results.

[0031] Based on this, the present invention provides an embodiment of a training method for an EBSD image completion model. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides a training method for an EBSD image completion model, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 2 This is a flowchart of the first method for training an EBSD image completion model according to an embodiment of the present invention, as follows: Figure 2 As shown, the process includes the following steps: Step S201: Obtain the training dataset, which includes multiple sets of training samples. Each set of training samples includes training EBSD images and training SEM images with the same field of view as the training EBSD images.

[0033] The scanning electron microscope (SEM) images can be images captured using secondary electron (SE) signals or images captured using backscattered electron (BSE) signals.

[0034] Step S202: For each training sample, perform the following preprocessing steps: extract the contours of different tissues in the training SEM image to obtain the training tissue boundary mask; extract the contrast data in the training EBSD image to obtain the training BC image; map the training tissue boundary mask to the coordinate system of the training BC image to obtain the training boundary mask registered with the training BC image; evaluate the confidence of the EBSD data points in the training EBSD image to obtain the training seed label image.

[0035] Step S203: Construct a preset neural network model, using the seed label map and BC map from the training dataset as input, and combine them with the corresponding training boundary mask map to construct a loss function, train the neural network model, and obtain the EBSD image completion model.

[0036] The training method for the EBSD image completion model provided in this embodiment provides high-precision tissue morphology boundaries by introducing scanning electron microscope images of the same field of view, and selects high-confidence seed labels by combining the confidence assessment of EBSD data points. This effectively solves the problem of supervision signal defects caused by the reliance on incomplete labels of EBSD itself in traditional methods. By introducing a tissue boundary mask map that is strictly registered with the EBSD field of view, the loss function is constructed by combining its clear physical contour constraints with the contrast features of the BC map, which guides the neural network model to perform reasonable inference and completion of crystallographic information within the determined physical boundaries. This not only significantly improves the completeness and accuracy of the calibration of key tissue regions in EBSD images, and realizes accurate identification and reliable reconstruction of small or occluded regions such as M / A islands and pearlites, but also fundamentally breaks through the calibration rate bottleneck caused by poor diffraction pattern quality and occlusion effect in traditional EBSD technology.

[0037] This embodiment provides a training method for an EBSD image completion model, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 3 This is a second flowchart of the EBSD image completion model training method according to an embodiment of the present invention, as follows: Figure 3 As shown, the process includes the following steps: Step S301: Obtain the training dataset, which includes multiple sets of training samples. Each set of training samples includes training EBSD images and training SEM images in the same field of view as the training EBSD images.

[0038] For example, a surface-polished standard sample is placed in a scanning electron microscope. A high-resolution secondary electron or backscattered image is acquired when the stage is tilted at 0°, denoted as I0. The stage is then tilted 70°, and an EBSD detector is used to acquire EBSD images of the same target area within the standard sample. The scan step size can be set from 0.05 μm to 1.0 μm, yielding a raw dataset D containing crystal orientation, phase, pattern quality, etc. ebsd .

[0039] Step S302: For each group of training samples, perform the following preprocessing steps: extract the contours of different tissues in the training SEM image to obtain the training tissue boundary mask; extract the contrast data in the training EBSD image to obtain the training BC image; map the training tissue boundary mask to the coordinate system of the training BC image to obtain the training boundary mask registered with the training BC image; evaluate the confidence of the EBSD data points in the training EBSD image to obtain the training seed label image.

[0040] For example, the U-Net deep learning model is used to perform semantic segmentation on the acquired I0 image, extract the contours of different tissues, and the output binarized tissue boundary mask is denoted as M0.

[0041] For example, extract the corresponding Band Contrast Map from the EBSD data, denoted as I. BC .

[0042] In some optional implementations, mapping the training tissue boundary mask to the coordinate system of the training BC image to obtain a training boundary mask registered with the training BC image includes: determining the matching points between the training SEM image and the training BC image; determining the affine transformation matrix using the matching points between the training SEM image and the training BC image; and mapping the training tissue boundary mask to the coordinate system of the training BC image according to the affine transformation matrix to obtain a training boundary mask registered with the training BC image.

[0043] For example, the scale-invariant feature transform (SIFT) algorithm and random sample consensus (RANSAC) are used to complete I0 and I. BC Feature matching and transformation matrix calculation between points. Calculate the transformation matrix from I0 to I0 by matching point pairs. BC The affine transformation matrix H {0->BC} Using bilinear interpolation, M0 is calculated based on H. {0->BC} Transform to I BC Coordinate system, to obtain the mapped boundary mask M BC .

[0044] In some optional implementations, the confidence assessment of EBSD data points in the training EBSD image to obtain a training seed label map includes: calculating the confidence score of each EBSD data point in the training EBSD image; determining whether each confidence score is greater than a preset confidence threshold; filtering EBSD data points with confidence scores greater than the confidence threshold, and combining them to generate a training seed label map.

[0045] For example, the system performs a quantitative confidence assessment on each EBSD data point, based on the confidence score formula:

[0046] in: C i : Confidence level of a point within the scanned area; PQ i The diffraction pattern quality value at a point within the scanning area is obtained from the EBSD results; PQ max The maximum value of the diffraction pattern quality within the scanning area, obtained from the EBSD results; F i : The calibration fitting error of a point within the scanning area, F i The larger the value, the lower the system's confidence in the calibration results, which is obtained from the EBSD results; F0: Error scaling constant, with a value ranging from 0.5 to 2; : Micro-region physical consistency criterion, δ(x) takes the value of 1 when the condition is true, otherwise it is 0; θ represents the average crystal orientation difference between this point and its eight neighboring points. c The orientation difference threshold is set to a value ranging from 2° to 15°. This effectively filters out abnormal calibration points that are inconsistent with the surrounding material due to surface contamination, scratches, or severe distortion of the Kikuchi strip pattern.

[0047] Set confidence threshold C high The value ranges from 0.5 to 1. Only C i ≥C high The point whose phase information is defined as valid and retained when C i <C high The phase information at this point is invalid and needs to be deleted. Traversing all data points will yield C. i ≥C high The phase identifiers of the points are written into a new label map, and the remaining positions are marked blank, generating a sparse seed label map L. seed .

[0048] Step S303: Construct a preset neural network model, using the seed label map and BC map from the training dataset as input, and combine them with the corresponding boundary mask map to construct a loss function, train the neural network model, and obtain the EBSD image completion model.

[0049] In this embodiment, the preset neural network model adopts an improved U-Net architecture. Its encoder uses a pre-trained ResNet-34 as the backbone network to extract multi-scale features from the input image. The ResNet-34 encoder structure is as follows: First, a 7×7 convolutional layer (stride 2) and a 3×3 max-pooling layer (stride 2) are used, followed by four residual stages (layer 1 to layer 4). Each residual stage consists of multiple residual blocks, each containing two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function, with skip connections to achieve identity mapping. The encoder outputs four feature maps at different scales, with sizes of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, and 64, 128, 256, and 512 channels, respectively. The decoder contains four upsampling layers. Each layer is upsampled by a 2×2 transposed convolutional layer (stride 2), then skip-connected to the feature map output by the corresponding layer of the encoder, and subsequently processed by two 3×3 convolutional layers. The final output layer uses 1×1 convolution and the Softmax activation function to output the predicted crystallographic parameter map.

[0050] In some optional implementations, a preset neural network model is constructed, and the seed label map and BC map of the training dataset are used as inputs. A loss function is constructed by combining the corresponding training boundary mask map, and the neural network model is trained to obtain the EBSD image completion model, including the following steps S3031 to S3035.

[0051] Step S3031: Input the seed label map and BC map for training from the training dataset into the neural network model, and output the predicted crystallographic parameter map.

[0052] In one optional implementation, the training seed label map and training BC map from the training dataset are input into the neural network model, and the predicted crystallographic parameter map is output by: extracting multi-scale features from the training BC map and the training seed label map respectively to obtain BC features and seed features; concatenating the seed features and BC features by channels and processing them through convolutional layers and activation functions to generate an attention map; weighting the attention map and the BC features to obtain fused features; and outputting the predicted crystallographic parameter map based on the fused features.

[0053] For example, the model encoders respectively from I BC and I seedExtract multi-scale features, denoted as F. BC and F seed The SEM topographic features and EBSD results are fused based on an attention-guided fusion mechanism, and the calculation formula is as follows: F fused = F BC ⊙ ( Conv( Concat(F BC ,F seed ) ) ) Concat performs channel concatenation, Conv is a convolutional layer, σ is the sigmoid activation function used to dynamically evaluate the importance of each pixel position, and ⊙ represents element-wise multiplication. This operation allows the system to perform F... seed The provided location and category prior information dynamically enhances F. BC The response corresponds to key morphological features and suppresses interference from irrelevant background.

[0054] Step S3032: Calculate the classification loss between the predicted crystallographic parameter map and the seed label map used for training.

[0055] Step S3033: Calculate the regional topological loss between the predicted crystallographic parameter map and the training boundary mask map.

[0056] Step S3034: Calculate the difference between the gradient magnitude map of the predicted crystallographic parameter map and the gradient magnitude map of the training BC map to obtain the edge constraint loss.

[0057] Step S3035: Update the parameters of the neural network model based on classification loss, region topology loss, and edge constraint loss.

[0058] For example, through the composite loss function L total Optimization of the shape of the entire region improves the consistency of image edges: L total =λ1×L seed +λ2×L dice + λ3×L edge .

[0059] Among them, L seed The weighted cross-entropy loss is calculated for all non-background (effective seed) pixel locations, with the weight coefficient λ1 ranging from 0.5 to 10.

[0060] L dice The Dice loss between the predicted results and the actual region is calculated to optimize the overall topological continuity. The weighting coefficient λ2 ranges from 0.3 to 2.

[0061] L edgeThe norm difference between the gradient magnitude map of the predicted result and the gradient magnitude map of the input SEM image I0 is calculated to improve the boundary alignment accuracy. The weight coefficient λ3 ranges from 0.02 to 1.

[0062] Minimize the total loss L using the Adam optimizer total， Under the premise of strictly adhering to the local constraints of sparse seed points, we can make comprehensive use of global morphological context information to complete reliable reasoning and semantic completion of unlabeled regions.

[0063] The training method for the EBSD image completion model provided in this embodiment automatically and accurately segments the complete tissue morphology from conventional scanning electron microscope images, and uses high-confidence EBSD calibration points as seed points to drive the deep learning model to learn and fuse rich morphological features in SEM images. This enables intelligent inference and pixel-level completion of areas that cannot be clearly calibrated by EBSD, significantly improving the accuracy and efficiency of the analysis of key microscopic tissues.

[0064] This embodiment provides a training method for an EBSD image completion model, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 4 This is a flowchart of the EBSD image completion method according to an embodiment of the present invention, as follows: Figure 4 As shown, the process includes the following steps: Step S401: Acquire the scanning electron microscope image of the sample to be tested and the EBSD signal data to be tested, extract the contrast data from the EBSD signal data to be tested, and obtain the BC image to be tested.

[0065] For example, the sample to be tested is placed in a scanning electron microscope, and a high-resolution backscattered electron image I'0 is acquired when the sample is tilted at 0°. The sample stage is then tilted 70°, and EBSD signals are acquired from the same target area in the sample using an EBSD detector, resulting in a dataset D'. ebsd and I' BC .

[0066] Step S402: Extract the contours of different tissues in the scanning electron microscope image to be tested to obtain the boundary mask of the tissue to be tested. Map the boundary mask of the tissue to be tested onto the coordinate system of the BC image to be tested to obtain the boundary mask of the tissue to be tested registered with the BC image to be tested.

[0067] Step S403: Evaluate the credibility of EBSD data points in the EBSD signal data to be tested, and obtain the seed label map to be tested.

[0068] Step S404: Input the BC image to be tested and the seed label image to be tested into the EBSD image completion model trained by the above method to obtain the predicted crystallographic parameter map.

[0069] In other words, based on the trained model, data collection, boundary extraction and mapping, and confidence evaluation are completed sequentially to obtain the result from I'. BC Mapped boundary mask M' BC and the generated sparse seed label map L' seed The three channels or superimposed data are input into the trained completion model.

[0070] Step S405: Based on the predicted crystallographic parameter map and the seed label map to be tested, construct the complete EBSD dataset.

[0071] In some optional implementations, a complete EBSD dataset is constructed based on the predicted crystallographic parameter map and the seed label map to be tested, including: replacing the data in the region of the seed label map to be tested with the data in the corresponding region of the predicted crystallographic parameter map, and retaining the original EBSD signal data in the region of the seed label map to be tested with the confidence level higher than or equal to the preset threshold, thereby generating a complete EBSD dataset.

[0072] For example, for M' BC Each independent closed tissue region R is divided i Perform the following operations in R: i Inside, L' seed Medium confidence level C i ≥ C high Data points, D' final Fully inherit D' ebsd Information in; L' seed Medium confidence level C i <C high Data points, copy R i D' of the seed point with the highest confidence ebsd Fill in the information; iterate through M' BC After completing the above steps for all the identified organizational regions, the final complete EBSD dataset D' is generated. final .

[0073] The EBSD image completion method provided in this embodiment, by fusing a high-precision tissue boundary mask and a high-confidence seed label, collaboratively guides the EBSD image completion model to make predictions, which can effectively ensure the accuracy of the completion results in terms of physical boundaries and crystallographic features.

[0074] To more clearly illustrate the training method of the EBSD image completion model and the EBSD image completion method of the present invention, a specific example is given: 1. Collection of data from the same source ① Select a hot-rolled silicon steel plate, and after multiple passes of grinding and polishing from coarse to fine on the surface to be tested, place it in a scanning electron microscope. Acquire a high-resolution backscattered electron (SE) image when the sample stage is tilted at 0°. (Example:) Figure 5 As shown, denoted as I1. The test range is... Figure 5 The part outlined in the middle.

[0075] ② Tilt the sample stage 70° and use an EBSD detector to perform EBSD data acquisition on the same target area in ①. Set the scan step size to 0.1μm and obtain the EBSD data result D1.

[0076] ③ The BC diagram of the sample is as follows Figure 6 As shown, denoted as I 1BC The corresponding IPF (Inverse Pole Figure Map) is as follows: Figure 7 As shown.

[0077] 2. High-precision image registration ① The images acquired in the same source data acquisition are processed, and the binary tissue boundary mask extracted by the U-Net deep learning model is denoted as M1.

[0078] ②For I1 and I 1BC Feature matching is performed between them to obtain the mapped boundary mask M. 1BC .

[0079] 3. Confidence assessment ① The system performs a quantitative confidence assessment on each EBSD data point, based on the confidence score formula:

[0080] Among them: PQ max =100, F0=1, θc=5°.

[0081] ② Set the confidence threshold C high The value is 0.9.

[0082] ③ Traverse all data points in D1, and you will have C i ≥ C high The phase identifiers of the points are written into a new label map, and the remaining positions are marked blank, generating a sparse seed label map L. 1seed .

[0083] 4. Deep learning model training ① Feature fusion: The model encoder obtains features from I... 1BC and I 1seed Extract multi-scale features, denoted as F. 1BC and F 1seed .

[0084] ② Loss function: through the composite loss function L total , where λ1=3, λ2=0.8, λ3=0.3, to complete the reasoning for the unlabeled region.

[0085] Specifically, the training dataset contains 2000 training samples, with 80% used for training, 10% for validation, and 10% for testing. Model training uses the Adam optimizer with an initial learning rate of 0.001 and a cosine annealing learning rate decay strategy. The training iterations (epochs) are 200, and the batch size is 100. During training, model parameters are saved every 5 epochs, and the optimal model is selected based on the Dice coefficients on the validation set.

[0086] 5. Intelligent Reasoning and Data Completion ① Traverse M` BC After completing the above steps for all the identified organizational regions, the complete EBSD dataset D is finally generated. 1final .

[0087] The output of the IPF-GB (Inverse Pole Figure - Grain Boundary Map) result is as follows: Figure 8 As shown.

[0088] In summary, this invention uses high-confidence calibration results from EBSD data as sparse "seed" or "guide" information, and utilizes a deep learning model to intelligently fuse the rich morphology and contrast information provided by high-resolution scanning electron microscope images, thereby inferring and completing missing or blurred tissue regions in EBSD data.

[0089] Specifically, firstly, distortion-free high-resolution SEM images are used to accurately segment the clear physical contours of key tissues such as M / A islands and pearlite, thus locking their spatial distribution from a morphological perspective. Based on this, EBSD data from the same region are quantitatively evaluated for confidence, and high-confidence points with high diffraction pattern quality, good fit, and consistency with neighboring orientations are selected as reliable "crystallographic seeds" within the region.

[0090] This strategy allows the completion process to reference both the objective morphological boundaries provided by SEM images, ensuring high spatial resolution, and the local high-precision crystallographic measurement truth values ​​provided by EBSD. This method effectively overcomes the blind spots in tissue region identification found in traditional methods, achieving accurate completion of occluded areas and fundamentally improving the completeness and quantitative statistical accuracy of the final phase distribution.

[0091] This embodiment also provides a training device for an EBSD image completion model, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0092] This embodiment provides a training device for an EBSD image completion model, such as... Figure 9 As shown, it includes: The first acquisition module 901 is used to acquire the training dataset, which includes multiple sets of training samples. Each set of training samples includes training EBSD images and training scanning electron microscope images in the same field of view as the training EBSD images.

[0093] The preprocessing module 902 is used to perform the following preprocessing steps for each group of training samples: extract the contours of different tissues in the training scanning electron microscope image to obtain the training tissue boundary mask; extract the contrast data in the training EBSD image to obtain the training BC image; map the training tissue boundary mask to the coordinate system of the training BC image to obtain the training boundary mask registered with the training BC image; and evaluate the confidence of the EBSD data points in the training EBSD image to obtain the training seed label image.

[0094] The training module 903 is used to construct a preset neural network model. It takes the seed label map and the BC map of the training dataset as input, and constructs a loss function in combination with the corresponding training boundary mask map to train the neural network model and obtain the EBSD image completion model.

[0095] In some optional implementations, the preprocessing module 902 is specifically used to: determine the matching points between the training SEM image and the training BC image; determine the affine transformation matrix through the matching points between the training SEM image and the training BC image; and map the training tissue boundary mask image onto the coordinate system of the training BC image according to the affine transformation matrix to obtain the training boundary mask image registered with the training BC image.

[0096] In some optional implementations, the preprocessing module 902 is specifically used to: calculate the confidence score of each EBSD data point in the training EBSD image; determine whether each confidence score is greater than a preset confidence threshold; filter EBSD data points with confidence scores greater than the confidence threshold, and combine them to generate a seed label image for training.

[0097] In some optional implementations, the training module 903 is specifically used to: input the training seed label map and the training BC map from the training dataset into the neural network model, and output the predicted crystallographic parameter map; calculate the classification loss between the predicted crystallographic parameter map and the training seed label map; calculate the region topology loss between the predicted crystallographic parameter map and the training boundary mask map; calculate the difference between the gradient magnitude map of the predicted crystallographic parameter map and the gradient magnitude map of the training BC map to obtain the edge constraint loss; and update the parameters of the neural network model based on the classification loss, the region topology loss, and the edge constraint loss.

[0098] In some optional implementations, the training module 903 is specifically used to: extract multi-scale features from the training BC map and the training seed label map respectively to obtain BC features and seed features; concatenate the seed features and BC features through channels and process them through convolutional layers and activation functions to generate an attention map; perform weighted processing on the attention map and BC features to obtain fused features; and output the predicted crystallographic parameter map based on the fused features.

[0099] This embodiment provides a device for completing EBSD images, such as... Figure 10 As shown, it includes: The second acquisition module 1001 is used to acquire the scanning electron microscope image of the sample to be tested and the EBSD signal data to be tested, extract the contrast data in the EBSD signal data to be tested, and obtain the BC image to be tested. The boundary mask determination module 1002 is used to extract the contours of different tissues in the scanning electron microscope image to be tested to obtain the boundary mask of the tissue to be tested, and to map the boundary mask of the tissue to be tested onto the coordinate system of the BC image to be tested to obtain the boundary mask of the tissue to be tested registered with the BC image to be tested. The seed label map determination module 1003 is used to evaluate the credibility of EBSD data points in the EBSD signal data to be tested, and to obtain the seed label map to be tested. The prediction module 1004 is used to input the BC image to be tested and the seed label image to be tested into the EBSD image completion model trained by the above method to obtain the predicted crystallographic parameter map. The combined module 1005 is used to construct the complete EBSD dataset based on the predicted crystallographic parameter map and the seed label map to be tested.

[0100] In some optional implementations, the combination module 1005 is specifically used to: replace the data in the region with a confidence level lower than a preset threshold in the seed label image to be tested with the data in the corresponding region in the predicted crystallographic parameter image, and retain the original EBSD signal data in the region with a confidence level higher than or equal to the preset threshold in the seed label image to be tested, thereby generating a complete EBSD dataset.

[0101] The image completion model training device and / or EBSD image completion device provided in this embodiment of the invention can execute the image completion model training method and / or EBSD image completion method provided in any embodiment of the invention, and have the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0102] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0103] The following is a detailed reference. Figure 11 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from memory 1108 into random access memory (RAM) 1103. The RAM 1103 also stores various programs and data required for the operation of the electronic device. The processor 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0104] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 1108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1109. Communication device 1109 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0105] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1109, or installed from memory 1108, or installed from ROM 1102. When the computer program is executed by processor 1101, it performs the functions defined in the image completion model training method and / or EBSD image completion method of the embodiments of the present invention.

[0106] Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0107] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the training method of the image completion model and / or the EBSD image completion method shown in the above embodiments are implemented.

[0108] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0109] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A training method for an EBSD image completion model, characterized in that, The method includes: Obtain a training dataset, which includes multiple sets of training samples. Each set of training samples includes a training EBSD image and a training scanning electron microscope image with the same field of view as the training EBSD image. For each set of training samples, the following preprocessing steps are performed: extract the contours of different tissues in the training SEM image to obtain a training tissue boundary mask; extract contrast data from the training EBSD image to obtain a training BC image; map the training tissue boundary mask onto the coordinate system of the training BC image to obtain a training boundary mask registered with the training BC image; evaluate the confidence of the EBSD data points in the training EBSD image to obtain a training seed label image. A preset neural network model is constructed. The seed label map and the BC map used for training in the training dataset are used as inputs, and a loss function is constructed by combining the corresponding boundary mask map used for training. The neural network model is then trained to obtain the EBSD image completion model.

2. The method according to claim 1, characterized in that, The step of mapping the training tissue boundary mask map to the coordinate system of the training BC map to obtain a training boundary mask map registered with the training BC map includes: Determine the matching points between the training scanning electron microscope image and the training BC image; The affine transformation matrix is ​​determined by the matching points between the training scanning electron microscope image and the training BC diagram; The training tissue boundary mask is mapped onto the coordinate system of the training BC image using the affine transformation matrix to obtain a training boundary mask registered with the training BC image.

3. The method according to claim 1, characterized in that, The step of evaluating the credibility of EBSD data points in the training EBSD image to obtain the training seed label image includes: Calculate the confidence score for each EBSD data point in the training EBSD image; Determine whether each confidence score is greater than a preset confidence threshold; EBSD data points with confidence scores greater than the confidence threshold are selected and combined to generate the seed label map for training.

4. The method according to claim 1, characterized in that, The training seed label map and the training BC map in the training dataset are used as inputs, and a loss function is constructed by combining the corresponding training boundary mask map. The neural network model is then trained to obtain the EBSD image completion model, which includes: The training seed label map and the training BC map in the training dataset are input into the neural network model, and the predicted crystallographic parameter map is output. Calculate the classification loss between the predicted crystallographic parameter map and the training seed label map; Calculate the region topological loss between the predicted crystallographic parameter map and the training boundary mask map; The difference between the gradient magnitude map of the predicted crystallographic parameter map and the gradient magnitude map of the training BC map is calculated to obtain the edge constraint loss; The parameters of the neural network model are updated based on the classification loss, the region topology loss, and the edge constraint loss.

5. The method according to claim 4, characterized in that, The training seed label map and the training BC map from the training dataset are input into the neural network model, and the output predicted crystallographic parameter map includes: Multi-scale features are extracted from the training BC map and the training seed label map respectively to obtain BC features and seed features; The seed features and the BC features are concatenated by channels and then processed through convolutional layers and activation functions to generate an attention map; The attention map and the BC features are weighted to obtain the fused features; The predicted crystallographic parameter map is output based on the fusion features.

6. A method for completing an EBSD image, characterized in that, The method includes: Acquire the scanning electron microscope image of the sample to be tested and the EBSD signal data to be tested, extract the contrast data from the EBSD signal data to be tested, and obtain the BC image to be tested. The contours of different tissues in the scanning electron microscope image to be tested are extracted to obtain the boundary mask of the tissue to be tested. The boundary mask of the tissue to be tested is mapped onto the coordinate system of the BC image to be tested to obtain the boundary mask of the tissue to be tested registered with the BC image to be tested. The reliability of the EBSD data points in the EBSD signal data to be tested is evaluated to obtain the seed label map to be tested; The BC image to be tested and the seed label image to be tested are input into the EBSD image completion model trained using the method described in any one of claims 1 to 5 to obtain the predicted crystallographic parameter map; Based on the predicted crystallographic parameter map and the seed label map to be tested, a complete EBSD dataset is constructed.

7. The method according to claim 6, characterized in that, The complete EBSD dataset is constructed based on the predicted crystallographic parameter map and the seed label map to be tested, including: The data in the regions of the seed label map to be tested with confidence levels below a preset threshold are replaced with the data in the corresponding regions of the predicted crystallographic parameter map, while the original EBSD signal data in the regions of the seed label map to be tested with confidence levels above or equal to the preset threshold are retained to generate a complete EBSD dataset.

8. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the training method of the EBSD image completion model according to any one of claims 1 to 5 and / or the EBSD image completion method according to any one of claims 6 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to execute the training method of the EBSD image completion model according to any one of claims 1 to 5 and / or the EBSD image completion method according to any one of claims 6 to 7.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the training method of the EBSD image completion model according to any one of claims 1 to 5 and / or the EBSD image completion method according to any one of claims 6 to 7.

Images