An EBSD efficient noise reduction and data repair system and method

Abstract

The present invention belongs to the technical field of image denoising, and in particular relates to an efficient EBSD denoising and data restoration system and method. The system comprises: a variational autoencoder (VAE) for generating a noise mask corresponding to an input image, learning the data's latent representation through encoding and decoding processes, and optimizing the modeling of noise distribution; a foreground network (ReNet) for extracting background information from noisy images and restoring a clearer background image by learning the residual information from the denoising process. PReNet is designed as a recursive convolutional neural network, primarily used to effectively extract and restore background information from noisy EBSD images. Its unique iterative and residual learning structure improves detail preservation and noise removal capabilities; and a generative adversarial network (SAGANDiscriminator) employs a convolutional neural network architecture with a self-attention mechanism and spectral normalization to ensure stability and efficiency during adversarial training.

Description

Technical Field

[0001] The present invention belongs to the technical field of image noise reduction, and in particular relates to an EBSD high-efficiency noise reduction and data repair system and method. Background Art

[0002] Electron backscatter diffraction (EBSD) is a technique widely used in scanning electron microscopy (SEM) to analyze the crystallographic structure of materials. By analyzing the pattern of electrons scattered back from the sample surface, microstructural information about the material, such as crystal orientation, phase identification, and grain size, can be obtained. However, EBSD data is often affected by various noise sources, such as the instrument itself, improper sample preparation, or external environmental factors. Therefore, image noise reduction plays a very important role in EBSD data analysis to ensure high-quality data acquisition.

[0003] Currently, EBSD image noise reduction primarily relies on various filter techniques. Commonly used methods include mean filters, median filters, Kuwahara filters, smoothing spline filters, halfquadratic filters, and infimal convolution filters. Each of these filters has its own advantages and disadvantages. Mean and median filters are simple and efficient, but they can destroy image details when dealing with large areas of noise. Kuwahara filters are effective at preserving edges, but are computationally complex. Smoothing spline and halfquadratic filters offer a good balance between noise removal and detail preservation, but they are still insufficient when dealing with complex noise.

[0004] Based on the above analysis, existing technologies face the following urgent technical challenges: Mean and median filters are simple and efficient, but they can destroy image details when dealing with large areas of noise. Kuwahara filters are effective at preserving edges, but they are computationally complex. Smoothing splines and semi-quadratic filters offer a good balance between noise removal and detail preservation, but they are still insufficient when dealing with complex noise. Summary of the Invention

[0005] To address the challenges of existing technologies, the present invention provides a highly efficient EBSD noise reduction and data restoration system and method. This system aims to optimize noise processing and improve image quality through a deep learning model, making it particularly suitable for microstructural analysis in materials science and engineering. The system consists of three core components: a variational autoencoder (VAE), a foreground network (PReNet), and a generative adversarial network discriminator (SAGANDiscriminator). Together, these components enable highly efficient noise reduction and data restoration in complex noisy environments.

[0006] The present invention is implemented as follows: an EBSD efficient noise reduction and data repair system, comprising:

[0007] Variational Autoencoder (VAE): used to generate a noise mask corresponding to the input image, learn the potential representation of the data through encoding and decoding processes, and optimize the modeling of the noise distribution;

[0008] Foreground Network (PReNet): Responsible for extracting background information from noisy images and restoring a clearer background image by learning the residual information from the denoising process. Designed as a recursive convolutional neural network, PReNet is primarily used to effectively extract and restore background information from noisy EBSD images. This network improves its ability to preserve detail and eliminate noise through its unique iterative and residual learning structure.

[0009] SAGANDiscriminator: A convolutional neural network architecture that uses a self-attention mechanism and spectral normalization to ensure stability and efficiency in adversarial training.

[0010] Furthermore, the variational autoencoder (VAE) in this invention consists of two core components: an encoder and a decoder. The encoder is designed to progressively extract image features through a series of convolutional layers, including five, each followed by a Reluctant Unit (ReLU) activation function to increase nonlinearity, helping to capture more complex data characteristics. The stride and padding of each convolution layer are designed to progressively reduce the spatial dimension while increasing the feature dimension. The output of the last convolutional layer is flattened and transformed through a fully connected layer (Linear layer) to an output with twice the dimension of the latent space (nz), in order to simultaneously generate the mean (mu) and logarithmic variance (logvar). Through a reparametrization technique, the mean and logarithmic variance of the encoder output are used to generate latent variables that satisfy a specific distribution. This step is critical to the model's generative capabilities and training stability. The decoder comprises a series of transposed convolutional layers (ConvTranspose2d) to progressively convert the latent space encoding back to the spatial resolution and number of channels of the original image. Each transposed convolutional layer is also followed by a Reluctant Unit (ReLU) activation function to preserve the nonlinear characteristics. The decoder is designed to gradually reconstruct image details, outputting the original image in the dimensions and number of channels from the final transposed convolutional layer. The VAE architecture connects the encoder and decoder to complete end-to-end image processing. The input image is first converted into a latent space representation by the encoder, then reconstructed by the decoder, outputting the reconstructed image.

[0011] Furthermore, during training, the quality of the reconstruction and the distribution of the latent representation are jointly optimized to achieve optimal noise reduction and data representation accuracy. This VAE architecture is particularly well-suited to handling complex noise patterns in EBSD images. By fine-tuning the configuration of the convolutional layers and the latent space, it can effectively capture and reproduce the subtle differences in the material's microstructure. Furthermore, the reparameterization technique provides the necessary stability for model training, allowing the network to better adapt to the characteristics of the data during learning, thereby improving the quality and realism of the generated images.

[0012] Furthermore, the foreground network architecture includes: (1) Initialization and input layer: PReNet first processes the input image through a sequence of standard convolutional layers and ReLU activation functions (conv0) to initialize feature extraction. (2) Residual blocks: The network contains five residual blocks (res_conv1 to res_conv5), each of which consists of two convolutional layers and ReLU activation functions. These residual blocks help the network learn the detailed features of the input data while maintaining the stability of the information flow and preventing information loss during training. (3) Recursion and gating mechanism: The network uses a recursive structure, and each iteration combines the previous output with the original input image to gradually refine the image recovery. This recursive strategy enhances the network's adaptability and ability to handle noise. At each iteration step, the flow of information is controlled by the gating mechanism (conv_i, conv_f, conv_g, conv_o). These gating layers use Sigmoid and Tanh activation functions to simulate the input gate, forget gate, output gate, and unit state of LSTM to optimize feature transfer and long-term dependency learning.

[0013] Furthermore, after each iteration, the network maps the final features back to the image space through a convolutional layer (conv) to generate a restored image. The image generated at each step is added to the original input, gradually reducing the residual noise and enhancing the image quality. Ultimately, the network outputs a series of gradually improved images (x_list), each of which represents the result of an iterative step and allows further analysis and selection of the optimal output image. PReNet effectively handles the noise problem in EBSD images through its complex recursive structure and sophisticated residual learning strategy. The use of recursion enables the network to repeatedly refine the image at multiple levels, thereby more accurately restoring the clean background. The gating mechanism ensures the effective flow of information in the network, avoiding the problem of information explosion or dissipation in multiple iterations. This structure not only improves the denoising performance, but also maintains the original details and quality of the image. It is a particularly effective technical solution in complex noisy environments.

[0014] Furthermore, the primary convolutional layer uses a spectral normalization-wrapped convolutional layer with the LeakyReLU activation function to initially extract basic image features. Intermediate convolutional layers further deepen feature extraction through spectral normalization and LeakyReLU, gradually increasing the level of feature abstraction. The self-attention layer calculates the correlation between different locations in the feature map to generate an attention weight map, strengthening the network's ability to focus on important areas of the image. For higher-resolution input images, the network further deepens feature extraction through additional convolutional layers. Finally, the output layer maps the features to an output, which is used to determine the authenticity of the input image.

[0015] Another object of the present invention is to provide an EBSD efficient noise reduction and data repair method for realizing the EBSD efficient noise reduction and data repair system, comprising:

[0016] S1, uses the Adam optimization algorithm to optimize the parameters of the three network models respectively, sets the learning rate and multi-step learning rate scheduler (MultiStepLR) to adapt to the training requirements of different stages;

[0017] S2, through iterative training, continuously updates the network parameters and uses the SSIM loss function to evaluate the image quality to ensure that the generated images are visually closer to the real images;

[0018] S3 applies the gradient penalty strategy to stabilize the training process and enhances the model's ability to learn details through interpolation techniques and adversarial losses.

[0019] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the EBSD efficient noise reduction and data repair method.

[0020] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the EBSD efficient noise reduction and data repair method.

[0021] Another object of the present invention is to provide an information data processing terminal, which includes the EBSD high-efficiency noise reduction and data repair system.

[0022] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0023] First, addressing the limitations of existing technologies, this paper proposes a method based on adversarial reinforcement learning. By leveraging deep learning techniques, specifically adversarial network architectures, this method effectively reduces noise in EBSD images while preserving image detail. This method not only effectively handles large areas of noise but also accurately fills in data gaps caused by noise, significantly improving the analysis quality and accuracy of EBSD data. Compared to traditional filters, this method offers significant advantages in handling complex and irregular noise, providing a more advanced technical solution for EBSD data analysis.

[0024] Second, as auxiliary evidence of the invention's inventiveness, it is also reflected in the following important aspects:

[0025] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0026] In actual use, a major pain point faced by EBSD (electron backscatter diffraction) technology is that almost all experimental data need to be noise-reduced after EBSD characterization. This is because EBSD data is often affected by factors such as improper sample preparation, instrument noise, or external environment. In particular, the noise problem is more significant for deformed samples. Effective noise reduction can not only improve data quality, but also ensure the accuracy of crystal structure analysis. Although commercial EBSD post-processing software currently on the market, such as AZTEC, OIM, and CHANNEL, provide certain noise reduction solutions, these software are usually expensive (at least 1,500 yuan / year) and most are based on traditional filtering technology. Although these filtering methods can reduce noise to a certain extent, their data recovery effect faces great challenges for samples with a resolution below 60%, and they are often unable to effectively restore or preserve the microscopic details of the sample, especially when dealing with samples with high noise or complex microstructures.

[0027] The present invention provides a highly efficient EBSD noise reduction and data restoration system and method. This system utilizes an advanced deep learning model to optimize noise processing, thereby significantly improving the quality and reliability of EBSD data. The anticipated benefits and commercial value of this technical solution are primarily reflected in its efficient noise reduction capabilities. The deep learning model can identify and separate noise and useful signals in the data, more accurately recovering microstructural details than traditional filtering techniques, particularly when processing high-noise or complex samples. Secondly, this technology can improve data processing efficiency. The automated deep learning processing process reduces reliance on specialized operators, making data processing faster and easier, significantly saving analysis time and costs. This further expands market potential. With its superior performance and reasonable cost, this technology is expected to capture the EBSD post-processing market and attract more users to switch from traditional software to the system of the present invention, particularly those in research and industry with stringent requirements for high data quality. The present invention can be marketed through various business models, such as subscription, licensing, or one-time purchase, to attract a broad potential customer base. In summary, the EBSD high-efficiency noise reduction and data repair system of the present invention addresses the limitations of existing technologies on the market by providing advanced and efficient solutions, bringing practical data quality improvements to users, thereby gaining an advantage in the fierce market competition.

[0028] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0029] The present invention's efficient EBSD noise reduction and data restoration system optimizes noise processing by incorporating a deep learning model, filling a technological gap in efficient noise reduction and high-quality data recovery both domestically and internationally. While existing EBSD technology and related software are widely used in the market, they largely rely on traditional filtering techniques. These methods are limited in their effectiveness when processing data from complex or high-noise environments, particularly when it comes to preserving microstructural details.

[0030] (3) The technical solution of the present invention solves the technical problems that people have been eager to solve but have never been able to solve successfully:

[0031] The present invention utilizes a highly efficient EBSD data noise reduction and restoration system, implemented through a deep learning model, to successfully address a key technical challenge that has long plagued the fields of materials science and engineering: how to effectively reduce EBSD data noise while preserving microstructural detail. Traditional noise reduction techniques, while capable of reducing data noise to a certain extent, often do so at the expense of image detail. This is particularly true when processing complex or high-noise data, and they are unable to effectively distinguish subtle differences between noise and microstructure. The deep learning approach employed in this invention, by training large amounts of data to learn the complex relationship between noise and true signal, enables precise noise identification and filtering while optimizing image detail. This breakthrough not only improves the accuracy of data analysis but also significantly enhances the application value of EBSD technology in materials characterization and performance evaluation. Furthermore, this advanced data processing method is crucial for increasing the automation of analysis and lowering operational barriers, enabling more researchers and engineers to utilize this technology for higher-quality materials analysis, thereby driving technological advancement and innovation across the entire field.

[0032] (4) The technical solution of the present invention overcomes technical prejudice:.

[0033] The present invention's efficient EBSD noise reduction and data repair system successfully overcomes the technical biases that have long existed in traditional EBSD data processing technology. Traditional noise reduction methods, especially those based on simple filters, are often based on assumptions or preset standards, such as the uniform distribution of noise or specific types of noise. These assumptions do not always hold true in practical applications. For example, under different material types and complex experimental conditions, the manifestation of noise can be extremely diverse, and traditional methods are unable to adapt in these situations, resulting in deviations or distortions in data processing results. In contrast, the deep learning model used in the present invention does not rely on these fixed assumptions, but is able to adaptively understand the complex relationship between noise and signal by learning from a large amount of actual data. This method makes the data processing process more precise and flexible, and can be optimized according to different data characteristics, thereby effectively reducing the errors and limitations caused by the biases of traditional technologies. Through this intelligent data analysis method, the present invention provides a more scientific and fair data processing capability for the application of EBSD technology, significantly improving the quality and reliability of the data, which is of great significance for both scientific research accuracy and industrial applications.

[0034] Third, existing electron backscatter diffraction (EBSD) image denoising and data restoration technologies face challenges such as complex noise, loss of detail, and low computational efficiency. Traditional methods typically rely on simple filtering techniques or linear models, which struggle to effectively remove complex noise. They also tend to lose image detail during the denoising process, resulting in poor restored image quality. Furthermore, most existing methods are computationally complex and time-consuming, making them unsuitable for the rapid processing of large datasets.

[0035] This paper introduces a variational autoencoder (VAE) to generate noise masks and learn latent representations of input images. Through encoding and decoding, VAEs can capture the complex characteristics of image data, optimize the modeling of noise distribution, and generate high-quality denoised images. The use of VAEs not only improves the effectiveness of image denoising but also enhances the accuracy of data restoration, ensuring the preservation of image detail and the effective removal of noise.

[0036] The foreground network in this paper, specifically the application of a recursive convolutional neural network (PReNet), represents a significant breakthrough in existing techniques. Through its unique iterative and residual learning architecture, PReNet effectively extracts and recovers background information from noisy EBSD images. This network preserves more image detail during denoising, improving image clarity and quality while addressing the detail loss common in traditional methods.

[0037] The present invention also introduces a generative adversarial network (GAN) discriminator, employing a convolutional neural network architecture with a self-attention mechanism and spectral normalization to ensure the stability and efficiency of adversarial training. The use of the GAN discriminator enables the denoising and data restoration system to generate more realistic, higher-quality images. The self-attention mechanism improves the model's ability to capture global information, while spectral normalization enhances the stability and efficiency of training, significantly improving the overall performance of the system.

[0038] This paper combines a variational autoencoder, a foreground network (PReNet), and a generative adversarial network (GAN) discriminator to construct an efficient EBSD denoising and data restoration system. This system excels at handling complex noise while preserving image detail, significantly improving image quality and processing efficiency. Compared to traditional methods, this paper not only addresses several key issues in the existing technology but also achieves significant technological advancements in the fields of image denoising and data restoration, providing a more powerful and efficient solution for related applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 1. This is a structural diagram of an EBSD efficient noise reduction and data repair system provided by an embodiment of the present invention;

[0040] Figure 2 This is a flow chart of the EBSD efficient noise reduction and data repair method provided by an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of verifying samples under different noise conditions provided by an embodiment of the present invention;

[0042] Figure 4 The following are the Loss evolutions under different learning rates provided by the embodiment of the present invention: (a) 0.01; (b) 0.001; (c) 0.0001;

[0043] Figure 5 It is a schematic diagram of the model structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] The following are two specific examples of EBSD's efficient noise reduction and data repair system:

[0046] Example 1:

[0047] In this embodiment, an EBSD efficient denoising and data restoration system is applied to EBSD image analysis of the microstructure of metallic materials. The system receives a noisy EBSD image as input and first generates a noise mask corresponding to the input image through a variational autoencoder (VAE). The encoder part of the VAE contains five convolutional layers, each of which is followed by a ReLU activation function to extract image features and reduce the spatial size while increasing the feature dimension. Then, latent variables are generated using the reparameterization technique, which are gradually converted back to the spatial resolution and number of channels of the original image through a series of transposed convolutional layers in the decoder part to output a reconstructed image. Next, the foreground network PReNet extracts background information from the noisy image and restores a clearer background image through its unique iterative and residual learning structure. Finally, the adversarial generative network discriminator adopts a convolutional neural network architecture with a self-attention mechanism and spectral normalization to ensure stability and efficiency in adversarial training, thereby outputting a denoised and restored EBSD image.

[0048] Example 2:

[0049] In this embodiment, an EBSD efficient denoising and data repair system is used for cell image analysis in the biomedical field. The system also receives a noisy cell EBSD image as input and generates a noise mask through a variational autoencoder (VAE). The encoder and decoder parts of the VAE are the same as those in Example 1, both containing convolutional layers and transposed convolutional layers, as well as ReLU activation functions, for extracting image features and reconstructing images. The foreground network PReNet is then responsible for extracting cell background information from the noisy image and restoring a clearer cell image. In this process, the iterative and residual learning structure of PReNet is particularly helpful in retaining the detailed features of the cell and eliminating noise. Finally, the adversarial generative network discriminator ensures stability and efficiency in adversarial training, outputs denoised and repaired cell EBSD images, and provides more accurate and reliable image data for biomedical research.

[0050] like Figure 1As shown, the system architecture is as follows: Variational Autoencoder (VAE): used to generate a noise mask corresponding to the input image, learn the potential representation of the data through the encoding and decoding process, and optimize the modeling of the noise distribution. The variational autoencoder (VAE) in the present invention includes two core parts: an encoder and a decoder. The encoder is designed to gradually extract image features through a series of convolutional layers. These layers include five convolutional layers, each followed by a ReLU activation function to increase nonlinearity, which helps to capture more complex data characteristics. The stride and padding of each convolution layer are designed to gradually reduce the spatial size while increasing the feature dimension. The output of the last convolutional layer is flattened and converted into an output with twice the dimension of the latent space through a fully connected layer (Linear layer) in order to simultaneously generate the mean and logarithmic variance. Through the reparametrization technique, the mean and logarithmic variance of the encoder output are used to generate latent variables that meet a specific distribution. This step is key to the model's generation ability and training stability. The decoder consists of a series of transposed convolutional layers (ConvTranspose2d), which are used to gradually convert the encoding of the latent space back to the spatial resolution and number of channels of the original image. Each transposed convolutional layer is also followed by a ReLU activation function to maintain nonlinear characteristics. The decoder is designed to gradually reconstruct image details, outputting the size and number of channels of the original image from the final transposed convolutional layer. The overall architecture of VAE completes end-to-end image processing by connecting these two main parts (encoder and decoder). The input image is first converted into a representation of the latent space by the encoder, and then reconstructed by the decoder, outputting the reconstructed image.

[0051] During training, the quality of the reconstruction and the distribution of the latent representation are jointly optimized to achieve optimal noise reduction and accurate data representation. This VAE architecture is particularly well-suited to handling complex noise patterns in EBSD images. By fine-tuning the configuration of the convolutional layers and the latent space, it can effectively capture and reproduce the subtle nuances of the material's microstructure. Furthermore, the reparameterization technique provides the necessary stability for model training, allowing the network to better adapt to the characteristics of the data during learning, thereby improving the quality and realism of the generated images.

[0052] Foreground network (PReNet): Responsible for extracting background information from noisy images. The network restores a clearer background image by learning the residual information in the denoising process. PReNet is designed as a recursive convolutional neural network, mainly used to effectively extract and restore background information from noisy EBSD images. The network significantly improves the ability to preserve details and eliminate noise through its unique iterative and residual learning structure. The network architecture includes: (1) Initialization and input layer: PReNet first processes the input image through a sequence of standard convolutional layers and ReLU activation functions to initialize feature extraction. (2) Residual block: The network contains five residual blocks, each of which consists of two convolutional layers and ReLU activation functions. These residual blocks help the network learn the detailed features of the input data while maintaining the stability of the information flow and preventing information loss during training. (3) Recursion and gating mechanism: The network uses a recursive structure, and each iteration combines the previous output with the original input image to gradually refine the image restoration. This recursive strategy enhances the network's adaptability and processing capabilities to noise. In each iteration step, the flow of information is controlled by the gating mechanism (conv_i, conv_f, conv_g, conv_o). These gating layers use Sigmoid and Tanh activation functions to simulate the input gate, forget gate, output gate and unit state of LSTM to optimize feature transfer and long-term dependency learning.

[0053] After each iteration, the network maps the final features back to the image space through a convolutional layer to generate a restored image. The image generated at each step is added to the original input, gradually reducing the residual noise and enhancing the image quality. Ultimately, the network outputs a series of gradually improved images, each of which represents the result of an iterative step. This allows further analysis and selection of the optimal output image. PReNet effectively handles the noise problem in EBSD images through its complex recursive structure and sophisticated residual learning strategy. The use of recursion enables the network to repeatedly refine the image at multiple levels, thereby more accurately recovering the clean background. The gating mechanism ensures the effective flow of information in the network, avoiding the problem of information explosion or dissipation in multiple iterations. This structure not only improves the noise reduction performance, but also maintains the original details and quality of the image. It is a particularly effective technical solution in complex noisy environments.

[0054] Generative Adversarial Network Discriminator (SAGANDiscriminator): This discriminator utilizes a convolutional neural network architecture with advanced self-attention mechanisms and spectral normalization to ensure stability and efficiency during adversarial training. The primary convolutional layer uses a spectral normalization-wrapped convolutional layer with a LeakyReLU activation function to initially extract basic image features. Intermediate convolutional layers further refine feature extraction using spectral normalization and LeakyReLU, gradually increasing the level of feature abstraction. The self-attention layer calculates the correlation between different locations in the feature map to generate an attention weight map, strengthening the network's ability to focus on important areas of the image. For higher-resolution input images, the network further deepens feature extraction through additional convolutional layers. The final output layer maps features to an output that is used to determine the authenticity of the input image. This design not only optimizes the performance of the discriminator but also ensures continuous improvement in the quality of the generated images during adversarial training, resulting in more accurate and reliable EBSD image analysis results. The combination of these techniques not only improves noise reduction efficiency but also optimizes image quality, ensuring that the final EBSD image more accurately reflects the microstructural properties of the material.

[0055] like Figure 2 As shown, the training process:

[0056] S1, use the Adam optimization algorithm to optimize the parameters of the three network models respectively, set the learning rate and multi-step learning rate scheduler (MultiStepLR) to adapt to the training requirements of different stages.

[0057] S2, through iterative training, continuously updates the network parameters and uses the SSIM loss function to evaluate the image quality to ensure that the generated images are visually closer to the real images.

[0058] S3 applies a gradient penalty strategy to stabilize the training process and enhances the model's ability to learn details through interpolation techniques and adversarial losses.

[0059] During the dynamic operation process, the system first loads the EBSD image from the training dataset and then processes the image through PReNet and VAE to separate noise and background. SAGANDiscriminator evaluates the quality of the synthesized image and provides feedback to optimize the performance of VAE and PReNet. The present invention adopts an adversarial training mechanism, in which SAGANDiscriminator continuously challenges the images generated by VAE and PReNet, forcing the generative network to produce outputs that are increasingly close to the real data. This adversarial training method significantly improves the ability to identify and suppress noise, especially in complex noisy environments. By optimizing the interaction between these networks, the present invention can effectively process noise in EBSD images while retaining important microstructural information, improving the accuracy and reliability of the analysis.

[0060] After training with the above model, the model training evaluation parameters are as follows:

[0061] Test loss: 0.0005482498090714216

[0062] R2 score: 0.9756881897384758

[0063] Mean square error: 0.0005482501

[0064] It can be seen that the overall error is very small, and the model performs very well on the training set.

[0065] In order to further check the generalization ability of the trained network, we verified the samples with different noise conditions. The specific implementation results are as follows: Figure 3 As shown:

[0066] The first row is the clean EBSD original result, the second row is the noise data input to the network. Here we have verified it for different types of noise. The third row is the output of our network. It can be seen that our network can not only have a very good effect on EBSD data with different noise ratios (40%-90%), but also perform well for different types of noise distribution.

[0067] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the EBSD efficient noise reduction and data repair method.

[0068] An application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of an EBSD efficient noise reduction and data repair method.

[0069] An application embodiment of the present invention provides an information data processing terminal, which includes an EBSD high-efficiency noise reduction and data repair system.

[0070] The main application area of ​​the EBSD high-efficiency noise reduction and data repair system of the present invention is the post-processing field of EBSD. Specific applications and related products can adopt the following two strategies:

[0071] 1. Complete buyout model, directly integrated into commercial software: This model allows collaboration with EBSD equipment manufacturers or analysis software companies to integrate the highly efficient noise reduction and data repair technology directly into existing EBSD analysis software as standard or advanced features. This model allows users to directly utilize the noise reduction and data repair capabilities provided by this model when using EBSD equipment and software, without any additional steps. Partners generate revenue by providing end users with upgraded software or paid subscriptions for additional features. This model offers the advantage of rapid technology adoption and stable market acceptance and commercial returns through existing sales and distribution channels.

[0072] 2. API system, which performs data noise reduction by uploading the original EBSD file and then returns the noise-reduced EBSD result, with a single fee: The present invention can also be developed into a cloud-based service platform, where users can upload their original EBSD files to the platform through the API. The platform uses a deep learning model to process the uploaded files, perform efficient data noise reduction and repair, and then returns the processed results to the user. The service can be charged on a per-process basis, or a certain amount of processing credit can be provided through a subscription model. This method is suitable for users who do not have integrated advanced noise reduction functions, or users who have more scattered needs for EBSD data processing. The advantage of this API service is its flexibility and accessibility. Users can enjoy advanced data processing functions without having to install complex software locally.

[0073] These two strategies offer flexible options tailored to varying market needs and user preferences, enabling the technical solution of this invention to be widely applied in the field of EBSD post-processing. This not only solves practical data quality issues for users but also brings significant economic benefits to the holders of this invention.

[0074] The learning rate is an important parameter in deep learning model training. It plays a vital role in the deep learning model training process and has a significant impact on the convergence speed and final performance of the model. Figure 4 As shown in the figure, the mean square error (MSE) changes with the increase of the number of iterations under different learning rates. When the learning rate is set to 0.01 (as Figure 4(a)), this relatively high learning rate enables the model to drop to a lower level of training loss at a rapid rate. However, after about 55 iterations, the model entered a fluctuating state. In addition, there is a significant error difference between the training set and the validation set, which means that the model learns too quickly on the training set, thereby limiting its generalization ability when faced with unseen data. When the learning rate is reduced to 0.001, we observe that the gap between the training loss and the validation loss has narrowed, indicating that the model has shown good adaptability to both training and validation data. This improvement is due to the model adjusting the weights more finely in each iteration, thereby avoiding drastic fluctuations on the loss surface. At the same time, the steady decline and smaller fluctuation of the validation loss after 105 iterations also indicate that a relatively low learning rate helps the model explore and converge to a better area on the loss surface. As the learning rate is further reduced to 0.0001, the convergence rate of the model becomes extremely slow. As Figure 4 As shown in (c), the decrease in both training loss and validation loss is very small, and the model barely reaches a plateau until the number of iterations reaches 400. Although this extremely low learning rate helps avoid overfitting, it also means that the model requires more iterations to reach the same performance level.

[0075] It should be noted that the embodiments of the present invention can be implemented using hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will understand that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such code is provided on a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays and programmable logic devices. They can also be implemented using software executed by various types of processors, or by a combination of the above-mentioned hardware circuits and software, such as firmware. The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by any person skilled in the art within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. An EBSD efficient noise reduction and data repair system, characterized in that: include: Variational autoencoders are used to generate noise masks corresponding to the input image, learn the potential representation of the data through encoding and decoding processes, and optimize the modeling of noise distribution; The foreground network is responsible for extracting background information from noisy images and restoring the background image by learning the residual information during the denoising process. PReNet is a recursive convolutional neural network used to effectively extract and restore background information from noisy EBSD images. Through its unique iterative and residual learning structure, it improves the ability to retain details and eliminate noise. The adversarial generative network discriminator uses a convolutional neural network architecture with self-attention mechanism and spectral normalization to ensure stability and efficiency in adversarial training; The variational autoencoder VAE includes: The encoder progressively extracts image features through a series of convolutional layers, including five convolutional layers. Each layer is followed by a ReLU activation function to increase nonlinearity, which helps capture data characteristics. The stride and padding of each convolution layer are designed to gradually reduce the spatial size while increasing the feature dimension. The output of the last convolutional layer is flattened and converted through a fully connected layer to an output with twice the dimension of the latent space, which is used to simultaneously generate the mean and logarithmic variance. Reparameterization technique: the mean and log variance of the encoder output are used to generate latent variables that satisfy a specific distribution. This step is critical to the model's generative ability and training stability. The decoder consists of a series of transposed convolutional layers that gradually convert the latent space encoding back to the spatial resolution and number of channels of the original image. Each transposed convolutional layer is also followed by a ReLU activation function to maintain nonlinear characteristics. The decoder is designed to gradually reconstruct image details, outputting the size and number of channels of the original image from the final transposed convolutional layer. The overall architecture of VAE completes end-to-end image processing by connecting an encoder and a decoder. The input image is first converted into a latent space representation by the encoder, and then reconstructed by the decoder to output the reconstructed image. The SAGAN Discriminator includes: The self-attention mechanism module is used to enhance the ability to capture global features in convolutional neural networks. By calculating the autocorrelation matrix of the input feature map, it generates a self-attention map to improve the perception of the global information of the image; Spectral normalization module, applied to each layer of the convolutional neural network to stabilize the training process, prevent gradient explosion or vanishing problems, and ensure that the adversarial training between the generator and the discriminator is more balanced and efficient; The discriminator network architecture includes multiple convolutional layers and fully connected layers. It uses a self-attention mechanism and spectral normalization technology to enhance the discriminative ability of the adversarial generative network by comparing the input image with the generated image, thus ensuring the quality and authenticity of the generated image.

2. A method for implementing the EBSD efficient noise reduction and data repair system according to claim 1, characterized in that: The following steps are involved: The Adam optimization algorithm is used to optimize the parameters of the variational autoencoder (VAE), the foreground network (PReNet), and the generative adversarial network (SAGAN) discriminator. The learning rate and multi-step learning rate scheduler are set to meet the training requirements of different stages. Through iterative training, the network parameters are continuously updated, and the structural similarity SSIM loss function is used to evaluate the image quality to ensure that the generated images are visually closer to the real images; A gradient penalty strategy is applied to stabilize the training process, and the model's ability to learn details is enhanced through interpolation techniques and adversarial losses.

3. The EBSD efficient noise reduction and data repair method according to claim 2, characterized in that: The following steps are involved: The foreground network architecture includes initialization and input layers. PReNet first processes the input image through a sequence of standard convolutional layers and ReLU activation functions to initialize feature extraction; Use five residual blocks, each consisting of two convolutional layers and a ReLU activation function. These residual blocks help the network learn the detailed features of the input data while maintaining the stability of the information flow and preventing information loss during training; The network uses a recursive structure. Each iteration combines the previous output with the original input image, gradually refining the image restoration. The recursive strategy enhances the network's adaptability and ability to handle noise. In each iterative step, the flow of information is controlled by a gating mechanism. The gating layer uses Sigmoid and Tanh activation functions to simulate the input gate, forget gate, output gate, and unit state of the long short-term memory (LSTM) to optimize feature transfer and long-term dependency learning.

4. The EBSD efficient noise reduction and data repair method according to claim 2, characterized in that: The following steps are involved: Step 1. After each iteration, the network maps the final features back to the image space through a convolutional layer to generate a restored image; Step 2. The image generated at each step is added to the original input, gradually reducing the residual noise and enhancing the image quality; Step 3. Ultimately, the network outputs a series of progressively improved images, each representing the result of an iterative step, which allows for further analysis and selection of the optimal output image. Step 4. PReNet effectively handles the noise problem in EBSD images through its complex recursive structure and sophisticated residual learning strategy; Step 5. The use of recursion enables the network to repeatedly refine the image at multiple levels, thereby more accurately recovering the clean background. The gating mechanism ensures the effective flow of information in the network, avoiding the problem of information explosion or dissipation in multiple iterations, thereby improving the denoising performance and maintaining the original details and quality of the image.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the EBSD efficient noise reduction and data repair method according to claim 2.

6. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the EBSD efficient noise reduction and data repair method according to claim 2.

7. An information data processing terminal, comprising the EBSD high-efficiency noise reduction and data repair system according to claim 1.

Images