Material microstructure prediction and characterization method based on improved space-time network

Abstract

The invention discloses a material microstructure prediction and characterization method based on an improved space-time network. The method comprises the following steps: placing a sample in a scanning electron microscope chamber for an in-situ tensile test; collecting EBSD data of a target area in different strain stages to obtain an IPF image; preprocessing the IPF images, and forming an IPF image sequence with a fixed length according to a time sequence; and inputting the IPF image of each time step into the trained improved space-time network in sequence according to a time sequence for prediction, and obtaining an IPF prediction image of a future time step. The invention innovatively provides a depth prediction model which is more suitable for microstructure evolution prediction and characterization in the material field, and by introducing a time attention mechanism based on differential information, the problems that an existing space-time prediction model is insufficient in abrupt change feature capture capability and fuzzy in prediction in a microstructure prediction task are solved; and the dynamic processes of grain deformation, orientation change, nucleation and growth of twin crystals and the like in the deformation process of the material can be accurately and efficiently represented.

Description

Technical Field

[0001] This invention relates to the field of deep learning-based material microstructure prediction technology, and in particular to a method for predicting and characterizing material microstructure based on an improved spatiotemporal network, used for multi-timestep prediction and visualization of the microstructure evolution of metallic materials during deformation. Background Technology

[0002] In materials science research, understanding the surface morphology changes of metallic materials under stress is crucial for revealing their mechanical properties and deformation mechanisms. In-situ microscopic characterization techniques acquire real-time information about the microstructure of materials during loading, providing an important means to reveal complex evolutionary behaviors such as grain deformation, orientation rotation, twin nucleation and growth, and grain boundary migration. However, in-situ experiments are usually time-consuming, costly, and limited by experimental conditions and equipment stability, making it difficult to conduct a large number of repetitive experiments under a wide range of operating conditions.

[0003] With the development of deep learning and spatiotemporal sequence prediction technologies, spatiotemporal networks based on models such as convolutional LSTM and spatiotemporal LSTM have been used for tasks such as video prediction and flow field prediction. Existing research has attempted to introduce such spatiotemporal networks into the prediction of material microstructure evolution, achieving prediction of the microstructure evolution process by learning the mapping relationship between historical image frames and future image frames. However, existing technologies still have the following problems when applied to image sequences of the microstructure of real metallic materials: 1. Most models use idealized simulation data as training objects, which makes it difficult to effectively characterize the high nonlinearity, nonstationarity and complex spatiotemporal correlation that are common in the microstructure of real materials, resulting in insufficient ability to capture abrupt behaviors such as twin nucleation and local severe deformation. 2. Traditional spatiotemporal networks typically use fixed forget gates and update gates when modeling time dependencies. They lack fine distinction of the importance of different time steps and are easily affected by noise interference from distant time steps or the masking of stable background information, resulting in phenomena such as blurry prediction images, unclear grain boundaries, and loss of details. 3. Existing attention mechanisms mostly act directly on original features or hidden states, which do not make full use of the "variable" in the micro-organizational evolution, fail to specifically model non-stationary information, and are difficult to take into account the unified representation of local mutation behavior and overall evolutionary trend.

[0004] Therefore, it is necessary to provide an improved spatiotemporal network that can utilize non-stationary information and has spatiotemporal joint attention modeling capabilities to improve the accuracy of microorganism prediction and the structural clarity of the results. Summary of the Invention

[0005] The purpose of this invention is to address the problems of insufficient ability to capture abrupt changes and fuzzy predictions in existing spatiotemporal networks for predicting the microstructure of materials. This invention provides a method for predicting and characterizing the microstructure of materials based on an improved spatiotemporal network, which can achieve high-precision, multi-time-step prediction of the actual microstructure evolution process of metallic materials, while maintaining details of grain morphology, grain boundary structure and twin evolution. This provides a new technical means for materials research and greatly reduces experimental costs and manpower input.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting and characterizing the microstructure of materials based on an improved spatiotemporal network, the method comprising: S1. The original metal sample is mechanically ground and low-temperature electropolished to obtain an in-situ experimental sample; the in-situ experimental sample is fixed in an in-situ tensile device, and then the in-situ tensile device is fixed in the scanning electron microscope chamber to perform an in-situ tensile test along a certain direction. S2. During the in-situ tensile test, electron backscatter diffraction (EBSD) data of the target region of the in-situ experimental specimen at different strain stages are collected. The EBSD data is then processed using EBSD analysis software to obtain the grain orientation distribution (IPF) image corresponding to the EBSD data. The IPF images for each strain stage are numbered from frame 1 to frame m according to the acquisition time sequence. The m consecutive frames of IPF images are then... } as the input sequence, used to predict the IPF image of the next n frames { }; S3. Preprocess the IPF images at each strain stage, and combine the m preprocessed IPF images into a fixed-length IPF image sequence in chronological order, which will be used as the input image sequence. S4. Input the IPF images of each time step of the input image sequence into the trained improved spatiotemporal network for prediction to obtain the IPF prediction image sequence from the (t+1)th time step to the (t+n)th time step. The improved spatiotemporal network uses ST-LSTM as the baseline network and includes ST-LSTM modules connected in sequence, at least two DT-MIM modules, and an output generation layer. The DT-MIM module is an improvement on the MIM module, consisting of an N-TA-S module composed of a cascaded MIM-N module, a temporal attention module (TA), and a MIM-S module. It is used to replace the forget gate in ST-LSTM that models time dependence, so that the update of long-term memory state shows dependence on the non-stationary features extracted within the time window and their temporal importance distribution.

[0007] Specifically, the MIM-N module models the difference between the hidden states of adjacent time steps and extracts non-stationary features representing changes in microstructure from the hidden states; the TA module collects the non-stationary features of the most recent T time steps within a configurable time window, stacks them into a three-dimensional spatiotemporal feature tensor, and jointly extracts local evolution patterns in the temporal and spatial dimensions through 3D convolution, generates temporal attention weights for each spatial location at different time steps, and performs weighted fusion of the non-stationary feature sequences to obtain attention-enhanced non-stationary features; the MIM-S module constructs an approximately stationary representation based on the attention-enhanced non-stationary features and the long-term memory state of the previous time step, which is used to update the long-term memory state of the current time step.

[0008] Furthermore, in step S1, during the stretching process, loading is paused and EBSD data is collected at several preset strain stages; at each strain stage, the sample position is kept unchanged, and crystallographic orientation information of the target area is collected through the EBSD probe equipped with the scanning electron microscope.

[0009] Furthermore, in step S3, the preprocessing of the IPF image includes: noise removal and pixel filling of the original IPF image; cropping and scaling the image to unify the field of view to a fixed-size window containing the target region.

[0010] Furthermore, the improved spatiotemporal network predicts at time step t in the following way: The ST-LSTM module at the bottom of the model is used to receive the input IPF image at the current time step. Hidden states output by ST-LSTM modules at the same layer in the previous time step and long-term memory state And the spatiotemporal memory state output by the highest-level DT-MIM module at the previous time step. Output the spatiotemporal memory state at the current time step. Long-term memory status and hidden state ;in, L This indicates the total number of layers in the ST-LSTM and DT-MIM modules. This is a true IPF image. Use the predicted image for IPF; if no real image is available, use the predicted image. The DT-MIM module, located at the next level below the bottom-level ST-LSTM module, receives the hidden state output by the bottom-level ST-LSTM module at the current time step. and spatiotemporal memory state The hidden state output by the lowest-level ST-LSTM module at the previous time step. The long-term memory state output by the DT-MIM module at the same level in the previous time step. and hidden state and the non-stationary long-term memory state after processing by the N-TA-S module. and stable long-term memory state Output the spatiotemporal memory state at the current time step. Long-term memory status and hidden state ; By analogy, the spatiotemporal memory state output by the highest-level DT-MIM module at the current time step is obtained. Long-term memory status and hidden state ; The output generation layer is a two-dimensional convolutional layer with a kernel size of 1×1, used to store the hidden state of the highest-level DT-MIM module. Mapped to the IPF prediction image at the current time step .

[0011] Furthermore, the specific implementation process of the N-TA-S module is as follows: At time step t in layer l, The difference between the hidden states of the MIM-N module at adjacent time steps in the previous layer - The non-stationary long-term memory state output by the MIM-N module at the same time step as the previous time step As input, non-stationary features representing the amount of change in microstructure evolution are extracted. ; Within the preset time window Within [this section], collect the most recent [time step]. Non-stationary feature sequences at each time step The features at each time step are stacked in the time dimension to form a tensor. Where B represents batch size, C represents number of channels, T represents time window size, H represents height, and W represents width; The The input TA module extracts local spatiotemporal features sequentially through a first 3D convolutional layer with a kernel size of 3×3×3, a group normalization layer, and a ReLU activation layer. Then, it maps these features to a single-channel attention weight tensor through a second 3D convolutional layer with a kernel size of 1×1×1. A ∈ R B×1×T×H×W Apply Softmax normalization to the attention weight tensor A along the time dimension T, so that for any spatial location, the sum of the attention weights at each time step is 1, thus obtaining the temporal attention weights. The time-weighted non-stationary features are obtained by multiplying the time attention weights element-wise with the non-stationary feature sequence and summing along the time dimension. and the attention-weighted non-stationary features Non-stationary characteristics of the current time step After fusion, the data is input into the MIM-S module to construct an approximately stationary representation and update the long-term memory state.

[0012] Furthermore, the non-stationary features of the attention-weighted system Non-stationary characteristics of the current time step The fusion method is as follows: Regarding the Normalization is performed, multiplied by a learnable scaling factor, and then the residual is compared with the non-stationary features of the current time step. Adding them together yields enhanced nonstationary features. This approach is beneficial for adaptively amplifying non-stationary patterns that contribute to prediction while preserving the stability of the basic difference representation. The enhanced non-stationary features combine non-stationary difference information with temporal attention selection information and are used as input to the MIM-S module.

[0013] Furthermore, the improved spatiotemporal network includes a bottom-level ST-LSTM module and several DT-MIM modules connected sequentially in ascending order of depth; each DT-MIM module is configured with an independent N-TA-S module to perform multi-scale temporal attention modeling of non-stationary information at different feature levels. The expression for the processing of the l-th layer DT-MIM module is as follows:

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021] Note that t and l here represent time step and network level, respectively; These are the input gate, modulation gate, forget gate, and output gate, respectively. The temporal weights learned by the temporal attention mechanism are represented; W and b represent the weight matrix and bias vector corresponding to different gates, respectively; σ, * and These represent the Sigmoid activation function, the convolution operator, and the Hadamard product, respectively. Indicates residual connection; MIM-N(∙) indicates the output of the MIM-N module; MIM-S(∙) indicates the output of the MIM-S module.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects: 1. The Temporal Attention (TA) module of this invention only operates on the non-stationary hidden state sequence extracted by the MIM-N module, rather than the original hidden state or the original image. It specifically extracts non-stationary feature sequences that represent changes in microstructure, and only performs weighted modeling on the non-stationary sequence in the Temporal Attention module. This suppresses the interference of the stationary background on attention allocation from the source, which is beneficial to highlighting local abrupt behaviors such as twin nucleation and growth, and grain boundary migration. 2. This invention sets a time window T of adjustable length in the TA module, collecting the non-stationary hidden states of the most recent T time steps in a sliding window manner at each time step, thus realizing the explicit constraint of the scope of temporal attention. By adjusting T, the effective memory depth of historical information of the model can be controlled, and the length of the time window can be matched with the physical time scale of microstructure evolution according to the material system, loading rate, and image acquisition interval, so that the model can highlight short-term strong non-stationary activations and cover long-term cumulative evolutionary information when needed; 3. This invention uses 3D convolution to perform spatiotemporal joint modeling of non-stationary features. The convolution kernel slides simultaneously in the temporal and spatial dimensions to learn the local non-stationary patterns in "a certain time step + spatial neighborhood". This allows the calculation of temporal attention weights to simultaneously perceive different strain stages and their corresponding spatial neighborhood structures, improving the ability to characterize the evolution process in the local temporal neighborhood and reducing the impact of distant historical noise on the current prediction. 4. This invention constructs an N-TA-S module by cascading MIM-N, a 3D convolution-based temporal attention module TA, and MIM-S, replacing the forgetting gate of the traditional LSTM. This makes the update of long-term memory state no longer dependent on a fixed form of gating function, but explicitly dependent on non-stationary difference features and their temporal importance distribution, thus realizing adaptive memory and forgetting of complex micro-organism evolution processes. 5. This invention normalizes and adjusts the intensity of temporal attention output through grouping normalization and learnable scaling parameters, and adds it to the current non-stationary hidden state in the form of residuals to achieve adaptive enhancement of non-stationary information; when temporal attention learns a non-stationary pattern that contributes to prediction, its effect is amplified by scaling parameters; when attention learning is insufficient, the residual path ensures that the original MIM-N representation is not destroyed, thus balancing model performance improvement and training stability. 6. Experimental results show that, compared with the standard ST-LSTM network without differential temporal attention mechanism and the original MIM network without TA module, the improved spatiotemporal network proposed in this invention achieves lower mean square error, higher PSNR and SSIM in IPF image prediction task. The predicted grain boundaries are clearer, the twin strip structure is more complete, and the evolution process is more coherent. It effectively solves the problems of insufficient ability to capture abrupt feature and fuzzy prediction in material microstructure prediction task of existing spatiotemporal networks. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below.

[0024] Figure 1 This is a flowchart of a method for predicting and characterizing the microstructure of materials based on an improved spatiotemporal network.

[0025] Figure 2 A schematic diagram of the architecture for improving the spatiotemporal network DT-MIM.

[0026] Figure 3 This is a schematic diagram of the DT-MIM module.

[0027] Figure 4 This is a schematic diagram of the N-TA-S module.

[0028] Figure 5 This is a schematic diagram showing the time-varying curves of MSE, PSNR, and SSIM for different spatiotemporal networks.

[0029] Figure 6 A visual comparison diagram of prediction results from different spatiotemporal networks. Detailed Implementation

[0030] To enable those skilled in the art to better understand the solutions of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] The purpose of this invention is to provide a method for predicting and characterizing the microstructure of materials based on an improved spatiotemporal network, which solves the problems of insufficient ability to capture abrupt features and fuzzy prediction in existing spatiotemporal networks for microstructure prediction tasks, improves the prediction accuracy of spatiotemporal networks in this type of task, provides a new technical means for materials research, and greatly reduces experimental costs and manpower input.

[0032] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Example: In this embodiment, an in-situ tensile test along the TD direction is performed on Zr-4 alloy using a TESCAN S8000 scanning electron microscope equipped with an in-situ tensile device, and EBSD data at different strain stages are collected. The corresponding IPF images are preprocessed to obtain spatiotemporal data, and an improved spatiotemporal network is used to predict the spatiotemporal data.

[0034] like Figure 1 As shown in the figure, the method for predicting and characterizing the microstructure of materials based on improved spatiotemporal networks provided in this embodiment includes the following steps: S1. Prepare a Zr-4 alloy sample suitable for in-situ tensile testing. The sample is dog-bone shaped, 46 mm long, and contains a gauge length of 1.5 mm and a width of 1.5 mm. Mechanically polish the original tensile specimen with SiC sandpaper of 400 mesh, 800 mesh, 1200 mesh, 3000 mesh, and 5000 mesh in sequence. Electropolish the polished sample in a mixed solution of 70 vol% methanol, 20 vol% ethylene glycol monobutyl ether, and 10 vol% perchloric acid at -30℃ with a voltage of 20V. Fix the sample on the in-situ tensile testing stage, and then fix the in-situ tensile testing stage in the scanning electron microscope chamber. Perform in-situ tensile testing along the transverse (TD) direction of the sample.

[0035] S2. During the in-situ tensile test, EBSD data of different strain levels are collected from the target area of ​​the sample. The corresponding IPF images are generated by EBSD analysis software, and IPF images from time step t-m+1 to time step t are obtained, forming a sequence of microstructure evolution images arranged in chronological order.

[0036] S3. Preprocess the IPF images at each strain stage, including noise removal, bad pixel repair, uniform cropping and scaling, and map the region of interest to a fixed-size image; and downsample the IPF images by bilinear interpolation to reduce the resolution of the IPF images; then combine the IPF images into a fixed-length IPF image sequence in chronological order, with each sequence containing m IPF images.

[0037] S4. The IPF image sequences are packaged into a data structure of [B, L, C, H, W] and input into the improved spatiotemporal network. Specifically, B is the batch size, representing the number of IPF image sequences input into the improved spatiotemporal network at one time; L is the length of the IPF image sequence, with a value of m; C is the number of channels of the IPF image, with a value of 3; H and W represent the height and width dimensions of each IPF image, respectively.

[0038] The network sequentially predicts the IPF image at each time step according to time order to obtain the IPF prediction images for time steps t+1 to t+n. This includes: inputting the IPF image at time step t-m+1 into the trained improved spatiotemporal network and outputting the IPF prediction image at time step t-m+2; inputting the IPF image at time step t-m+2 into the trained improved spatiotemporal network and outputting the IPF prediction image at time step t-m+3; and so on, until the IPF image at time step t is input into the trained improved spatiotemporal network and outputting the IPF prediction image at time step t+1; then inputting the IPF prediction image at time step t+1 into the trained improved spatiotemporal network and outputting the IPF prediction image at time step t+2; and so on, until the IPF prediction image at time step t+n-1 is input into the trained improved spatiotemporal network and outputting the IPF prediction image at time step t+n.

[0039] Among them, such as Figure 2 The improved spatiotemporal network architecture shown is described below. The improved spatiotemporal network makes predictions at time step t as follows: The ST-LSTM module at the bottom of the model is used to receive the input IPF image at the current time step. Hidden states output by ST-LSTM modules at the same layer in the previous time step and long-term memory state And the spatiotemporal memory state output by the highest-level DT-MIM module at the previous time step. Output the spatiotemporal memory state at the current time step. Long-term memory status and hidden state ;in, L This indicates the total number of layers in the ST-LSTM and DT-MIM modules. This is a true IPF image. Use the predicted image for IPF; if no real image is available, use the predicted image.

[0040] The DT-MIM module, located at the next level below the bottom-level ST-LSTM module, receives the hidden state output by the bottom-level ST-LSTM module at the current time step. and spatiotemporal memory state The hidden state output by the lowest-level ST-LSTM module at the previous time step. The long-term memory state output by the DT-MIM module at the same level in the previous time step. and hidden state and the non-stationary long-term memory state after processing by the N-TA-S module. and stable long-term memory state Output the spatiotemporal memory state at the current time step. Long-term memory status and hidden state ; By analogy, the spatiotemporal memory state output by the highest-level DT-MIM module at the current time step is obtained. Long-term memory status and hidden state ; The output generation layer is a two-dimensional convolutional layer with a kernel size of 1×1, used to store the hidden state of the highest-level DT-MIM module. Mapped to the IPF prediction image at the current time step .

[0041] The structure of the DT-MIM network is as follows Figure 3 As shown, the specific processing procedure is expressed as follows:

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049] Note that t and l here represent time step and network level, respectively; These are the input gate, modulation gate, forget gate, and output gate, respectively. The temporal weights learned by the temporal attention mechanism are represented; W and b represent the weight matrix and bias vector corresponding to different gates, respectively; σ, * and These represent the Sigmoid activation function, the convolution operator, and the Hadamard product, respectively. Indicates residual connection; MIM-N(∙) indicates the output of the MIM-N module; MIM-S(∙) indicates the output of the MIM-S module.

[0050] The structure of the N-TA-S module is as follows: Figure 4 As shown, the specific implementation process is as follows: At time step t in layer l, The difference between the hidden states of the MIM-N module at adjacent time steps in the previous layer - The non-stationary long-term memory state output by the MIM-N module at the same time step as the previous time step As input, non-stationary features representing the amount of change in microstructure evolution are extracted. ; Within the preset time window Within [this section], collect the most recent [time step]. Non-stationary feature sequences at each time step The features at each time step are stacked in the time dimension to form a tensor. Where B represents batch size, C represents number of channels, T represents time window size, H represents height, and W represents width; The The input TA module extracts local spatiotemporal features sequentially through a first 3D convolutional layer with a kernel size of 3×3×3, a group normalization layer, and a ReLU activation layer. Then, it maps these features to a single-channel attention weight tensor through a second 3D convolutional layer with a kernel size of 1×1×1. A ∈ R B×1×T×H×W Apply Softmax normalization to the attention weight tensor A along the time dimension T, so that for any spatial location, the sum of the attention weights at each time step is 1, thus obtaining the temporal attention weights. The time-weighted non-stationary features are obtained by multiplying the time attention weights element-wise with the non-stationary feature sequence and summing along the time dimension. and the attention-weighted non-stationary features Non-stationary characteristics of the current time step After fusion, the data is input into the MIM-S module to construct an approximately stationary representation and update the long-term memory state.

[0051] By using the N-TA-S structure described above, the simple forget gate in the traditional LSTM is replaced with a complex gating mechanism that explicitly depends on non-stationary differential information and temporal attention weights, enabling the update of long-term memory state to adaptively adjust to the mutation behavior in the micro-organism evolution.

[0052] By comparing the improved spatiotemporal network of this invention with the original ST-LSTM network without the N-TA-S structure and a network using only the MIM structure without the TA module, the predictive performance of the different spatiotemporal networks was evaluated on the same training and test datasets. Figure 5 The schematic diagrams showing the mean square error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) of different spatiotemporal networks over time illustrate how their prediction errors change with each prediction time step. The results show that DT-MIM consistently exhibits the lowest MSE value across all time steps (lower values ​​indicate better performance), while its PSNR and SSIM values ​​are consistently the highest (higher values ​​indicate better performance), demonstrating a consistent improvement in prediction performance compared to the other two spatiotemporal networks.

[0053] like Figure 6 The diagram showing the visualization comparison of prediction results from different spatiotemporal networks illustrates the visual differences between the prediction results of different spatiotemporal networks at the same time step and the actual frames. The input sequence is displayed at three-frame intervals, and the output sequence is displayed at two-frame intervals. For ease of description, several grains are marked. Twin nucleation and growth occur in grains 1 and 2, while grain 3 exhibits obvious grain boundary contraction and orientation rearrangement. The image predicted by PredRNN only roughly maintains the overall grain layout, with blurred grain boundaries, inconsistent twin evolution trends, and distorted shape in the predicted outline of grain 3, poor temporal continuity, and abrupt changes between frames. MIM has some ability to capture the initial expansion trend of twins, but the predicted twin growth process is not smooth enough. In addition, the evolution of grains 1 and 3 shows that the MIM network does not accurately capture grain deformation. In contrast, the image generated by the improved spatiotemporal network DT-MIM of this invention is clearer, the twin growth process is clear, and the grain boundaries are complete and realistic. The structural changes between multiple frames are coherent, demonstrating good temporal modeling capabilities.

[0054] The results show that the improved spatiotemporal network outperforms existing spatiotemporal networks in predicting the evolution of microorganisms, preserving the integrity, detail, and evolutionary dynamics of the microorganism structure. It demonstrates a strong ability to model complex deformation mechanisms and solves the problems of insufficient ability to capture mutation features and fuzzy prediction in existing spatiotemporal networks in microorganism prediction tasks. It provides new technical means for materials research and greatly reduces experimental costs and manpower input.

[0055] The structure, features, and effects of the present invention have been described in detail above with reference to the embodiments shown in the figures. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, shall be within the protection scope of the present invention as long as they do not exceed the spirit covered by the specification and figures.

Claims

1. A method for material microstructure prediction and characterization based on improved space-time network, characterized in that, The method comprises the following steps: S1, mechanical polishing and low-temperature electric polishing are performed on an original metal sample to obtain an in-situ experimental sample; the in-situ experimental sample is fixed on an in-situ stretching device, and then the in-situ stretching device is fixed in a scanning electron microscope chamber to perform in-situ stretching test; S2, during the in-situ stretching test, EBSD data of a target region of the in-situ experimental sample at different strain stages are collected, IPF images corresponding to the EBSD data are obtained, and IPF images at a t-m+1 time step to a t time step are obtained in time sequence, wherein t is a current time step, and m is a positive integer; S3, the IPF images are preprocessed, and IPF images of a fixed length are sequentially arranged in time sequence to form an input image sequence; S4, each time step IPF image of the input image sequence is sequentially input into a trained improved space-time network for prediction to obtain an IPF prediction image sequence of future time steps; The improved space-time network comprises ST-LSTM modules, at least two DT-MIM modules and an output generation layer which are sequentially connected in a hierarchical manner; the DT-MIM module is an N-TA-S module based on an improved MIM module, which is composed of a MIM-N module, a time attention module TA and a MIM-S module, and is used to replace the forgetting gate in the ST-LSTM for modeling time dependence, so that the update of the long-term memory state depends on the non-stationary features extracted in the time window and the time importance distribution thereof.

2. The method for material microstructure prediction and characterization based on improved space-time network according to claim 1, wherein, In step S1, during the stretching process, the loading is paused at a plurality of preset strain stages, and EBSD data are collected; at each strain stage, the sample position is kept unchanged, and the crystallographic orientation information of the target region is collected by an EBSD probe of a scanning electron microscope.

3. The method for material microstructure prediction and characterization based on improved space-time network according to claim 1, wherein, In step S3, the preprocessing of the IPF image comprises: noise removal and pixel filling of the original IPF image; the image is cropped and scaled to unify the field of view to a fixed size window containing the target region.

4. The method for material microstructure prediction and characterization based on improved space-time network according to claim 1, wherein, The prediction mode of the improved space-time network at the t time step is as follows: The ST-LSTM module at the bottom layer of the model is used to receive the input IPF image at the current time step , the hidden state output by the same layer ST-LSTM module at the previous time step , and the long-term memory state , and the spatiotemporal memory state output by the highest layer DT-MIM module at the previous time step , output the spatiotemporal memory state at the current time step , the long-term memory state , and the hidden state ; wherein, L represents the total number of layers of the ST-LSTM module and the DT-MIM module, is the IPF real image, is the IPF predicted image, and the predicted image is selected if there is no real image; The DT-MIM module at the next layer of the bottommost ST-LSTM module receives the hidden state output by the bottommost ST-LSTM module at the current time step and the spatial-temporal memory state , the hidden state output by the bottommost ST-LSTM module at the previous time step , the long-term memory state output by the DT-MIM module at the same layer at the previous time step and the hidden state , and the non-stationary long-term memory state processed by the N-TA-S module and the stationary long-term memory state , outputs the spatial-temporal memory state at the current time step , the long-term memory state and the hidden state ; By analogy, the spatiotemporal memory state output by the highest-level DT-MIM module at the current time step is obtained , the long-term memory state , and the hidden state ; The output generation layer is a two-dimensional convolution layer with a convolution kernel size of 1x1, used to generate the IPF prediction image at the current time step from the hidden state of the highest layer DT-MIM module mapping to the IPF prediction image at the current time step .

5. The method for material microstructure prediction and characterization based on improved space-time network according to claim 4, characterized in that, The specific implementation process of the N-TA-S module is as follows: At the l-th layer, the t-th time step, , the difference between the hidden states of the MIM-N module at the previous time step and the previous layer - and the non-stationary long-term memory state output by the MIM-N module at the same layer at the previous time step as input to extract non-stationary features representing the change in microstructure evolution ; Within a pre-set time window , collect the non-stationary feature sequence of the last time steps of the current time step , stack the features of each time step in the time dimension to form a tensor , where B represents the batch size, C represents the number of channels, T represents the size of the time window, H represents the height, and W represents the width. The method comprises the following steps: The input TA module sequentially extracts local space-time features through a first 3D convolution layer with a convolution kernel size of 3*3*3, a group normalization layer and a ReLU activation layer, and maps the features into a single-channel attention weight tensor through a second 3D convolution layer with a convolution kernel size of 1*1*1 A ∈ R B×1×T×H×W Softmax normalization is applied to the attention weight tensor A along the time dimension T, so that the sum of the attention weights of each time step is 1 for any spatial position, and the time attention weight is obtained. multiplying the temporal attention weight with the non-stationary feature sequence element-wise and summing along the time dimension to obtain an attention-weighted non-stationary feature and inputting the attention-weighted non-stationary feature to the MIM-S module after fusion with the non-stationary feature of the current time step to construct an approximately stationary representation and update the long-term memory state.

6. The method for material microstructure prediction and characterization based on improved space-time network according to claim 5, characterized in that, The attention weighted non-stationary features The fusion manner of the non-stationary features at the current time step is: to the current time step normalized and multiplied by a learnable scaling factor, and added to the non-stationary features of the current time step in a residual manner to obtain enhanced non-stationary features The enhanced non-stationary features have both non-stationary difference information and time attention selection information, and are used as the input of the MIM-S module.​ 7. The method for material microstructure prediction and characterization based on improved space-time network according to claim 6, characterized in that, The improved space-time network comprises a bottommost ST-LSTM module and a plurality of DT-MIM modules which are sequentially connected in an increasing depth manner; each DT-MIM module is configured with an independent N-TA-S module to model multi-scale time attention of non-stationary information at different feature levels, and the expression of the processing process of the lth layer DT-MIM module is as follows: where t and l represent time step and network level, respectively; are input gate, modulation gate, forget gate and output gate, respectively; denotes the time weight learned by the temporal attention mechanism; W and b denote the weight matrix and bias vector corresponding to different gates, respectively; σ, * and denote Sigmoid activation function, convolution operator and Hadamard product, respectively; denotes residual connection; MIM-N(∙) denotes the output of the MIM-N module; and MIM-S(∙) denotes the output of the MIM-S module.

Images