A rock crack propagation time series prediction method and system based on fusion of gan and lstm

Abstract

The application provides a rock crack propagation time sequence prediction method and system based on GAN and LSTM fusion, relates to the technical field of image processing, and comprises the following steps: obtaining a plurality of groups of preprocessed crack propagation image sequence samples; each group of crack propagation image sequence samples comprises a plurality of standard time sequence samples; a generator in an initial prediction model is used to extract time-dependent features and global spatial features from the crack propagation image sequence samples, obtain fused space-time features, input the fused space-time features into a residual attention module for feature fusion and feature weight adjustment, and generate an initial prediction image; the generator comprises a bidirectional LSTM module and a Transformer encoder; the initial prediction model is alternately optimized by using the initial prediction image to optimize a discriminator and the generator in the initial prediction model until a loss function corresponding to the discriminator and the generator converges or a set training number of rounds is reached, so that a target prediction model is obtained; and a to-be-processed image is input into the target prediction model to obtain a target crack propagation prediction image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a rock crack propagation time sequence prediction method and system based on GAN and LSTM fusion. BACKGROUND

[0002] The rock crack propagation process is an important instability and failure mechanism in geotechnical engineering, which is influenced by stress disturbance, structural weak plane, material heterogeneity and other factors, and has obvious nonlinearity and time correlation. In the fields of mine exploitation, tunneling, stability evaluation of hydraulic structures, accurate prediction of crack time evolution is of great significance for engineering safety evaluation and disaster warning.

[0003] Current rock crack propagation prediction often relies on numerical simulation methods, especially the finite difference-discrete element coupling method FDEM. Although such methods can accurately simulate the crack initiation, propagation and bifurcation process, they are time-consuming and resource-consuming, making it difficult to meet the actual needs of multi-working condition, rapid response or large-scale parallel prediction. Traditional time sequence models, such as LSTM, have the problem of insufficient information capture when dealing with highly nonlinear and spatiotemporal coupled behaviors, especially in the case of large sequence span or time sequence anomalies. The classic generative adversarial network GAN lacks an effective time sequence modeling mechanism, making it difficult to ensure the temporal consistency and physical rationality of the generated images. SUMMARY

[0004] Therefore, the present application provides a rock crack propagation time sequence prediction method and system based on GAN and LSTM fusion.

[0005] The technical scheme of the present application is as follows: The present application provides a rock crack propagation time sequence prediction method based on GAN and LSTM fusion in the first aspect, comprising:

[0006] Obtaining a plurality of pre-processed crack propagation image sequence samples; each of the crack propagation image sequence samples comprises a plurality of standard time sequence samples;

[0007] Extracting time-dependent features and global spatial features from the crack propagation image sequence samples using a generator in an initial prediction model to obtain fused spatiotemporal features, and inputting the fused spatiotemporal features into a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image; the generator comprises a bidirectional LSTM module and a Transformer encoder;

[0008] Alternately optimizing the discriminator and the generator in the initial prediction model using the initial prediction image until the loss functions corresponding to the discriminator and the generator converge or reach a set number of training rounds to obtain a target prediction model;

[0009] inputting the image to be processed into the target prediction model to obtain a target crack propagation prediction image.

[0010] Based on the above technical solutions, preferably, the obtaining of the plurality of groups of pre-processed crack propagation image sequence samples comprises:

[0011] numerical simulation is performed on the rock model containing cracks under two-dimensional stretching conditions by using the finite-discrete element method to generate a plurality of groups of crack propagation image sequences; each group of sequences comprises a plurality of continuous time sequence images;

[0012] The crack propagation image sequence is subjected to region cropping, grayscale operation, normalization mapping and binarization processing to obtain crack propagation image sequence samples.

[0013] Based on the above technical solutions, preferably, the bidirectional LSTM comprises a forward LSTM and a backward LSTM; the generator in the initial prediction model extracts time-dependent features and global spatial features from the crack propagation image sequence samples to obtain fused spatio-temporal features, comprising:

[0014] Each frame of image of the crack propagation image sequence sample is mapped into a fixed-dimension low-dimensional feature vector by a convolutional encoder;

[0015] The low-dimensional feature vectors are processed in parallel by the forward LSTM and the backward LSTM to respectively generate a forward hidden state sequence and a backward hidden state sequence;

[0016] The forward hidden state sequence and the backward hidden state sequence are spliced at each time step to obtain the time-dependent features.

[0017] Based on the above technical solutions, preferably, the Transformer encoder comprises a multi-head attention mechanism and a feedforward neural network stacked into shape; the generator in the initial prediction model extracts time-dependent features and global spatial features from the crack propagation image sequence samples to obtain fused spatio-temporal features, comprising:

[0018] Dynamic position encoding is added to the time-dependent features to obtain a time sequence dependent relationship;

[0019] The time sequence dependent relationship of different time scales is calculated in parallel by the multi-head attention mechanism and the feedforward neural network to obtain the global spatial features;

[0020] The time-dependent features and the global spatial features are aligned in dimension by a linear projection layer and then fused to obtain fused spatio-temporal features.

[0021] On the basis of the above technical scheme, preferably, the fusion spatio-temporal features are input into a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image, and the method further comprises:

[0022] The residual attention module is used to perform spatial attention weighting and channel attention weighting on the fusion spatio-temporal features respectively to obtain weighted spatio-temporal features.

[0023] The weighted spatio-temporal features are deconvoluted or upsampled to generate the initial prediction image.

[0024] On the basis of the above technical scheme, preferably, the initial prediction model is alternately optimized by using the initial prediction image to optimize the discriminator and the generator in the initial prediction model until the loss functions corresponding to the discriminator and the generator converge or reach a set number of training rounds to obtain a target prediction model, and the method comprises:

[0025] In the discriminator optimization stage, the parameters of the generator are kept constant, the real image sequence and the initial prediction image are input into the discriminator, and the discriminator parameters are updated according to the gradient descent method to minimize the first loss function corresponding to the discriminator.

[0026] In the generator optimization stage, the parameters of the discriminator are kept constant, the fusion spatio-temporal features and the initial prediction image are input into the generator, and the generator parameters are updated according to the gradient descent method to minimize the second loss function corresponding to the generator.

[0027] The alternating optimization step is repeatedly performed until the first loss function and the second loss function are both less than a preset threshold or a set number of training rounds is reached.

[0028] On the basis of the above technical scheme, preferably, the initial prediction model is alternately optimized by using the initial prediction image to optimize the discriminator and the generator in the initial prediction model until the loss functions corresponding to the discriminator and the generator converge or reach a set number of training rounds to obtain a target prediction model, and the method comprises:

[0029] The physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function in the training process of the generator are obtained.

[0030] The physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function are weighted to obtain a fusion loss function.

[0031] The generator is optimized by using the fusion loss function.

[0032] Further preferably, the second aspect of the present application provides a rock crack propagation time series prediction system based on GAN and LSTM fusion, comprising: a sample acquisition module, a feature fusion module, a parameter optimization module and a crack propagation prediction module; wherein,

[0033] The sample acquisition module is configured to acquire a plurality of groups of preprocessed crack propagation image sequence samples; each group of crack propagation image sequence samples comprises a plurality of standard time series samples;

[0034] The feature fusion module is configured to use a generator in the initial prediction model to extract time-dependent features and global spatial features from the crack propagation image sequence samples to obtain fused spatio-temporal features, and input the fused spatio-temporal features into a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image; the generator comprises a bidirectional LSTM module and a Transformer encoder;

[0035] The parameter optimization module is configured to use the initial prediction image to alternately optimize a discriminator and the generator in the initial prediction model until the loss functions corresponding to the discriminator and the generator converge or reach a set number of training rounds to obtain a target prediction model;

[0036] The crack propagation prediction module is configured to input a to-be-processed image into the target prediction model to obtain a target crack propagation prediction image.

[0037] Further preferably, the third aspect of the present application provides an electronic device comprising a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the rock crack propagation time series prediction method based on GAN and LSTM fusion of the first aspect.

[0038] Further preferably, the fourth aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the rock crack propagation time series prediction method based on GAN and LSTM fusion of the first aspect.

[0039] The rock crack propagation time series prediction method and system based on GAN and LSTM fusion of the present application have the following beneficial effects relative to the prior art:

[0040] 1. By constructing a deep prediction model fused with GAN and bidirectional LSTM, the bidirectional LSTM network is good at capturing long-term dependence and time evolution law in image sequence, which ensures the time sequence consistency and physical continuity of the predicted image; while the GAN structure has strong image fitting and detail generation ability, which effectively improves the authenticity and spatial expression accuracy of the crack morphology. The cooperation of the two makes the model not only have coherent time dynamic expression, but also keep the image details clear and accurate when generating crack images in highly nonlinear and complex evolution process, which significantly improves the overall performance of rock crack propagation prediction.

[0041] 2. By modeling the bidirectional time-dependent features of crack images, combining residual attention mechanism to focus on key expansion areas, and fusing Transformer encoder to capture global dynamic relationships, and cooperating with multi-scale discriminators to enhance the joint discriminant ability of image spatial structure and crack local details, high-precision modeling and time sequence prediction of complex crack propagation paths are realized. On this basis, a physical guidance loss function is introduced to ensure that the crack prediction results follow the continuous expansion law, effectively avoiding non-physical sudden growth.

[0042] 3. By introducing a dynamic position encoding module related to the input data, the time-dependent features can adapt to dynamic patterns at different time scales, overcoming the defect that fixed position encoding cannot represent non-uniform time sequence relationships. Through multi-head attention mechanism, time-dependent relationships at different time scales are extracted in parallel, which significantly improves the modeling efficiency of the model for complex time sequence patterns while keeping the computational complexity linear. The linear projection layer is used to realize the dimension alignment and semantic alignment of spatio-temporal features, solving the performance degradation problem caused by feature space misalignment in traditional fusion methods, and significantly improving the discriminability of the fused features. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief descriptions will be given below to the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0044] Figure 1 A flowchart of a rock crack propagation time sequence prediction method based on fusion of GAN and LSTM provided by an embodiment of the present application is shown in the figure.

[0045] Figure 2 A crack image after gray processing provided by an embodiment of the present application is shown in the figure.

[0046] Figure 3A schematic diagram of a crack propagation constraint mechanism provided for an embodiment of the present application;

[0047] Figure 4 A visual comparison result of a predicted image (right) and a real image (left) of a verification sample for a model provided for an embodiment of the present application;

[0048] Figure 5 A structural schematic diagram of a rock crack propagation time series prediction system based on GAN and LSTM fusion provided for an embodiment of the present application;

[0049] Figure 6 A structural schematic diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0051] In some embodiments, as shown in Figure 1 , Figure 1 A flowchart of a rock crack propagation time series prediction method based on GAN and LSTM fusion provided for an embodiment of the present application; the rock crack propagation time series prediction method based on GAN and LSTM fusion provided by the present application comprises:

[0052] S110, a plurality of groups of preprocessed crack propagation image sequence samples are obtained; each group of crack propagation image sequence samples comprises a plurality of standard time sequence samples.

[0053] Here, a series of crack propagation images can be obtained by using a rock simulation model to simulate cracks, and a plurality of standard time sequence samples can be obtained according to the crack propagation time sequence. The crack propagation image sequence samples are preprocessed, which can improve the quality of the images and provide high-quality data sources for the subsequent training process.

[0054] In some embodiments, S110, a plurality of groups of preprocessed crack propagation image sequence samples are obtained, comprising:

[0055] A finite-discrete element method is used to numerically simulate a rock model containing cracks under two-dimensional stretching conditions to generate a plurality of groups of crack propagation image sequences; each group of sequences contains a plurality of continuous time sequence images;

[0056] The crack propagation image sequence is subjected to region clipping, grayscale operation, normalization mapping and binarization processing to obtain a crack propagation image sequence sample.

[0057] In this embodiment, the finite-discrete element method FDEM is used to simulate the two-dimensional tensile condition of the rock model containing cracks. The simulation records the whole process from crack initiation to complete penetration, generating a series of crack propagation images as the original data basis for subsequent model training and verification. The crack image is grayed and converted to a grayscale image. In the FDEM simulation, the rock mass is set as a two-dimensional rectangular structure, multiple micro-cracks are initially precast, and the length of each crack is set to be randomly distributed in the model. The upper and lower boundaries are subjected to vertical tensile load, and the left and right boundaries are fixed. The crack propagates under the action of tensile stress. During the calculation process, a frame of image is derived every 1000 steps, and the whole process of crack propagation is recorded. The crack propagation image sequence derived by FDEM is standardized. First, the image is regionally cropped to remove redundant boundary information and only keep the core area containing crack evolution characteristics. Then, the grayscale operation is performed to convert the original color image to a single channel form, simplify the calculation complexity and highlight the structural features. Then, the normalization mapping is completed, and the pixel value is compressed to the range of [0, 1]. In order to further enhance the model recognition ability, the threshold judgment method is used for binaryzation of the image, only keeping the crack area (pixel value 0) and background area (pixel value 1), forming a clear black and white structure diagram. All processed images are saved in.npy format for subsequent batch training and calling.

[0058] In one example, image standardization can perform pixel value normalization processing using the following formula:

[0059] ;

[0060] wherein, is the original image pixel value, is the image pixel mean, is the pixel standard deviation, is the normalized output. The binaryzation processing adopts a fixed threshold of 0.5, that is, greater than the value is set to 1 (background) and less than 0 (crack), so as to obtain a clear crack propagation diagram.

[0061] In an optional embodiment, please refer to Figure 2 , Figure 2 is a crack image after gray processing provided by the embodiment of the present application; wherein the crack is represented by dark pixels, and the background is a bright area, the image structure is clear, and the boundary is distinct, which is suitable for automatic prediction modeling. First, a two-dimensional finite difference-discrete element method FDEM is used to simulate the tensile rock mass model containing 10 initial precast cracks. The model is set as a rectangular domain with a size of The upper and lower boundaries apply vertical tensile loads, and the left and right boundaries are fixed. Each crack is about 4 mm long and is evenly distributed in the transverse direction. The simulation parameters are kept consistent. During the simulation, an image is derived every 1000 steps to record the development of the crack from initiation to penetration. A total of 40 crack propagation image sequences are obtained, each covering multiple time frames of the entire crack process. These image sequences serve as the basis for subsequent modeling. The first 36 sequences are used for model training, and the last 4 are used for verification. The original images are standardized to meet the input requirements of the neural network model. First, the images are cropped to retain a 216x216 pixel window containing the main area of crack propagation. Then, the color images are converted to single-channel grayscale images, and the pixel values are normalized to the [0, 1] range to eliminate scale interference and accelerate model convergence. After further binarization, the crack region is mapped to 0 and the background region is mapped to 1, resulting in clear black and white crack images with distinct boundaries. All images are saved in.npy format for easy loading and batch training. According to the natural order of crack propagation, crack propagation image sequence samples are constructed. Each image sequence contains 5 consecutive historical images as input and the 6th image as the true image (prediction target). The multiple image sequences obtained from simulation are divided into training and validation sets according to their group numbers, and are organized into crack propagation image sequence samples in chronological order to improve the model's dynamic learning ability for crack propagation.

[0062] In S120, the generator in the initial prediction model is used to extract time-dependent features and global spatial features from the crack propagation image sequence samples, obtain fused spatio-temporal features, and input the fused spatio-temporal features into a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image. The generator includes a bidirectional LSTM module and a Transformer encoder.

[0063] In this embodiment, the bidirectional LSTM module (BiLSTM) is used to extract time-dependent features from the crack propagation image sequence samples. This module can simultaneously learn the forward and reverse dynamics of the time series, effectively enhancing the model's ability to model crack evolution trends. The BiLSTM output features are further processed by the residual attention module. The residual attention module uses a cross-layer jump connection mechanism to fuse low-level spatial structures and high-level semantic information, while introducing attention weights to adaptively adjust the feature focus area, thereby enhancing the expression ability of crack region features and suppressing redundant background interference. The enhanced features are then input into a multi-layer perceptron network (MLP) to generate an initial prediction image.

[0064] In some embodiments, the bidirectional LSTM includes a forward LSTM and a backward LSTM; S120 extracts time-dependent features and global spatial features from the crack propagation image sequence sample by using a generator in the initial prediction model to obtain fused spatio-temporal features, including:

[0065] Each frame of image of the crack propagation image sequence sample is mapped into a fixed-dimension low-dimensional feature vector by a convolutional encoder;

[0066] The low-dimensional feature vector is processed in parallel by the forward LSTM and the backward LSTM to respectively generate a forward hidden state sequence and a backward hidden state sequence;

[0067] The forward hidden state sequence and the backward hidden state are spliced at each time step to obtain time-dependent features.

[0068] In the present embodiment, at each time , the BiLSTM updates the state of the historical sequence and the future sequence respectively, and the core is composed of the following standard LSTM unit:

[0069] ;

[0070] wherein, represents the image feature of the current time step; , are the hidden state and the memory unit of the previous step respectively; , , are the forget gate, the input gate and the output gate respectively; is the candidate memory; , , , are the weight matrices corresponding to each gate; , , , are the bias terms; is a sigmoid function, is a hyperbolic tangent function.

[0071] In the bidirectional structure, the forward hidden state and the backward hidden state are calculated respectively, and finally the joint feature vector is obtained by splicing in the time dimension:

[0072] ;

[0073] wherein, is the total hidden state vector of the bidirectional LSTM at time step t; is the hidden state of the forward LSTM at time step t, which processes the input sequence from front to back in time sequence; is the hidden state of the backward LSTM at time step t, and the input sequence is processed from back to front in time series.

[0074] The above bidirectional structure enables the model to not only capture historical features in the crack propagation process, but also perceive the symmetry and correlation between local temporal context, further improving the structural consistency and physical rationality of the predicted image.

[0075] In some embodiments, the Transformer encoder includes a multi-head attention mechanism and a feedforward neural network stacked together; S120 extracts time-dependent features and global spatial features from the crack propagation image sequence samples using the generator in the initial prediction model, obtaining fused spatio-temporal features, including:

[0076] Adding dynamic position encoding to the time-dependent features obtains the time-dependent relationship;

[0077] Parallel computing of the multi-head attention mechanism and the feedforward neural network obtains the global spatial features;

[0078] The time-dependent features and the global spatial features are aligned in dimension through a linear projection layer and then fused to obtain the fused spatio-temporal features.

[0079] In this embodiment, the Transformer encoder extracts long-distance dependent features between image frames through a self-attention mechanism. Each image frame is mapped to a fixed-dimensional low-dimensional feature vector through a convolutional encoder, and position encoding is introduced to retain the time series information, and then a Transformer encoder is used for global modeling. The Transformer encoder is stacked by a multi-head attention mechanism and a feedforward neural network, which can process time series data in parallel and capture multi-scale dynamic evolution relationships between different attention heads. The final output sequence feature vector is fused with the hidden state of the upstream LSTM module through linear mapping and then input into the image reconstruction module to generate the next time prediction image. Compared with the traditional unidirectional LSTM, the Transformer can effectively model the long-range dependency between image frames and the spatial response consistency, improve the coherence and authenticity of the predicted image, and thus improve the modeling accuracy and stability of the complex crack propagation path.

[0080] The single-head attention mechanism of the Transformer encoder is:

[0081] ;

[0082] wherein, is the Query query matrix; is the Key key matrix; is the Value value matrix; is the dimension of the key vector, used for scaling, to prevent gradient explosion.

[0083] The multi-head attention mechanism of the Transformer encoder is:

[0084] ;

[0085] wherein, is the output of the i-th attention head; are the query, key, value mapping matrices of the i-th head, respectively; is the output mapping matrix.

[0086] The feed-forward neural network of the Transformer encoder is:

[0087] ;

[0088] wherein, is the intermediate vector from the attention module; are the weight matrices of the two linear transformations; are the corresponding bias terms.

[0089] In some embodiments, S120, the spatio-temporal fused features are input into a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image, further comprising:

[0090] The residual attention module is used to perform spatial attention weighting and channel attention weighting on the spatio-temporal fused features respectively to obtain weighted spatio-temporal features;

[0091] The weighted spatio-temporal features are deconvoluted or upsampled to generate an initial prediction image.

[0092] The residual attention module as a whole can be regarded as a sub-module that can be nested at any position of a neural network, and generally includes the following three parts: a main branch, an attention branch, and a fusion output branch.

[0093] The main branch is used to capture the original feature representation, and is generally composed of several convolutional layers, represented as:

[0094] ;

[0095] wherein, is the input feature map; is the main output; is a rectified linear unit activation function; is a first layer convolution operation.

[0096] The attention branch is used to learn an element-level attention weight map, represented as:

[0097] ;

[0098] wherein, denotes a sigmoid activation function; the output of denotes the importance of each pixel point; is the 3rd layer convolution operation, denotes the 4th layer convolution.

[0099] The fusion output branch applies the attention map as a gating factor on the backbone output and adds a residual connection:

[0100] ;

[0101] wherein, is the final output of the residual attention module; is the attention map generated by the attention branch; is the output of the backbone branch; The residual idea of attention enhancement is emphasized, which is equivalent to introducing the information of the biased region based on the identity mapping.

[0102] S130, the initial prediction image is used to alternately optimize the discriminator and the generator in the initial prediction model until the loss function corresponding to the discriminator and the generator converges or reaches a set training number, and a target prediction model is obtained.

[0103] In some embodiments, S130, the initial prediction image is used to alternately optimize the discriminator and the generator in the initial prediction model until the loss function corresponding to the discriminator and the generator converges or reaches a set training number, and a target prediction model is obtained, comprising:

[0104] In the discriminator optimization phase, the parameters of the generator are kept constant, the real image sequence and the initial prediction image are input into the discriminator, and the discriminator parameters are updated according to the gradient descent method to minimize the first loss function corresponding to the discriminator;

[0105] In the generator optimization phase, the parameters of the discriminator are kept constant, the spatio-temporal feature fusion and the initial prediction image are input into the generator, and the generator parameters are updated according to the gradient descent method to minimize the second loss function corresponding to the generator;

[0106] The steps of alternating optimization are repeatedly performed until the first loss function and the second loss function are both less than a preset threshold, or a set training number is reached.

[0107] In this embodiment, the model training process follows the generative adversarial mechanism: the generator and the discriminator are alternately optimized, the generator aims to generate images with realistic and physical consistency, and the discriminator is used to distinguish between fake images and real images.

[0108] In this embodiment, the model parameters are updated by an Adam optimizer, and the update formula is:

[0109]

[0110] wherein, are the bias correction values of the gradient first and second moments respectively; is the average of the current gradient; is the parameter for controlling the length of memory; is the square average of the current gradient; is the current training round number; is the learning rate, is a constant to prevent division by zero, and are the model parameters after and before iteration respectively. In each iteration, the discriminator parameters are first updated to improve the identification ability of real images, and then the generator parameters are updated to make the output more consistent in time sequence and physically reasonable.

[0111] In some embodiments, S130, the initial prediction image is used to alternately optimize the discriminators and generators in the initial prediction model until the loss functions corresponding to the discriminators and generators converge or reach the set training round number, to obtain the target prediction model, comprising:

[0112] Obtaining the physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function in the training process of the generator;

[0113] Weighting the physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function to obtain a fusion loss function;

[0114] Optimizing the generator using the fusion loss function.

[0115] The training loss fuses three indicators: the physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function, which respectively measure the extension rationality, the pixel accuracy and the image fidelity. The final goal is to minimize the above weighted loss at the same time.

[0116] The physical constraint loss function is defined as follows:

[0117]

[0118] wherein, is the physical guidance constraint loss function; is the total number of all predicted pixels; is the index of the current pixel; is the i-th pixel position in the current frame prediction image; is the prediction value of the i-th pixel in the previous frame.​​ a local neighborhood centered on the previous frame prediction value; is an indicator function that outputs 1 if the condition in the brackets is true, and 0 otherwise; represents that if the current predicted crack pixel point does not appear in the neighborhood of the previous frame, it is considered to violate the physical expansion rule.

[0119] The training target of the generator is to deceive the discriminator to believe that the fake image is a real image, and the image reconstruction loss function of the generator is:

[0120] ;

[0121] The adversarial mean square error loss function is used to measure the pixel difference between the generated image and the real image :

[0122] ;

[0123] wherein, represents the total number of samples; represents the i-th real label value; represents the i-th predicted value; represents the square error between the real value and the predicted value.

[0124] The final generator total loss function is:

[0125] ;

[0126] wherein, are the weighting factors of the image reconstruction loss function, the adversarial mean square error loss function and the physical constraint loss function, respectively.

[0127] S140, input the image to be processed into the target prediction model to obtain a target crack propagation prediction image.

[0128] In this embodiment, the time sequence consistency constraint can be performed on the image to be processed first, and the crack is limited to propagate only within the surrounding range of the crack region of the previous frame, thereby improving the prediction rationality. On this basis, the crack prediction is completed, and the target crack propagation prediction image and the model evaluation result are output, and the crack propagation process is displayed in the form of an image. The specific steps of the time sequence consistency constraint can be: performing an inverse normalization operation on the image to be processed to restore the Tanh output to the range of 0-1, and then performing a binaryzation processing to further strengthen the boundary between the crack (pixel value 0) and the background (pixel value 1). On this basis, the distribution of the crack pixel points at the previous time is tracked, and the crack is only allowed to propagate within a set radius range around it, forming a restricted growth area, thereby avoiding the sudden growth of the crack in an area without a physical basis.

[0129] In one example, refer to Figure 3 , Figure 3 a schematic diagram of the crack propagation constraint mechanism provided by the embodiment of the present application. The red boundary area in the figure represents the allowed expansion area within the 6 pixel range of the crack edge of the previous frame, and only the new crack appearing in this area is accepted to enhance the physical rationality and continuity of the predicted image.

[0130] In another example, refer to Figure 4 , Figure 4 a visual comparison result of the predicted image (right) and the real image (left) of the validation sample for the model provided by the embodiment of the present application. As can be seen from the figure, the two maintain good consistency in crack morphology, extension trend and spatial position, verifying the high-precision prediction ability of the model.

[0131] In some embodiments, refer to Figure 5 , Figure 5 a structural schematic diagram of a rock crack propagation time series prediction system based on the fusion of GAN and LSTM provided by the embodiment of the present application. The present application provides a rock crack propagation time series prediction system 500 based on the fusion of GAN and LSTM, comprising: a sample acquisition module 510, a feature fusion module 520, a parameter optimization module 530 and a crack propagation prediction module 540; wherein,

[0132] The sample acquisition module 510 is configured to acquire a plurality of groups of preprocessed crack propagation image sequence samples; each group of crack propagation image sequence samples comprises a plurality of standard time series samples;

[0133] The feature fusion module 520 is configured to extract time-dependent features and global spatial features from the crack propagation image sequence samples using a generator in the initial prediction model to obtain fused spatio-temporal features, and input the fused spatio-temporal features to a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image; the generator comprises a bidirectional LSTM module and a Transformer encoder;

[0134] The parameter optimization module 530 is configured to alternately optimize the discriminator and the generator in the initial prediction model using the initial prediction image until the loss functions corresponding to the discriminator and the generator converge or reach a set training number of rounds to obtain a target prediction model;

[0135] The crack propagation prediction module 540 is configured to input the image to be processed into the target prediction model to obtain a target crack propagation prediction image.

[0136] In some embodiments, the sample acquisition module 510 is specifically configured to:

[0137] A finite-discrete element method is used to simulate a two-dimensional tensile crack rock model to generate a plurality of crack propagation image sequences, each of which includes a plurality of time-series images;

[0138] The crack propagation image sequences are subjected to region cropping, grayscale operation, normalization mapping and binarization processing to obtain crack propagation image sequence samples.

[0139] In some embodiments, the bidirectional LSTM includes a forward LSTM and a backward LSTM; the feature fusion module 520 is specifically configured to:

[0140] Each frame of image of the crack propagation image sequence sample is mapped to a fixed-dimensional low-dimensional feature vector by a convolutional encoder;

[0141] The low-dimensional feature vectors are processed in parallel by the forward LSTM and the backward LSTM to generate a forward hidden state sequence and a backward hidden state sequence, respectively;

[0142] The forward hidden state sequence and the backward hidden state sequence are spliced at each time step to obtain a time-dependent feature.

[0143] In some embodiments, the Transformer encoder includes a multi-head attention mechanism and a feedforward neural network stacked into shape; the feature fusion module 520 is specifically configured to:

[0144] Dynamic position encoding is added to the time-dependent feature to obtain a time-dependent relationship;

[0145] The multi-head attention mechanism and the feedforward neural network are used to calculate the time-dependent relationship of different time scales in parallel to obtain a global spatial feature;

[0146] The time-dependent feature and the global spatial feature are aligned in dimension by a linear projection layer and then fused to obtain a fused spatio-temporal feature.

[0147] In some embodiments, the feature fusion module 520 is specifically configured to:

[0148] The residual attention module is used to perform spatial attention weighting and channel attention weighting on the fused spatio-temporal feature to obtain a weighted spatio-temporal feature;

[0149] The weighted spatio-temporal feature is subjected to deconvolution or upsampling operation to generate an initial prediction image.

[0150] In some embodiments, the parameter optimization module 530 is specifically configured to:

[0151] In the discriminator optimization stage, the parameters of the generator are kept constant, the real image sequence and the initial prediction image are input into the discriminator, the parameters of the discriminator are updated according to the gradient descent method, and the first loss function corresponding to the discriminator is minimized;

[0152] In the generator optimization stage, the parameters of the discriminator are kept constant, the spatio-temporal feature fusion and the initial prediction image are input into the generator, the parameters of the generator are updated according to the gradient descent method, and the second loss function corresponding to the generator is minimized.

[0153] The alternating optimization steps are repeatedly performed until the first loss function and the second loss function are both less than a preset threshold, or a set number of training rounds is reached.

[0154] In some embodiments, the parameter optimization module 530 is specifically configured to:

[0155] Obtain the physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function in the generator training process;

[0156] The physical constraint loss function, the image reconstruction loss function and the adversarial mean square error loss function are weighted to obtain a fusion loss function;

[0157] The generator is optimized using the fusion loss function.

[0158] It should be noted that the rock crack propagation time series prediction system based on GAN and LSTM fusion provided in the embodiments of the present application and the rock crack propagation time series prediction method based on GAN and LSTM fusion provided in the embodiments of the present application are based on the same application concept, so the specific implementation of this embodiment can be referred to the implementation of the aforementioned rock crack propagation time series prediction method based on GAN and LSTM fusion, and the repeated parts will not be described again.

[0159] In some embodiments, please refer to Figure 6 , Figure 6 A structural schematic diagram of an electronic device is provided for the embodiments of the present application. The electronic device 600 provided by the embodiments of the present application includes a processor 610 and a memory 620; the memory 620 stores a computer program, wherein the computer program implements the aforementioned rock crack propagation time series prediction method based on GAN and LSTM fusion when executed by the processor.

[0160] Specifically, the processor 610 may, for example, include a general-purpose microprocessor, an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 610 can also include on-board memory for cache use. The processor 610 can be a single processing unit or a plurality of processing units for performing different actions of the method process according to the embodiments of the present application.

[0161] Memory 620, for example, can be any media able to contain, store, communicate, propagate or transport the program code. For example, memory 620 can include, but is not limited to, storage and / or memory such as random-access memory (RAM), read-only memory (ROM), floppy disks, hard disks, optical disks, or tape. Common forms of computer-readable media include, but are not limited to, portable computer diskette, hard disk, RAM, ROM, FLASH, and / or the Internet or other communication networks or media. The RAM allows information, such as computer program code, to be written to or executed.

[0162] The application also provides a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the above-described rock crack propagation timing prediction method based on fusion of GAN and LSTM. The computer readable medium can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The computer readable medium carries one or more programs which, when executed, implement the method according to the embodiments of the application.

[0163] According to an embodiment of the present application, the computer readable medium can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer readable storage medium can be any tangible medium that contains or stores a program code that can be used by or in connection with an instruction execution system, apparatus or device. In this application, the computer readable signal medium can include a data signal that propagates in a baseband or as part of a carrier wave. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that can send, propagate or transport a program code for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted using any suitable medium, including, but not limited to, wireless, wired, optical, electromagnetic, infrared, or any suitable combination thereof.

[0164] It will be appreciated by those skilled in the art that features of the various embodiments and / or claims of the present application can be combined and / or interchanged, even though such combinations and / or interchanges are not expressly disclosed in the present application. Specifically, features of the various embodiments and / or claims of the present application can be combined and / or interchanged, even though such combinations and / or interchanges are not expressly disclosed in the present application. All of these combinations and / or interchanges are within the scope of the present application. Accordingly, the scope of the present application should be determined by the appended claims and equivalents thereof, rather than by the description of the embodiments of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A method for predicting the temporal propagation of rock cracks based on the fusion of GAN and LSTM, characterized in that, include: Obtain multiple sets of preprocessed crack propagation image sequence samples; Each set of crack propagation image sequence samples includes multiple standard time-series samples; The generator within the initial prediction model extracts temporal-dependent features and global spatial features from the crack propagation image sequence samples to obtain fused spatiotemporal features. These fused spatiotemporal features are then input into a residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image. The generator includes a bidirectional LSTM module and a Transformer encoder. The bidirectional LSTM includes a forward LSTM and a backward LSTM. The Transformer encoder includes a stacked multi-head attention mechanism and a feedforward neural network. The step of extracting temporal-dependent features and global spatial features from the crack propagation image sequence samples using the generator within the initial prediction model to obtain fused spatiotemporal features includes: The crack propagation image sequence samples are mapped to a fixed-dimensional low-dimensional feature vector using a convolutional encoder. The low-dimensional feature vector is processed in parallel by the forward LSTM and the backward LSTM to generate the forward hidden state sequence and the backward hidden state sequence, respectively. The forward hidden state sequence and the backward hidden state are concatenated at each time step to obtain the time-dependent feature; Add dynamic position encoding to the time-dependent features to obtain the temporal dependency relationship; Global spatial features are obtained by using a multi-head attention mechanism and the feedforward neural network to compute temporal dependencies at different time scales in parallel. The temporal-dependent features and the global spatial features are aligned in dimensions through a linear projection layer and then fused to obtain fused spatiotemporal features. The discriminator and generator in the initial prediction model are alternately optimized using the initial prediction image until the loss functions corresponding to the discriminator and generator converge or reach a set number of training rounds, thereby obtaining the target prediction model; The image to be processed is input into the target prediction model to obtain the target crack propagation prediction image.

2. The rock crack propagation time series prediction method based on GAN and LSTM fusion as described in claim 1, characterized in that, The acquisition of multiple sets of preprocessed crack propagation image sequence samples includes: The finite-discrete element method was used to numerically simulate a cracked rock model under two-dimensional tensile conditions, generating multiple sets of crack propagation image sequences; each set of sequences contains multiple consecutive time-series images. The crack propagation image sequence is subjected to region cropping, grayscale conversion, normalization mapping, and binarization to obtain crack propagation image sequence samples.

3. The rock crack propagation time series prediction method based on GAN and LSTM fusion as described in claim 1, characterized in that, The step of inputting the fused spatiotemporal features into the residual attention module for feature fusion and feature weight adjustment to generate an initial prediction image further includes: The residual attention module is used to perform spatial attention weighting and channel attention weighting on the fused spatiotemporal features to obtain weighted spatiotemporal features. The weighted spatiotemporal features are deconvolved or upsampled to generate an initial prediction image.

4. The rock crack propagation time series prediction method based on GAN and LSTM fusion as described in claim 1, characterized in that, The step of alternately optimizing the discriminator and generator within the initial prediction model using the initial prediction image until the loss functions corresponding to the discriminator and generator converge or reach a set number of training rounds to obtain the target prediction model includes: During the discriminator optimization phase, the parameters of the generator are kept constant, the real image sequence and the initial predicted image are input into the discriminator, and the discriminator parameters are updated according to the gradient descent method to minimize the first loss function corresponding to the discriminator. During the generator optimization phase, the parameters of the discriminator are kept constant, the fused spatiotemporal features and the initial predicted image are input into the generator, and the generator parameters are updated according to the gradient descent method to minimize the second loss function corresponding to the generator. Repeat the alternating optimization steps until both the first loss function and the second loss function are less than a preset threshold, or until a set number of training rounds are reached.

5. The rock crack propagation time series prediction method based on GAN and LSTM fusion as described in claim 1, characterized in that, The step of alternately optimizing the discriminator and generator within the initial prediction model using the initial prediction image until the loss functions corresponding to the discriminator and generator converge or reach a set number of training rounds to obtain the target prediction model includes: Obtain the physical constraint loss function, image reconstruction loss function, and adversarial mean square error loss function during the generator training process; The physical constraint loss function, the image reconstruction loss function, and the adversarial mean square error loss function are weighted to obtain the fusion loss function; The generator is optimized using the fusion loss function.

6. A time-series prediction system for rock crack propagation based on the fusion of GAN and LSTM, characterized in that, include: The system includes a sample acquisition module, a feature fusion module, a parameter optimization module, and a crack propagation prediction module; among which, The sample acquisition module is configured to acquire multiple sets of preprocessed crack propagation image sequence samples; each set of crack propagation image sequence samples includes multiple standard time-series samples; The feature fusion module is configured to extract time-dependent features and global spatial features from the crack propagation image sequence samples using the generator within the initial prediction model, obtaining fused spatiotemporal features, and inputting the fused spatiotemporal features into the residual attention module for feature fusion and feature weight adjustment to generate the initial prediction image; the generator includes a bidirectional LSTM module and a Transformer encoder; the bidirectional LSTM includes a forward LSTM and a backward LSTM; the Transformer encoder includes a stacked multi-head attention mechanism and a feedforward neural network; the feature fusion module is specifically configured to: use a convolutional encoder to extract time-dependent features and global spatial features from the crack propagation image sequence samples. Each frame of image is mapped to a low-dimensional feature vector of fixed dimensions; the low-dimensional feature vector is processed in parallel by the forward LSTM and the backward LSTM to generate a forward hidden state sequence and a backward hidden state sequence, respectively; the forward hidden state sequence and the backward hidden state sequence are concatenated at each time step to obtain the temporal dependency feature; dynamic position encoding is added to the temporal dependency feature to obtain the temporal dependency relationship; the temporal dependency relationship at different time scales is calculated in parallel by the multi-head attention mechanism and the feedforward neural network to obtain the global spatial feature; the temporal dependency feature and the global spatial feature are aligned in dimensions by a linear projection layer and then fused to obtain the fused spatiotemporal feature; The parameter optimization module is configured to use the initial prediction image to alternately optimize the discriminator and the generator in the initial prediction model until the loss functions corresponding to the discriminator and the generator converge or reach a set number of training rounds, thereby obtaining the target prediction model. The crack propagation prediction module is configured to input the image to be processed into the target prediction model to obtain the target crack propagation prediction image.

7. An electronic device comprising a processor and a memory; said memory having storage for computer programs, wherein, When the computer program is executed by the processor, it implements the rock crack propagation time-series prediction method based on the fusion of GAN and LSTM as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, It stores a computer program, wherein when the computer program is executed by a processor, it implements the rock crack propagation time series prediction method based on the fusion of GAN and LSTM as described in any one of claims 1 to 5.

Images