A dense sandstone image augmentation method based on deep learning

Abstract

The present invention relates to the technical field of oil and gas exploration and development, and in particular to a method for augmenting dense sandstone images based on deep learning. The method comprises the following steps: obtaining labeled dense sandstone images and performing preprocessing; constructing a dense sandstone image generation model based on a StyleGAN network; training the constructed dense sandstone image generation model based on the acquired dense sandstone images; during the training process, adjusting the augmentation strength based on the degree of overfitting, and dynamically adjusting the position for the next overfitting determination; and generating dense sandstone images using the trained dense sandstone image generation model. The present invention can generate high-quality dense sandstone images, improve data diversity, augment dense sandstone image samples, and provide support for intelligent dense sandstone image segmentation.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of oil and gas exploration and development, and in particular to a method, device, electronic device, and storage medium for enhancing dense sandstone images based on deep learning. Background Art

[0002] Unconventional oil and gas has become a key alternative for sustainable development in the global oil industry. Unlike conventional oil and gas, unconventional reservoirs, typically tight sandstones, possess complex pore structures and unique fluid migration and accumulation mechanisms, making large-scale exploration and development challenging. Accurate identification and segmentation of mineral particles and pores are essential for accurate characterization of reservoir pore structure and fluid migration and accumulation mechanisms.

[0003] In the past, dense sandstone image recognition was typically based on expert experience or machine learning, which was highly subjective, overly reliant on manual labor, and time-consuming. In recent years, with the rapid development of deep learning technology, intelligent image recognition methods based on large datasets have the advantages of less human-computer interaction, faster recognition speed, and more accurate results compared to traditional recognition methods. As a result, they are gradually being applied to dense sandstone image analysis tasks.

[0004] Image recognition and segmentation methods based on deep learning rely on large datasets for neural network training. However, due to the complex and high cost of producing thin sections of dense sandstone, the number of thin section samples is very limited, which to some extent limits the training effect and recognition accuracy of the model. Summary of the Invention

[0005] Based on the problem that it is generally difficult to obtain a sufficient number of dense sandstone images with existing technologies, the embodiments of the present invention provide a method, device, electronic device and storage medium for dense sandstone image augmentation based on deep learning, which can use deep learning technology to generate dense sandstone images and realize dense sandstone image data augmentation.

[0006] In a first aspect, an embodiment of the present invention provides a method for enhancing dense sandstone images based on deep learning, comprising:

[0007] Obtaining a labeled dense sandstone image and performing preprocessing; the label is used to indicate the type of the corresponding dense sandstone image;

[0008] A dense sandstone image generation model based on the StyleGAN network was constructed. This dense sandstone image generation model was modified based on the StyleGAN network, using only one latent code to control the key features of the generated image, reducing the number of times noise was added in the generation network, and adding self-attention layers in both the generation network and the discriminator network.

[0009] Based on the acquired dense sandstone image, the constructed dense sandstone image generation model is trained; during the training process, the augmentation strength is adjusted according to the degree of overfitting, and the position for determining whether overfitting is to be determined next time is dynamically adjusted;

[0010] The trained dense sandstone image generation model is used to generate a dense sandstone image.

[0011] Optionally, the preprocessing includes filtering and denoising the acquired dense sandstone image.

[0012] Optionally, the dense sandstone image generation model adds noise in the following manner:

[0013] For the generator network, noise is added twice in the first generator module and only once in the other generator modules except the first one.

[0014] Optionally, the dense sandstone image generation model adopts the following method to add self-attention layers in the generation network and the identification network respectively:

[0015] For the generator network, a self-attention layer is added between the two convolutional layers of the penultimate and second-to-last generator modules respectively;

[0016] For the discriminator network, a self-attention layer is added between the two convolutional layers in each discriminator module.

[0017] Optionally, the dynamically adjusting the position for determining whether overfitting is performed next time includes:

[0018] Set the initial mini-batch interval a0=N0, where N0 represents the maximum judgment interval value set;

[0019] The recursive formula is used to dynamically calculate the position where the adaptive discriminator network will judge the degree of model fit next time. The expression of the recursive formula is:

[0020]

[0021] Among them, r = 0 means no overfitting occurs, r = 1 means overfitting occurs, a n Indicates the interval corresponding to the nth judgment of whether overfitting occurs.

[0022] Optionally, the method further includes:

[0023] Constructing a search space for augmentation strategies; the search space for augmentation strategies includes multiple augmentation strategies, each augmentation strategy includes two augmentation operations, and the attributes of each augmentation operation include an operation type and an operation amplitude;

[0024] Adopting the augmentation strategy search method based on the adaptive stochastic natural gradient method, the search space of the constructed augmentation strategy is searched to obtain multiple augmentation strategies;

[0025] The obtained augmentation strategy is applied to a data set consisting of the acquired dense sandstone image and the generated dense sandstone image to obtain an augmented dense sandstone image.

[0026] Optionally, the augmentation strategy search method based on the adaptive stochastic natural gradient method is used to search in the constructed augmentation strategy search space, including:

[0027] Determine the optimization problem to be solved by the search, the expression is:

[0028]

[0029] Among them, f train () represents the training loss function of the dense sandstone image segmentation model, f val () represents the verification loss function of the dense sandstone image segmentation model, w represents the network weight of the dense sandstone image segmentation model, w∈W, W represents the network weight set in the gradient descent process, s represents the augmentation strategy, s∈S, S represents the search space of the constructed augmentation strategy, w * Indicates minimizing the training loss to find the optimal network weight, s * Indicates finding the optimal augmentation strategy by minimizing the verification loss using the optimal network weights;

[0030] The problem of solving the optimal augmentation strategy is converted into the optimization function J(w,θ), which is expressed as:

[0031]

[0032] Among them, θ is a hyperparameter, p θ (s) is a probability distribution with θ as a parameter, so that the verification loss function f val () minimizes the validation loss with respect to θ; J(w,θ) represents the random relaxation objective function;

[0033] The Monte Carlo method is used to estimate the gradient of w, which is expressed as:

[0034]

[0035] in, s i represents the i-th augmentation strategy, θ t Indicates the parameter value of θ at the current time step, w t Indicates the w parameter value at the current time step, N w The number of augmentation strategies collected under the current weight is updated using adaptive stochastic natural gradient descent.t , w t+1 =w t -l w G w (w t ,θ t ), G w (w t ,θ t ) represents the current gradient, l w represents the adaptive learning rate adjusted according to the parameters;

[0036] Use Monte Carlo to estimate the gradient with respect to θ Update θ using natural gradient descent t , the expression is:

[0037]

[0038] Among them, l θ represents the learning rate adjusted according to the parameters, F(θ t ) represents the Fisher information matrix, N θ Indicates the number of acquisition augmentation strategies under the current hyperparameters;

[0039] Determine the augmentation strategy based on the optimization results.

[0040] In a second aspect, an embodiment of the present invention further provides a dense sandstone image augmentation device based on deep learning, comprising:

[0041] A data acquisition module is used to acquire and pre-process labeled dense sandstone images; the labels are used to indicate the type of the corresponding dense sandstone images;

[0042] A model construction module is used to construct a dense sandstone image generation model based on the StyleGAN network. The dense sandstone image generation model is adjusted based on the StyleGAN network, using only one latent code to control the key features of the generated image, reducing the number of times noise is added in the generation network, and adding self-attention layers in the generation network and the discrimination network respectively.

[0043] A model training module is used to train the constructed dense sandstone image generation model based on the acquired dense sandstone image; during the training process, the augmentation strength is adjusted according to the degree of overfitting, and the position of the next overfitting judgment is dynamically adjusted;

[0044] The image generator module is used to generate a dense sandstone image using the trained dense sandstone image generation model.

[0045] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0046] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0047] Embodiments of the present invention provide a method, device, electronic device, and storage medium for augmenting dense sandstone images based on deep learning. Based on the characteristics of dense sandstone images, the present invention makes adjustments based on the StyleGAN network to obtain an image generation model that is more suitable for dense sandstone images. This model can generate high-quality dense sandstone images and amplify dense sandstone image data for use in training dense sandstone image recognition and segmentation, providing support for intelligent dense sandstone image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a flow chart of a method for enhancing dense sandstone images based on deep learning provided by one embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of a dense sandstone image generation model structure provided by an embodiment of the present invention;

[0051] Figure 3 This is a flow chart of another method for enhancing dense sandstone images based on deep learning provided by one embodiment of the present invention;

[0052] Figure 4 This is a hardware architecture diagram of an electronic device provided by one embodiment of the present invention;

[0053] Figure 5 This is a structural diagram of a dense sandstone image augmentation device based on deep learning provided by one embodiment of the present invention;

[0054] Figure 6 This is a structural diagram of another dense sandstone image augmentation device based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0056] As mentioned above, intelligent image recognition methods based on large data sets have the advantages of less human-computer interaction, faster recognition speed and more accurate recognition results compared to traditional recognition methods, and are gradually being applied to dense sandstone image analysis tasks. However, this type of image recognition and segmentation method relies on large-scale data sets for neural network training. When the training sample data is small, it will affect the training effect of the model. However, due to factors such as the difficulty of data collection, high annotation costs and privacy protection, dense sandstone image samples are scarce and usually cannot meet the training requirements of deep learning image recognition models. In order to increase the number of samples and improve the training effect of deep learning models, the present invention proposes a method, device, electronic device and storage medium for dense sandstone image augmentation based on deep learning, which can generate dense sandstone images using deep learning technology and realize image data augmentation for use in dense sandstone image recognition and segmentation training, providing technical support for intelligent identification of unconventional oil and gas reservoirs.

[0057] The specific implementation of the above concept is described below.

[0058] Please refer to Figure 1 , an embodiment of the present invention provides a dense sandstone image augmentation method based on deep learning, the method comprising:

[0059] Step 100: Acquire a labeled dense sandstone image and perform preprocessing; the label is used to indicate the type of the corresponding dense sandstone image;

[0060] Optionally, the Labelme tool can be used to label the original unlabeled image. The labels of tight sandstone images can be divided into seven types: quartz (Qu), feldspar (Fe), limestone (Li), primary pore (PP), casting pore (CP), cemented dissolution pore (CDP), and microcrack (Mi).

[0061] Step 102: construct a dense sandstone image generation model based on the StyleGAN network;

[0062] The dense sandstone image generation model is modified based on the existing StyleGAN network. It abandons the hybrid regularization operation and uses only one latent code to control the key features of the generated image. The number of times noise is added in the generation network is reduced, and self-attention layers are added to the generation network and the discriminator network respectively.

[0063] Step 104: training the constructed dense sandstone image generation model based on the acquired dense sandstone image;

[0064] During the training process, an adaptive augmentation strength adjustment cycle is adopted to adjust the augmentation strength according to the overfitting degree of the dense sandstone image generation model, and the position of the next overfitting judgment is dynamically adjusted;

[0065] Step 106: Generate a dense sandstone image using the trained dense sandstone image generation model.

[0066] The StyleGAN network uses latent codes to control key features of foreground objects in the generated image, such as shape, color, and texture. It further achieves fixed-ratio feature control with the help of a hybrid regularization operation, which uses two latent codes. However, in dense sandstone images, foreground objects are numerous, and hybrid regularization can cause feature fusion between adjacent foreground objects of different types, making the generated image difficult to distinguish and process. To address this issue, the present invention abandons hybrid regularization and uses only a single latent code to control key features in the generated image, eliminating the distortion, overlap, and unclear nature of foreground objects in the generated image.

[0067] The StyleGAN network includes a generation network (G) and a discrimination network (D). The generation network includes a mapping module and multiple generator modules, and the discrimination network includes multiple discriminator modules. In the generation network of the StyleGAN network, noise is added to each pixel in the generated image after each convolution to achieve random changes in image features. This noise only affects the feature details of the generated image and does not change the key features and overall structure of the image. However, for dense sandstone images, the foreground target features of the same type are similar and there are no obvious differences. Therefore, there is no need for too much random change. Adding too much noise will cause the generated image to contain unnecessary noise, reducing the image quality. Therefore, the present invention changes the original noise addition mechanism of the StyleGAN network. In the generation network, the number of times noise is added is reduced, thereby reducing the impact of noise on the generated image and improving the quality of dense sandstone images.

[0068] Taking into account the large number of foreground targets in dense sandstone images, the present invention also adds self-attention layers in the generation network and the identification network respectively as a supplement to the traditional convolution, so as to better extract the pattern features of the dense sandstone images and improve the imaging quality.

[0069] During the training process, the StyleGAN network uses a heuristic algorithm to determine the degree of model overfitting every four minibatches. The augmentation strength is reduced or increased based on the degree of model overfitting to alleviate the overfitting problem caused by limited data training. The heuristic algorithm is defined as:

[0070] r=E[sign(D trian )]

[0071] Among them, D trian Represents the performance of the discriminator module, sign() represents the indicator function, which returns 1 when the input value is greater than zero, otherwise it returns 0, E() represents the expectation, E[sign(D trian )] represents the mean of the discrimination results of the identification network on four consecutive small batches. When r = 0, it means that there is no overfitting, and when r = 1, it means that there is overfitting.

[0072] For dense sandstone images, due to the limited number of original images available as samples, using a fixed judgment interval to adjust the augmentation strength is difficult to effectively alleviate the overfitting problem. Therefore, this paper proposes an adaptive augmentation strength adjustment cycle, allowing the model to adjust the augmentation strength based on the degree of overfitting and dynamically adjust the position of the next judgment, thereby solving the overfitting problem.

[0073] Based on the characteristics of dense sandstone images, the embodiment of the present invention makes adjustments based on the StyleGAN network to obtain an image generation model that is more suitable for dense sandstone images. The model is trained using an adaptive augmentation intensity adjustment cycle, which can generate high-quality dense sandstone images for training dense sandstone image recognition and segmentation, providing technical support for realizing intelligent image recognition of dense sandstone images.

[0074] Described below Figure 1 How to perform the steps shown.

[0075] With respect to step 100 , the preprocessing includes filtering and denoising the acquired dense sandstone image.

[0076] During the acquisition of dense sandstone images, the camera's photosensitive element will produce a certain degree of response deviation to ambient light, resulting in some random noise in the image. The degree of noise directly affects the image quality. Filtering and noise reduction can effectively improve image quality.

[0077] Common denoising methods include mean filtering, Gaussian filtering, median filtering, and bilateral filtering. Given that dense sandstone image recognition and segmentation is essentially a semantic segmentation task, high requirements are placed on the edge quality of the objects to be segmented in dense sandstone images. This is because object edges are important features of sandstone grains and pores and are crucial for accurate segmentation. Bilateral filtering simultaneously considers both pixel proximity and grayscale similarity in the image during noise removal. Therefore, bilateral filtering is the preferred method for filtering denoising, as it better preserves edge information of objects in the image, thereby improving the accuracy and effectiveness of semantic segmentation.

[0078] For the dense sandstone image generation model constructed in step 102, noise can be added in the following manner:

[0079] For the generator network, noise is added twice in the first generator module and only once in the other generator modules except the first one.

[0080] Preferably, except for the first generator module (which adds noise twice), the other generator modules only add noise once after the second 3×3 convolutional layer.

[0081] The generative network consists of multiple generator modules, each of which includes two 3×3 convolutional layers. The first generator module adds noise twice, adding noise to the generated image after each convolution. Other generator modules add noise less frequently to minimize its impact on the generated image. Adding this after the second 3×3 convolutional layer ensures that image features are fully extracted, and noise does not affect the extracted features, thus improving image quality.

[0082] For the dense sandstone image generation model constructed in step 102, self-attention layers can be added to the generation network and the identification network respectively in the following manner:

[0083] For the generator network, a self-attention layer is added between the two convolutional layers of the penultimate generator module and the penultimate generator module respectively;

[0084] For the discriminator network, a self-attention layer is added between the two convolutional layers in each discriminator module.

[0085] like Figure 2As shown in a specific embodiment, the dense sandstone image generation model outputs an image of 512×512 pixels, and the generation network includes various generator modules corresponding to 4×4, 8×8 to 512×512 pixels, the penultimate generator module and the penultimate generator module, i.e., generator modules of 256×256 and 512×512 pixels, wherein a self-attention layer is added between the two 3×3 convolution layers of a single generator module. For the discriminator network, a self-attention layer is added between the two convolution layers of each discriminator module as a supplement to the traditional convolution, so as to better extract the pattern features of the dense sandstone image.

[0086] Preferably, if Figure 2 As shown in the figure, a self-attention layer is also added between the first and third fully connected layers (FC) of the mapping module to introduce cross-channel dependencies. The self-attention layer can learn the interdependencies between different channels to capture long-range dependencies between channels. This cross-channel dependency can help enhance local image features such as texture and lines, thereby improving the quality of the generated image. At the same time, the self-attention layer can also improve the model's representation capabilities, allowing it to better adapt to complex data distributions.

[0087] Furthermore, after adding the corresponding self-attention layers in the generator network and the discriminator network, the corresponding output feature maps are obtained in the following way:

[0088] Calculate the attention weight, the expression is:

[0089]

[0090] Among them, the attention weight α j,i Indicates the degree of attention paid to position i when synthesizing position j, represents the feature map from the previous convolutional layer, where N represents the number of locations in the feature map and W k Indicates that the input is linearly transformed to obtain the linear transformation matrix of the key vector. A 1×1 convolution module can be used, N p Indicates the number of attention weights that need to be calculated for each position. Usually, N p The value of is related to the length of the input sequence;

[0091] Based on the attention weight α j,i , calculate the global feature o of position j by weighted average j , the output expression of the attention layer is:

[0092]

[0093] Among them, W vRepresents a linear transformation matrix that linearly transforms the input to obtain a value vector, which can be implemented using a 1×1 convolution module.

[0094] Add the output of the attention layer to the original feature map by matrix to obtain the corresponding output feature map after adding the self-attention layer The expression is:

[0095] z i =x i +o i .

[0096] Regarding step 104, during the training process, dynamically adjusting the position for determining whether overfitting is next performed includes:

[0097] Set the initial mini-batch interval a0=N0, where N0 represents the maximum judgment interval value set;

[0098] The recursive formula is used to dynamically calculate the position where the adaptive discriminator network will judge the degree of model fit next time. The expression of the recursive formula is:

[0099]

[0100] Among them, r = 0 means no overfitting occurs, r = 1 means overfitting occurs, a n Indicates the interval corresponding to the nth judgment of whether overfitting occurs, that is, the position of the next judgment of whether overfitting occurs.

[0101] Using the above embodiment, first set the interval of the initial small batch a0 = N0, and use the recursive formula a during training n Dynamically calculate the position of the adaptive discrimination network to judge the degree of overfitting of the model next time, that is, every time a n After each round of training, an overfitting judgment is performed to determine whether the model is overfitting. If r = 1, it means that the model is overfitting, and the adjustment period interval a is reduced. n , until a n =1, if r=0, the adjustment interval can be increased until a n =N0. The degree of overfitting can be determined by the above formula r=E[sign(D trian )].

[0102] Alternatively, as Figure 3 As shown, the dense sandstone image augmentation method further includes:

[0103] Step 108, constructing a search space for the augmentation strategy;

[0104] The search space of the augmentation strategy includes multiple augmentation strategies, each augmentation strategy includes two augmentation operations, and the attributes of each augmentation operation include an operation type and an operation amplitude;

[0105] Step 110: using an augmentation strategy search method based on an adaptive stochastic natural gradient method to search in the constructed augmentation strategy search space to obtain multiple augmentation strategies;

[0106] Step 112 : Apply the obtained augmentation strategy to the data set consisting of the acquired dense sandstone image and the generated dense sandstone image to obtain an augmented dense sandstone image.

[0107] After obtaining the dense sandstone image generated by the dense sandstone image generation model, the above embodiment further augments the data set consisting of the original dense sandstone image and the dense sandstone image generated based on deep learning using an augmentation strategy to obtain more dense sandstone image data samples for training the dense sandstone image segmentation model.

[0108] Preferably, based on the basic architecture of automatic data augmentation (AutoAugment), the augmentation operation and the range of the operation amplitude are shown in Table 1 below:

[0109] Table 1 Augmentation operation and operation range

[0110]

[0111] To increase the diversity of augmented images while ensuring that the category distribution of the augmented images remains within a safe range, the present invention sets the following constraints:

[0112] (1) To increase the diversity of augmentation strategies, a fixed amplitude interval is set for each augmentation operation, the left and right boundaries of the amplitude interval are determined, and the amplitude is discretized into 10 values with uniform intervals; for example, an augmentation operation can be expressed as adjusting the image contrast to 1.3 times the original image;

[0113] (2) In order to maintain the rationality of the augmentation strategy and avoid the situation where the augmented image category deviates from the actual data distribution due to multiple augmentation operations on the same image, it is stipulated that each augmentation strategy only contains two augmentation operations.

[0114] Considering that searching for an augmentation operation has 10 11 Therefore, an efficient augmentation strategy search method is needed, that is, an augmentation strategy search method based on the adaptive stochastic natural gradient method.

[0115] Furthermore, step 110 uses an augmentation strategy search method based on an adaptive stochastic natural gradient method to search in the constructed augmentation strategy search space, including:

[0116] Determine the optimization problem to be solved by the search, the expression is:

[0117]

[0118] Among them, f train () represents the training loss function of the dense sandstone image segmentation model, f val () represents the verification loss function of the dense sandstone image segmentation model, w represents the network weight of the dense sandstone image segmentation model, w∈W, W represents the network weight set in the gradient descent process, s represents the augmentation strategy, s∈S, S represents the search space of the constructed augmentation strategy, w * Indicates minimizing the training loss to find the optimal network weight, s * This indicates finding the optimal augmentation strategy by minimizing validation loss using optimal network weights. The dense sandstone image segmentation model is a deep learning model used to perform intelligent image recognition, such as a U-Net network. After augmenting data based on one segmentation model, the method provided by the present invention can also be applied to other similar segmentation models for training.

[0119] In the above formula, f(w,s) (i.e. f train (),f val ()) represents the differentiable objective function of the augmented strategy s∈S and the network parameters w∈W. The goal of the augmented strategy search method is to use the gradient and Optimize w and s separately, first find the optimal network weight w by minimizing the training loss * , and then use the optimal network weight to minimize the verification loss to find the optimal augmentation strategy s * However, it is impossible to obtain the gradient of the objective function with respect to the augmented strategy s Therefore, it is difficult to optimize the augmentation strategy s using gradient descent. To optimize the strategy s using gradient descent, the present invention first uses random relaxation to transform the coupled optimization problem of weights and strategies into an optimization problem of a differentiable objective function J. Then, the natural gradient descent strategy is used in combination with an adaptive learning rate to simultaneously optimize the weights and strategies.

[0120] The idea of random relaxation is not to directly optimize the gradient of strategy s Instead, we define a probability distribution p with θ as parameter θ (s), so that the verification loss function f val () The verification loss about θ is minimized, that is, the problem of solving the optimal augmentation strategy is converted into the optimization function J(w,θ), which is expressed as:

[0121]

[0122] Here, J(w,θ) represents the random relaxation objective function, which has all the properties of the objective function f(w,s) and makes both w and θ differentiable, so gradient descent can be used to update w and θ. θ is a hyperparameter, which usually refers to some hyperparameters in data augmentation methods, such as parameters for image scaling, rotation, cropping, etc., which are used to control the degree and method of data augmentation. θ (s) is a probability distribution with θ as a parameter. That is, given the parameter θ, a set of probability values can be calculated to represent the probability of the corresponding data augmentation method appearing in the data set. By adjusting the parameter θ, the verification loss function f can be optimized. val (), thereby improving the generalization performance of the model.

[0123] Due to the gradient It is difficult to handle, so the Monte Carlo method (MC) is used to estimate the gradient about w, which is expressed as:

[0124]

[0125] in, s i represents the i-th augmentation strategy, θ t Indicates the parameter value of θ at the current time step, w t Indicates the w parameter value at the current time step, N w The number of augmentation strategies collected under the current weight is updated using adaptive stochastic natural gradient descent. t , due to the use of G w (w t ,θ t ) instead of the gradient with respect to w w t+1 =w t -l w G w (w t ,θ t ) represents the process of updating network weights according to time steps, G w (w t ,θ t ) represents the current gradient, l w represents the adaptive learning rate adjusted according to the parameters, used to find w.

[0126] Use Monte Carlo to estimate the gradient with respect to θ Since the distance between two probability distributions does not have the Euclidean property, we use natural gradient descent to update θ. t , the expression is:

[0127]

[0128] Among them, lθ represents the learning rate adjusted according to the parameters, used to find θ, F(θ t ) represents the Fisher information matrix (i.e., Fisher matrix), the superscript “-1” represents the inverse matrix, N θ Indicates the number of acquisition augmentation strategies under the current hyperparameters.

[0129] Determine the augmentation strategy based on the optimization results.

[0130] The above embodiment uses an adaptive learning rate to accelerate training so as to quickly determine the optimal augmentation strategy. At the same time, since the search process takes into account the impact on the segmentation model, the obtained data can be used as samples to train the segmentation model, thereby obtaining a better intelligent segmentation model.

[0131] like Figure 4 、 Figure 5 As shown, the embodiment of the present invention provides a dense sandstone image augmentation device based on deep learning (abbreviated as image augmentation device, device). The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 4 As shown in FIG, a hardware architecture diagram of an electronic device in which a dense sandstone image augmentation device based on deep learning is provided in an embodiment of the present invention, except Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 5 As shown, as a device in a logical sense, the CPU of the electronic device in which it is located reads the corresponding computer program in the non-volatile memory into the internal memory and runs it. This embodiment provides a dense sandstone image augmentation device based on deep learning, including:

[0132] The data acquisition module 501 is used to acquire a labeled dense sandstone image and perform preprocessing; the label is used to indicate the type of the corresponding dense sandstone image;

[0133] Model construction module 502 is used to construct a dense sandstone image generation model based on the StyleGAN network; the dense sandstone image generation model is adjusted based on the StyleGAN network, using only one latent code to control the key features of the generated image, reducing the number of times noise is added in the generation network, and adding self-attention layers in the generation network and the discrimination network respectively;

[0134] The model training module 503 is used to train the constructed dense sandstone image generation model based on the acquired dense sandstone image; during the training process, the augmentation strength is adjusted according to the degree of overfitting, and the position of the next overfitting judgment is dynamically adjusted;

[0135] The image generator module 504 is configured to generate a dense sandstone image using the trained dense sandstone image generation model.

[0136] Preferably, if Figure 6 As shown, the device also includes:

[0137] A space construction module 505 is used to construct a search space for augmentation strategies; the search space for augmentation strategies includes multiple augmentation strategies, each augmentation strategy includes two augmentation operations, and the attributes of each augmentation operation include an operation type and an operation amplitude;

[0138] A strategy search module 506 is configured to use an augmentation strategy search method based on an adaptive stochastic natural gradient method to search in the constructed augmentation strategy search space to obtain multiple augmentation strategies;

[0139] The data augmentation module 507 is configured to apply the obtained augmentation strategy to a data set consisting of the acquired dense sandstone image and the generated dense sandstone image to obtain an augmented dense sandstone image.

[0140] In an embodiment of the present invention, the data acquisition module 501 can be used to execute step 100 in the above method embodiment, the model construction module 502 can be used to execute step 102 in the above method embodiment, the model training module 503 can be used to execute step 104 in the above method embodiment, the image generator module 504 can be used to execute step 106 in the above method embodiment, the space construction module 505 can be used to execute step 108 in the above method embodiment, the strategy search module 506 can be used to execute step 110 in the above method embodiment, and the data augmentation module 507 can be used to execute step 112 in the above method embodiment.

[0141] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on a device for deep learning-based dense sandstone image augmentation. In other embodiments of the present invention, a device for deep learning-based dense sandstone image augmentation may include more or fewer components than illustrated, or may combine or separate certain components, or employ different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of both.

[0142] The information interaction, execution process, etc. between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0143] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, a method for enhancing dense sandstone images based on deep learning in any embodiment of the present invention is implemented.

[0144] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes a method for enhancing dense sandstone images based on deep learning in any embodiment of the present invention.

[0145] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0146] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0147] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0148] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0149] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0150] It should be noted that, in this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0151] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A dense sandstone image augmentation method based on deep learning, characterized in that: include: Obtaining a labeled dense sandstone image and performing preprocessing; the label is used to indicate the type of the corresponding dense sandstone image; A dense sandstone image generation model based on the StyleGAN network was constructed. This dense sandstone image generation model was modified based on the StyleGAN network, using only one latent code to control the key features of the generated image, reducing the number of times noise was added in the generation network, and adding self-attention layers in both the generation network and the discriminator network. Based on the acquired dense sandstone image, training the constructed dense sandstone image generation model; During the training process, the augmentation strength is adjusted according to the degree of overfitting, and the position of the next overfitting judgment is dynamically adjusted; generating a dense sandstone image using the trained dense sandstone image generation model; The dense sandstone image generation model adopts the following method to add self-attention layers in the generation network and the identification network respectively: For the generator network, a self-attention layer is added between the two convolutional layers of the penultimate and second-to-last generator modules respectively; For the discriminator network, a self-attention layer is added between the two convolutional layers of each discriminator module; The dynamic adjustment of the position for determining whether overfitting is performed next time includes: Set the initial mini-batch interval ,in N 0 indicates the maximum judgment interval value set; The recursive formula is used to dynamically calculate the position where the adaptive discriminator network will judge the degree of model fit next time. The expression of the recursive formula is: in, When , it means that no overfitting occurs. When , it indicates overfitting. Indicates the interval corresponding to the nth judgment of whether overfitting occurs.

2. The dense sandstone image augmentation method according to claim 1, characterized in that: The preprocessing includes filtering and denoising the acquired dense sandstone image.

3. The dense sandstone image augmentation method according to claim 1, characterized in that: The dense sandstone image generation model adds noise in the following way: For the generator network, noise is added twice in the first generator module and only once in the other generator modules except the first one.

4. The dense sandstone image augmentation method according to claim 1, further comprising: Constructing the search space for augmentation strategies; The search space of the augmentation strategy includes multiple augmentation strategies, each augmentation strategy includes two augmentation operations, and the attributes of each augmentation operation include an operation type and an operation amplitude; Adopting the augmentation strategy search method based on the adaptive stochastic natural gradient method, the search space of the constructed augmentation strategy is searched to obtain multiple augmentation strategies; The obtained augmentation strategy is applied to a data set consisting of the acquired dense sandstone image and the generated dense sandstone image to obtain an augmented dense sandstone image.

5. The dense sandstone image augmentation method according to claim 4, characterized in that: The augmentation strategy search method based on the adaptive stochastic natural gradient method is used to search in the constructed augmentation strategy search space, including: Determine the optimization problem to be solved by the search, the expression is: in, () represents the training loss function of the dense sandstone image segmentation model, () represents the validation loss function of the dense sandstone image segmentation model, w represents the network weight of the dense sandstone image segmentation model, , W represents the network weight set in the gradient descent process, s represents the augmentation strategy, , S represents the search space of the constructed augmentation strategy, Indicates minimizing the training loss to find the optimal network weights, Indicates finding the optimal augmentation strategy by minimizing the verification loss using the optimal network weights; Convert the problem of solving the optimal augmentation strategy into an optimization function , the expression is: in, is a hyperparameter, For is the probability distribution of the parameters, so that the validation loss function ()about The verification loss is minimal; represents the random relaxation objective function; Use the Monte Carlo method to estimate The gradient of is expressed as: in, ~ , Indicates the i An augmentation strategy, Indicates the current time step parameter values, Indicates the current time step parameter values, The number of augmentation strategies collected under the current weight is updated using adaptive stochastic natural gradient descent , , represents the current gradient, represents the adaptive learning rate adjusted according to the parameters; Use the Monte Carlo method to estimate Gradient , updated using natural gradient descent , the expression is: in, represents the learning rate adjusted according to the parameters, represents the Fisher information matrix, Indicates the number of acquisition augmentation strategies under the current hyperparameters; Determine the augmentation strategy based on the optimization results.

6. A dense sandstone image augmentation device based on deep learning, characterized in that: include: A data acquisition module is used to acquire and pre-process labeled dense sandstone images; the labels are used to indicate the type of the corresponding dense sandstone images; A model construction module is used to construct a dense sandstone image generation model based on the StyleGAN network. The dense sandstone image generation model is adjusted based on the StyleGAN network, using only one latent code to control the key features of the generated image, reducing the number of times noise is added in the generation network, and adding self-attention layers in the generation network and the discrimination network respectively. A model training module is used to train the constructed dense sandstone image generation model based on the acquired dense sandstone image; during the training process, the augmentation strength is adjusted according to the degree of overfitting, and the position of the next overfitting judgment is dynamically adjusted; An image generator module, configured to generate a dense sandstone image using the trained dense sandstone image generation model; The dense sandstone image generation model adopts the following method to add self-attention layers in the generation network and the identification network respectively: For the generator network, a self-attention layer is added between the two convolutional layers of the penultimate and second-to-last generator modules respectively; For the discriminator network, a self-attention layer is added between the two convolutional layers of each discriminator module; The dynamic adjustment of the position for determining whether overfitting is performed next time includes: Set the initial mini-batch interval ,in N 0 indicates the maximum judgment interval value set; The recursive formula is used to dynamically calculate the position where the adaptive discriminator network will judge the degree of model fit next time. The expression of the recursive formula is: in, When , it means that no overfitting occurs. When , it indicates overfitting. Indicates the interval corresponding to the nth judgment of whether overfitting occurs.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the dense sandstone image augmentation method according to any one of claims 1 to 5 is implemented.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the dense sandstone image augmentation method according to any one of claims 1 to 5.

Images