Rock multi-scale three-dimensional reconstruction method and device based on deep learning

Abstract

The invention provides a rock multi-scale three-dimensional reconstruction method and device based on deep learning, and relates to the technical field of computer images, and the method comprises the steps: obtaining rock geometric features and boundary features of a rock test piece under different scales; training a generative adversarial network based on geometric features and boundary features of the rock test piece under different scales, and sequentially generating a structure image sequence of each scale according to a sequence of the scales from small to large based on the trained generative adversarial network; and under the condition that the generated structure image sequence of each scale reaches the corresponding target scale, obtaining a rock multi-scale three-dimensional reconstruction model based on the structure image sequence of each scale. According to the method, the rock multi-scale three-dimensional reconstruction model is obtained through structure image sequence rendering based on all scales, the scale limitation is broken, macro-fine-micro rock structure feature information association and three-dimensional model construction are realized, and multi-type feature autonomous classification reconstruction and high interpretability under different scales of rocks are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer image technology, and in particular, to a method and device for multi-scale three-dimensional reconstruction of rocks based on deep learning. Background Art

[0002] As a porous and multi-mineral heterogeneous rock material, the rock structure at different scales will decisively affect the mechanical and fracture properties of the rock. The research on the evolution characteristics and fracture mechanism of the rock multi-scale structure has great significance for the technologies of deep underground engineering disasters (such as rock bursts and water inrush from surrounding rocks, etc.) and deep-earth resources (such as geothermal energy and petroleum, etc.). The research results have broad application prospects and economic value.

[0003] Modern high-resolution imaging technology enables the study of rock structures from the macroscopic to the microscopic scale. However, due to the limitation of the physical size of the rock specimen by the imaging resolution, it is still impossible to effectively correlate the macro-fine-micro rock structure characteristics of the rock. When the size of the rock specimen is large, it is impossible to obtain the nano-micro rock structure characteristics (such as pores); when the size of the rock specimen is small, it is impossible to obtain the macroscopic rock structure characteristics.

[0004] Therefore, all the rock 3D reconstruction technologies based on these imaging technologies are unable to effectively correlate the multi-scale structure characteristics of the rock. When establishing a 3D model of the rock structure at the macroscopic scale, characteristic structures such as pore connectivity are lacking. When establishing a 3D model at the fine-microscopic scale, it is impossible to represent characteristic structures such as macroscopic fractures. The reconstructed fine-microscopic scale rock 3D model is not sufficient to characterize the physical and mechanical properties and structural characteristics of the rock material and cannot be applied to actual engineering research. Summary of the Invention

[0005] The present invention provides a method and device for multi-scale three-dimensional reconstruction of rocks based on deep learning, aiming to solve the defect that the rock three-dimensional reconstruction technology based on high-resolution imaging technology in the prior art is difficult to effectively correlate the multi-scale characteristics of the rock, resulting in low calculation accuracy of the physical properties of the rock, and to realize a method and device for multi-scale three-dimensional reconstruction of rocks with high reconstruction accuracy.

[0006] The present invention provides a method for multi-scale three-dimensional reconstruction of rocks based on deep learning, including:

[0007] Obtaining the rock geometric features and boundary features of the rock specimen at different scales;

[0008] Training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, and generating a structural image sequence for each scale in ascending order of scale based on the trained generative adversarial network;

[0009] In the case where the generated structural image sequences at each scale reach the corresponding target scale, a rock multi-scale three-dimensional reconstruction model is obtained based on the structural image sequences at each scale.

[0010] According to a rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention, the step of obtaining the rock geometric features and boundary features of the rock specimen at different scales specifically includes:

[0011] Respectively obtain the structural image sequences of the rock specimen at multiple different scales;

[0012] Segment each image in the structural image sequence of the rock specimen at each scale to obtain the rock structural feature image at each scale;

[0013] Scan the rock structural feature image at each scale to obtain the geometric features and boundary features of the rock specimen at different scales.

[0014] According to a rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention, the step of segmenting each image in the structural image sequence of the rock specimen at each scale to obtain the rock structural feature image at each scale specifically includes:

[0015] Input the structural image sequence of the rock specimen at each scale into a pre-trained deep learning segmentation model to obtain the rock structural feature image at each scale output by the deep learning segmentation model;

[0016] Wherein, the deep learning segmentation model is trained with a loss function constructed based on the percentage loss, geometric loss, and size loss of the rock specimen at each scale with a U-shaped convolutional neural network as the basic framework.

[0017] According to a rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention, the step of segmenting each image in the structural image sequence of the rock specimen at each scale to obtain the rock structural feature image at each scale specifically includes:

[0018] Segment each image in the structural image sequence of the rock specimen at each scale, and perform labeled representation on the segmentation result to obtain the labeled rock structural feature image at each scale;

[0019] The step of scanning the rock structural feature image at each scale to obtain the geometric features and boundary features of the rock specimen at different scales specifically includes:

[0020] Scan the rock structure feature images at each of the scales using a pre - constructed feature extraction template to obtain the geometric features and boundary features of the rock specimen at different scales;

[0021] Before the step of training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, the following steps are further included:

[0022] Construct a feature classification vectorizer for each rock structure feature at each scale based on the scanning results, and construct a feature classification fusion device representing the structural features of the rock specimen at all scales based on the feature classification vectorizers of each rock structure feature.

[0023] According to a multi - scale three - dimensional reconstruction method of rock based on deep learning provided by the present invention, the step of training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales specifically includes:

[0024] Establish a generator of the generative adversarial network based on the feature classification fusion device;

[0025] Construct a discriminator of the generative adversarial network based on the true values and predicted values of the rock specimen at each scale.

[0026] According to a multi - scale three - dimensional reconstruction method of rock based on deep learning provided by the present invention, the step of obtaining a multi - scale three - dimensional reconstruction model of rock based on the structural image sequences at each scale specifically includes:

[0027] Form three - dimensional voxel data sets of different scales of rock in a three - dimensional voxel space based on the structural image sequences at each scale;

[0028] Perform visual rendering on the three - dimensional voxel data sets of different scales of the rock to obtain a multi - scale three - dimensional reconstruction model of the rock.

[0029] The present invention also provides a multi - scale three - dimensional reconstruction device of rock based on deep learning, including:

[0030] An acquisition module for acquiring the geometric features and boundary features of a rock specimen at different scales;

[0031] A training module for training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, and sequentially generating structural image sequences at each scale in ascending order of scale based on the trained generative adversarial network;

[0032] A construction module for obtaining a multi - scale three - dimensional reconstruction model of rock based on the structural image sequences at each scale when the structural image sequences at each generated scale reach their corresponding target scales.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method for multi-scale three-dimensional reconstruction of rock based on deep learning as described in any one of the above is implemented.

[0034] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for multi-scale three-dimensional reconstruction of rock based on deep learning as described in any one of the above is implemented.

[0035] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method for multi-scale three-dimensional reconstruction of rock based on deep learning as described in any one of the above is implemented.

[0036] The method and device for multi-scale three-dimensional reconstruction of rock based on deep learning provided by the present invention generate structural image sequences of different scales through the geometric features and boundary features of rock specimens at multiple different scales, and render a multi-scale three-dimensional reconstruction model of rock based on the structural image sequences of all scales. It breaks through the scale limitation, realizes the association of macro-fine-micro rock structure feature information and the construction of a three-dimensional model, realizes the autonomous classification reconstruction and high interpretability of multiple types of features of rock at different scales, and provides a reliable real geometric model and construction design parameters for actual engineering research. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 is a schematic flowchart of the method for multi-scale three-dimensional reconstruction of rock based on deep learning provided by the present invention;

[0039] Figure 2 In (a) of, it is a macroscopic-scale slope entity model reconstructed by the method for multi-scale three-dimensional reconstruction of rock based on deep learning provided by the present invention;

[0040] Figure 2 In (b) of, it is a partial enlarged view of the mesoscopic-scale slope frame line reconstructed by the method for multi-scale three-dimensional reconstruction of rock based on deep learning provided by the present invention;

[0041] Figure 2 In (c) of, it is a further enlarged view of the microscopic scale reconstructed by the method for multi-scale three-dimensional reconstruction of rock based on deep learning provided by the present invention;

[0042] Figure 3 Among them, (a) is the original microscopic sandstone microstructure image used in the image segmentation process of the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0043] Figure 3 Among them, (b) is the original mesoscopic sandstone microstructure image used in the image segmentation process of the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0044] Figure 3 Among them, (c) is the original macroscopic sandstone microstructure image used in the image segmentation process of the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0045] Figure 3 Among them, (d) is the deep learning segmentation model used in the image segmentation process of the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0046] Figure 3 Among them, (e) is the segmentation result of 1-microscopic pores obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0047] Figure 3 Among them, (f) is the segmentation result of 3-1-microscopic feldspar obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0048] Figure 3 Among them, (g) is the segmentation result of 3-2-microscopic quartz obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0049] Figure 3 Among them, (h) is the segmentation result of 3-3-microscopic muscovite obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0050] Figure 3 Among them, (i) is the segmentation result of 4-mesoscopic pores obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0051] Figure 3 Among them, (j) is the segmentation result of 5-mesoscopic solid matrix obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0052] Figure 3 Among them, (k) is the segmentation result of 6-macroscopic cracks obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0053] Figure 3 The (l) in it is the segmentation result of the 7-macroscopic solid matrix obtained by image segmentation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0054] Figure 4 The (a) in it is a schematic diagram of the feature extraction template in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0055] Figure 4 The (b) in it is a schematic diagram of the scanning process in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0056] Figure 4 The (c) in it is a schematic diagram of the feature classification and fusion device in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0057] Figure 5 The (a) in it is the microscopic structure image input during generation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0058] Figure 5 The (b) in it is the rock multi-scale information fusion deep learning model used when generating the mesoscopic structure image in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0059] Figure 5 The (c) in it is the mesoscopic structure image input during generation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0060] Figure 5 The (d) in it is the macroscopic structure image input during generation in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0061] Figure 5 The (e) in it is the rock multi-scale information fusion deep learning model and the fine-microscopic feature information cell used when generating in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0062] Figure 5 The (f) in it is the rock macroscopic three-dimensional model reconstructed in the rock multi-scale three-dimensional reconstruction method based on deep learning provided by the present invention;

[0063] Figure 6 It is a schematic diagram of the structure of the rock multi-scale three-dimensional reconstruction device based on deep learning provided by the present invention;

[0064] Figure 7 It is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0065] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0066] The following will introduce Figures 1 to 5 the method for multi-scale three-dimensional reconstruction of rocks based on deep learning according to the present invention. As Figure 1 shown, it includes:

[0067] Step 101: Obtain the rock geometric features and boundary features of the rock specimen at different scales;

[0068] In order to provide effective and reliable construction technical parameter indicators for the prevention and control of actual engineering geological disasters and deep-earth resource development projects based on the constructed three-dimensional rock model, rock samples are obtained from the target area and processed into rock specimens so that the structural features of the rock specimens can represent the rock structural features of the target area.

[0069] Specifically, standard-sized rock specimens are processed and prepared according to research requirements. In this embodiment, standard rock specimens with a length, width, and height of 100 millimeters are processed and prepared.

[0070] Furthermore, based on the prepared rock specimens, obtain their rock geometric features and boundary features at different scales.

[0071] Among them, different scales can be macroscopic scale, mesoscopic scale, and microscopic scale, and the specific scale range can be determined according to research requirements.

[0072] The rock geometric features and boundary features characterize the geometric features and boundary features of the rock structural features of the rock specimen at each scale. It can be understood that different scales correspond to different rock structural features of the rock specimen.

[0073] For example, at the microscopic scale, the rock structural features of the rock specimen include microscopic pores, microscopic cracks, and mineral phase aggregates. Therefore, the rock geometric features and boundary features at the microscopic scale are the geometric features and boundary features of microscopic pores, microscopic cracks, and mineral phase aggregates.

[0074] At the mesoscopic scale, the rock structural features of the rock specimen include mesoscopic pore phase and mesoscopic solid matrix phase. Therefore, the rock geometric features and boundary features at the mesoscopic scale are the geometric features and boundary features of the mesoscopic pore phase and mesoscopic solid matrix phase.

[0075] At the macroscopic scale, the rock structure characteristics of a rock specimen include macroscopic cracks and a macroscopic solid matrix phase. Therefore, the geometric characteristics and boundary characteristics of the rock at the macroscopic scale are the geometric characteristics and boundary characteristics of the macroscopic cracks and the macroscopic solid matrix phase.

[0076] It can be understood that the geometric characteristics and boundary characteristics of the rock specimen at different scales characterize the shape and distribution characteristics of each substructure of the rock specimen at the corresponding scale.

[0077] Step 102: Train a generative adversarial network based on the geometric characteristics and boundary characteristics of the rock specimen at different scales, and generate a sequence of structural images for each scale in ascending order of scale based on the trained generative adversarial network.

[0078] Optionally, after vectorizing and characterizing the geometric characteristics and boundary characteristics of the rock specimen at different scales, a dataset for training a generative adversarial network (GAN) is constructed, and the GAN is trained so that the trained GAN can generate two-dimensional structural images corresponding to each scale based on the input geometric characteristics and boundary characteristics of the rock specimen at each scale.

[0079] Use the trained GAN to repeatedly generate based on the geometric characteristics and boundary characteristics of the rock specimen at each scale. When generating images, a sequence of structural images for each scale is generated in ascending order of scale.

[0080] Step 103: When the sequence of structural images for each generated scale reaches the corresponding target scale, obtain a rock multi-scale three-dimensional reconstruction model based on the sequence of structural images for each scale.

[0081] Optionally, the target scale is determined according to the target size and / or reconstruction accuracy of the three-dimensional reconstruction model to be reconstructed. The larger the target size and the higher the reconstruction accuracy, the larger the target scale, that is, the more images in the sequence of structural images for each scale.

[0082] The sequence of structural images for each scale generated by the GAN reaches the corresponding target scale, that is, the number of structural images for each generated scale can be used to render and generate a rock multi-scale three-dimensional reconstruction model. On this basis, the rock multi-scale three-dimensional reconstruction model can be reconstructed according to the obtained two-dimensional sequence of structural images for each scale.

[0083] At this time, the reconstructed rock multi-scale three-dimensional reconstruction model can simultaneously characterize the structural characteristics of the rock at each scale. In this embodiment, it can simultaneously characterize the structural characteristics of the rock at the macroscopic, microscopic, and mesoscopic scales.

[0084] For example, after sampling the slope, rock specimens corresponding to the slope are prepared, and the method in the present invention is used for three-dimensional reconstruction based on the rock geometric features and boundary features of the rock specimens to obtain a three-dimensional reconstruction model of the slope at the macroscopic scale as shown in Figure 2 (a); magnifying it can obtain a wireframe model of the three-dimensional slope and a partial enlarged view at the mesoscopic scale, as shown in Figure 2 (b); further magnifying it can obtain an enlarged view representing the mesoscopic structure of the slope, as shown in Figure 2 (c).

[0085] The present invention generates structural image sequences at different scales through the geometric features and boundary features of rock specimens at multiple different scales, and renders a multi-scale three-dimensional reconstruction model of the rock based on the structural image sequences at all scales, breaking through the scale limitation, realizing the association of macro-meso-micro rock structure feature information and the construction of a three-dimensional model, achieving the autonomous classification and reconstruction of multiple types of features and high interpretability of the rock at different scales, and providing a reliable true geometric model and construction design parameters for actual engineering research.

[0086] In the method for multi-scale three-dimensional reconstruction of rock based on deep learning in the present invention, the step of obtaining the rock geometric features and boundary features of the rock specimens at different scales specifically includes:

[0087] Obtaining structural image sequences of the rock specimens at multiple different scales respectively;

[0088] For the prepared rock specimens, digital imaging is performed using high-resolution imaging technology with imaging resolutions corresponding to different scales.

[0089] Optionally, the imaging resolution is determined according to the scale requirement.

[0090] In this embodiment, the rock is digitally imaged macroscopically, mesoscopically, and microscopically by setting a macroscopic imaging resolution of 0.75 mm, a mesoscopic imaging resolution of 20 μm, and a microscopic imaging resolution of 5 μm to obtain structural image sequences corresponding to the macroscopic scale, mesoscopic scale, and microscopic scale respectively.

[0091] Segmenting each image in the structural image sequence of the rock specimens at each scale to obtain rock structure feature images at each scale;

[0092] On this basis, each image in the structural image sequence of the rock specimens at each scale is segmented to obtain rock structure feature images at each scale.

[0093] For example, segmenting the images in the structural image sequence at the macroscopic scale to obtain two rock structure feature images corresponding to the macroscopic crack and the macroscopic solid matrix phase respectively.

[0094] Segment the images in the structural image sequence at the mesoscopic scale to obtain two rock structural feature images corresponding to the mesoscopic pore phase and the mesoscopic solid matrix phase respectively.

[0095] Segment the images in the structural image sequence at the microscopic scale to obtain three rock structural feature images corresponding to the microscopic pores, microscopic fractures, and mineral phase aggregates.

[0096] Optionally, an image segmentation network can be pre-trained based on existing image segmentation models for segmentation, so as to obtain rock structural feature images at each scale according to the structural images at each scale.

[0097] Scan the rock structural feature images at each scale to obtain the geometric features and boundary features of the rock specimen at different scales.

[0098] Optionally, scan the rock structural feature images at each scale, convert the images into feature representations, so as to obtain the geometric features and boundary features of the rock specimen at different scales for training the GAN.

[0099] In the method for multi-scale three-dimensional reconstruction of rocks based on deep learning of the present invention, the step of segmenting each image in the structural image sequence of the rock specimen at each scale to obtain rock structural feature images at each scale specifically includes:

[0100] Input the structural image sequence of the rock specimen at each scale into a pre-trained deep learning segmentation model to obtain rock structural feature images at each scale output by the deep learning segmentation model;

[0101] Among them, the deep learning segmentation model is trained with a loss function constructed based on the percentage loss, geometric loss, and size loss of the rock specimen at each scale with a U-shaped convolutional neural network as the basic framework.

[0102] In order to segment the images in the structural image sequence of the rock specimen at different scales, in this embodiment, a deep learning segmentation (Multiscale-U-Net) model based on a multi-class discrimination damage function and a multi-scale convolution kernel function at different scales is established with a U-shaped convolutional neural network (U-Net) as the basis to realize the automatic recognition, selection, and segmentation of the geometric features and boundary features of the rock structure at different scales of the rock.

[0103] Among them, the loss function of the Multiscale-U-Ne model is as follows:

[0104]

[0105] In the formula, L totalrepresents the total loss function, represents the percentage loss function of the rock structure, Loss boudary represents the geometric structure loss function, Loss r represents the size loss function; λ represents the weight coefficient of different scales, represents the predicted percentage content, represents the true percentage content value, represents the percentage content function, f i j represents the structure of the i-th rock of the j-th class at different scales.

[0106] N represents the image dimension, represents the predicted boundary image, represents the true boundary image; P r represents the predicted size of the rock structure, T r represents the true size of the rock structure, f i j-k and r represent the k-th sub-phase and size information of the j-th class structure of the i-th rock at different scales.

[0107] The multi-scale convolution kernel function of the Multiscale-U-Ne model is as follows:

[0108]

[0109] In the formula, F(x i ; θ) represents the multi-scale convolution kernel function, Γ concat represents the connection function, g represents the Gaussian convolution kernel function, θ represents the weight bias of the multi-scale convolution layer, x i represents the input of the multi-scale convolution kernel function.

[0110] Optionally, take some images in the structural image sequence of the rock specimen obtained at each scale as the input dataset, train the Multiscale-U-Net model, and obtain the optimized Multiscale-U-Net segmentation model by adjusting the training times, batch size and learning rate hyperparameters.

[0111] Optionally, in the test phase, take another part of the images as the input dataset, test and verify the optimized Multiscale-U-Net model obtained by training to ensure the accuracy of image segmentation.

[0112] Using the Multiscale-U-Net model can complete the segmentation of rock structure features at different scales respectively.

[0113] In the method for multi-scale three-dimensional reconstruction of rock based on deep learning of the present invention, the step of segmenting each image in the structural image sequence of the rock specimen at each scale to obtain the rock structure feature image at each scale specifically includes:

[0114] Segment each image in the structural image sequence of the rock specimen at each scale, and perform labeled characterization on the segmentation result to obtain the labeled rock structure feature image at each scale;

[0115] Furthermore, in order to more quickly convert the rock structure represented by the segmented image into a feature representation for generative adversarial network training, during the segmentation process, the segmentation result, that is, the segmented rock structure feature image, is subjected to labeled characterization.

[0116] Optionally, perform digital label characterization on the segmented rock structure feature image in sequence, as Figure 3 shown.

[0117] For example, the rock structure features at the microscale are 1 - micro-pores, 2 - micro-fractures, 3 - mineral phase aggregates (for mineral sub-phases, they are respectively characterized by sub-sequence numbers, such as 3-1 - mineral phase 1, 3-2 - mineral phase 2).

[0118] The rock structure features at the mesoscale are 4 - meso-pore phase, 5 - meso-solid matrix phase (composed of mineral aggregates, and different types of mineral phases cannot be identified at the mesoscale, so it is characterized by a single digital label, which can be called a micro-feature information cell unit);

[0119] The rock structure features at the macroscale are 6 - macro-cracks, 7 - macro-solid matrix phase, (composed of fine - micro solid matrices, including all its fine - micro structural characteristics, so it can be called a meso - micro feature information cell unit).

[0120] Optionally, pre-train the Multiscale-U-Net model so that the model can automatically complete the labeled characterization of the segmentation result during segmentation:

[0121]

[0122] In the formula, F i j-k represents the identified segmented and labeled rock structure feature image; gl represents the label sequence number of the rock structure phase at different scales identified and divided.

[0123] The step of scanning the rock structure feature image at each scale to obtain the geometric features and boundary features of the rock specimen at different scales specifically includes:

[0124] Scan the rock structure feature images at each scale using a pre - constructed feature extraction template to obtain the geometric features and boundary features of the rock specimen at different scales;

[0125] On this basis, scan the labeled rock structure feature images at each scale using a pre - constructed feature extraction template to obtain the geometric features and boundary features of the rock specimen at different scales.

[0126] Optionally, as shown in (a) of Figure 4 , a 3×3 identity matrix feature extraction template is used for scanning in this embodiment.

[0127] Before the step of training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, the following steps are further included:

[0128] Construct a feature classification vectorizer for each rock structure feature at each scale based on the scanning results, and construct a feature classification fusion vectorizer representing the structural features of the rock specimen at all scales based on the feature classification vectorizers of each rock structure feature.

[0129] Furthermore, based on the scanning results, establish a feature classification vectorizer for the percentage content of each type of structure and the size information distribution of its sub - phases at different scales, and construct a corresponding feature classification fusion vectorizer based on the feature classification vectorizer. Taking macro - scale, meso - scale and micro - scale as examples:

[0130] Multiscale C ={Macro - class, Meso - calss, Miccro - class};

[0131]

[0132] In the formula, Multiscale C represents the rock multi - scale feature classification fusion vectorizer; Macro - class represents the rock macro - feature classification vectorizer, Meso - calss represents the rock meso - feature classification vectorizer, Miccro - class represents the rock micro - feature classification vectorizer; represents the feature classification vectorizer of the percentage content of the rock structure, and subclass r represents the feature classification vectorizer of the rock size information.

[0133] It can be understood that different scales are selected, and the feature classification vectorizers corresponding to each scale are also different.

[0134] In the method for multi-scale three-dimensional reconstruction of rock based on deep learning of the present invention, the step of training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales specifically includes:

[0135] Based on the feature classification and fusion device, a generator of the generative adversarial network is established;

[0136] Based on the true value and predicted value of the rock specimen at each scale, a discriminator of the generative adversarial network is constructed.

[0137] In order to generate corresponding rock sequence images based on the geometric features and boundary features of the rock structure at different scales, in this embodiment, a deep learning (Multiscale-FF-GAN) model for multi-scale information fusion of rock based on a generalized generative network (GAN) and a feature fusion algorithm (FF) is established, and a multi-scale relative error analyzer is constructed as a discriminator to determine whether to stop the learning and / or generation process of the rock sequence images.

[0138] Among them, the Multiscale-FF-GAN model is composed of GAN and FF, and the multi-scale relative error analyzer is determined by the relative error at different scales. The corresponding expressions are respectively:

[0139]

[0140] In the formula, X out represents the output feature image, X represents the multi-scale comprehensive feature image, X 1 represents the macroscopic feature image, X 2 represents the mesoscopic feature image, X 3 represents the microscopic feature image, G represents the global feature fusion function, L represents the local feature fusion function, BN represents the normalization operation function, represents the 1×1 convolution function, represents the convolution operation.

[0141] P, T are respectively the multi-scale relative error of the rock, the predicted value and the true value, where i = 1, 2, 3 respectively represent the macroscopic scale, the mesoscopic scale and the microscopic scale.

[0142] The running process of the constructed Multiscale-FF-GAN model is as Figure 5 shown.

[0143] Using the constructed Multiscale-FF-GAN model, image sequences of the rock structure at each scale can be generated.

[0144] Optionally, taking the macroscale, mesoscale, and microscale as examples, blank images corresponding to each scale are established. For example, the macroscale (physical size: 750mm×750mm×750mm, image size: 1000×1000), mesoscale (physical size: 10mm×10mm×10mm, image size: 500×500), and microscale (physical size: 1.5mm×1.50mm×1.5mm, image size: 300×300). The Multiscale-FF-GAN model is used to generate a sequence of structural images at the macroscale, a sequence of structural images at the mesoscale, and a sequence of structural images at the microscale in turn until all the structural image sequences reach the corresponding target scale, completing the generation process.

[0145] Specifically, as Figure 5 shown, taking the percentage content of the microscopic structure of the rock characterized by 1 - microscopic pores, 2 - microscopic cracks, and 3 - mineral phase aggregates and the characteristic classification vectorizer of the size information distribution of its sub-phases as the input, using the Multiscale-FF-GAN model, setting the 2D image size (300×300) required for the study, and generating a microscopic rock structure image of the set size.

[0146] Taking the percentage content of the mesoscopic structure of the rock characterized by 4 - mesoscopic pore phase and 5 - mesoscopic solid matrix phase (microscopic characteristic information cell unit) and the characteristic classification vectorizer of the size information distribution of its sub-phases as the input, using the Multiscale-FF-GAN model, setting the 2D image size (500×500) required for the study, and generating a mesoscopic rock structure image of the set size.

[0147] Taking the percentage content of the macroscopic structure of the rock characterized by 6 - macroscopic cracks and 7 - macroscopic solid matrix phase (mesoscopic - microscopic characteristic information cell unit) and the characteristic classification vectorizer of the size information distribution of its sub-phases as the input, using the Multiscale-FF-GAN model, setting the 2D image size (1000×1000) required for the study, and generating a macroscopic rock structure image of the set size.

[0148] In the method for multi-scale three-dimensional reconstruction of rocks based on deep learning of the present invention, the step of obtaining a multi-scale three-dimensional reconstruction model of rocks based on the structural image sequence of each scale specifically includes:

[0149] Forming three-dimensional voxel datasets of rocks at different scales in the three-dimensional voxel space based on the structural image sequence of each scale;

[0150] Performing visual rendering on the three-dimensional voxel datasets of rocks at different scales to obtain a multi-scale three-dimensional reconstruction model of rocks.

[0151] According to the research requirements, set the number of two-dimensional images in the sequence of rock structure images to be generated, such as 1000 images at the macroscopic scale, 500 images at the mesoscopic scale, and 300 images at the microscopic scale, and repeat the generation until the generation process of multiple images of the rock macro-meso-micro scale structure under the set target scale is completed.

[0152] Form the three-dimensional voxel datasets of rocks at different scales in the three-dimensional voxel space for the generated structure image sequences at each scale, and perform visual rendering on the three-dimensional voxel datasets of rocks at different scales to obtain a multi-scale three-dimensional reconstruction model of the rock.

[0153] In a specific implementation, taking the establishment of a macroscopic scale model as a characteristic cell, a 3D model of a slope with a length = 20m, a height = 8m, and a width = 5m is established using the Multiscale-FF-GAN model, as Figure 2 shown.

[0154] The following describes the deep learning-based rock multi-scale three-dimensional reconstruction device provided by the present invention. The deep learning-based rock multi-scale three-dimensional reconstruction device described below can be mutually referred to with the deep learning-based rock multi-scale three-dimensional reconstruction method described above.

[0155] As Figure 6 shown, the deep learning-based rock multi-scale three-dimensional reconstruction device includes an acquisition module 601, a training module 602, and a construction module 603:

[0156] The acquisition module 601 is used to acquire the rock geometric features and boundary features of the rock specimen at different scales;

[0157] In order to provide effective and reliable construction technical parameter indicators for the prevention and control of actual engineering geological disasters and deep-earth resource development projects based on the constructed rock three-dimensional model, rock samples are obtained from the target area and processed into rock specimens so that the structural features of the rock specimens can characterize the rock structural features of the target area.

[0158] Specifically, rock specimens of standard sizes are processed and prepared according to the research requirements. In this implementation, standard rock specimens with a length, width, and height of 100 millimeters are processed and prepared.

[0159] Furthermore, based on the prepared rock specimens, the rock geometric features and boundary features at different scales are acquired.

[0160] Among them, the different scales can be the macroscopic scale, the mesoscopic scale, and the microscopic scale, and the specific scale range can be determined according to the research requirements.

[0161] The geometric and boundary features of the rock characterize the geometric and boundary features of the rock structure of the rock specimen at each scale. It is understandable that different scales correspond to different rock structure features of the rock specimen.

[0162] For example, at the microscale, the rock structure features of the rock specimen include micro-pores, micro-cracks, and mineral phase aggregates. Therefore, the geometric and boundary features of the rock at the microscale are the geometric and boundary features of the micro-pores, micro-cracks, and mineral phase aggregates.

[0163] At the mesoscale, the rock structure features of the rock specimen include meso-pore phase and meso-solid matrix phase. Therefore, the geometric and boundary features of the rock at the mesoscale are the geometric and boundary features of the meso-pore phase and meso-solid matrix phase.

[0164] At the macroscale, the rock structure features of the rock specimen include macro-cracks and macro-solid matrix phase. Therefore, the geometric and boundary features of the rock at the macroscale are the geometric and boundary features of the macro-cracks and macro-solid matrix phase.

[0165] It is understandable that the geometric and boundary features of the rock specimen at different scales characterize the shape and distribution features of each sub-structure of the rock specimen at the corresponding scale.

[0166] The training module 602 is used to train a generative adversarial network based on the geometric and boundary features of the rock specimen at different scales, and generate a sequence of structural images for each scale in ascending order of scale based on the trained generative adversarial network;

[0167] Optionally, after vectorizing and characterizing the geometric and boundary features of the rock specimen at different scales, a dataset for training a generative adversarial network (GAN) is constructed, and the GAN is trained so that the trained GAN can generate two-dimensional structural images at the corresponding scale based on the geometric and boundary features of the input rock specimen at each scale.

[0168] Using the trained GAN, generation is repeated based on the geometric and boundary features of the rock specimen at each scale. When generating images, a sequence of structural images for each scale is generated in ascending order of scale.

[0169] The construction module 603 is used to obtain a multi-scale three-dimensional reconstruction model of the rock based on the sequence of structural images for each scale when the sequence of structural images for each generated scale reaches the corresponding target scale.

[0170] Optionally, the target scale is determined according to the target size and / or reconstruction accuracy of the three-dimensional reconstruction model to be reconstructed. The larger the target size and the higher the reconstruction accuracy, the larger the target scale, that is, the more images in the structural image sequence at each scale.

[0171] The structural image sequence at each scale generated by the GAN reaches the corresponding target scale, that is, the number of structural images generated at each scale can be used to render and generate the multi-scale three-dimensional reconstruction model of the rock. On this basis, the multi-scale three-dimensional reconstruction model of the rock can be reconstructed according to the obtained two-dimensional structural image sequence at each scale.

[0172] At this time, the reconstructed multi-scale three-dimensional reconstruction model of the rock can simultaneously represent the structural characteristics of the rock at each scale. In this embodiment, it can simultaneously represent the structural characteristics of the rock at the macroscopic, microscopic, and mesoscopic levels.

[0173] For example, after sampling the slope, a rock specimen corresponding to the slope is prepared, and the method in the present invention is used for three-dimensional reconstruction based on the rock geometric characteristics and boundary characteristics of the rock specimen, and the three-dimensional reconstruction model of the slope at the macroscopic scale as shown in (a) in Figure 2 is obtained; after magnifying it, the mesoscopic-scale three-dimensional slope wireframe model and local enlarged view can be obtained, as shown in (b) in Figure 2 ; further magnifying it, an enlarged view representing the mesoscopic structure of the slope can be obtained, as shown in (c) in Figure 2 .

[0174] The present invention generates structural image sequences at multiple different scales through the geometric characteristics and boundary characteristics of rock specimens, renders and obtains a multi-scale three-dimensional reconstruction model of the rock based on the structural image sequences at all scales, breaks through the scale limitation, realizes the association of macro-micro-mesoscopic rock structure feature information and three-dimensional model construction, realizes the autonomous classification reconstruction and high interpretability of multiple types of features of the rock at different scales, and provides a reliable real geometric model and construction design parameters for actual engineering research.

[0175] Figure 7 An example of a schematic physical structure diagram of an electronic device is shown in Figure 7As shown in the figure, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communications interface 720, and the memory 730 complete their mutual communication through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute a method for multi-scale three-dimensional reconstruction of rocks based on deep learning. The method includes: obtaining the rock geometric features and boundary features of a rock specimen at different scales; training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, and generating a structural image sequence for each scale in ascending order of scale based on the trained generative adversarial network; when the generated structural image sequences for each scale all reach the corresponding target scale, obtaining a multi-scale three-dimensional reconstruction model of the rock based on the structural image sequences for each scale.

[0176] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0177] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for multi-scale three-dimensional reconstruction of rocks based on deep learning provided by the above-mentioned various methods. The method includes: obtaining the rock geometric features and boundary features of a rock specimen at different scales; training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, and generating a structural image sequence for each scale in ascending order of scale based on the trained generative adversarial network; when the generated structural image sequences for each scale all reach the corresponding target scale, obtaining a multi-scale three-dimensional reconstruction model of the rock based on the structural image sequences for each scale.

[0178] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a rock multi-scale three-dimensional reconstruction method based on deep learning provided by the above-mentioned various methods. The method includes: obtaining the rock geometric features and boundary features of a rock specimen at different scales; training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, and sequentially generating a structural image sequence for each scale in ascending order of scale based on the trained generative adversarial network; and obtaining a rock multi-scale three-dimensional reconstruction model based on the structural image sequence for each scale when the structural image sequence for each generated scale reaches the corresponding target scale.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0180] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A rock multi-scale 3D reconstruction method based on deep learning, characterized in that: include: Obtain rock geometric characteristics and boundary characteristics of rock specimens at different scales; Based on the geometric features and boundary features of the rock specimen at different scales, a generative adversarial network is trained, and based on the trained generative adversarial network, a sequence of structural images of each scale is generated in order from small to large scales; When the generated structural image sequences at each scale reach the corresponding target scale, a multi-scale three-dimensional reconstruction model of rock is obtained based on the structural image sequences at each scale.

2. The rock multi-scale 3D reconstruction method based on deep learning according to claim 1, characterized in that: The step of obtaining rock geometric characteristics and boundary characteristics of rock specimens at different scales specifically includes: Respectively acquiring structural image sequences of the rock specimen at multiple different scales; Segmenting each image in the structural image sequence of the rock specimen at each scale to obtain a rock structural characteristic image at each scale; The rock structure characteristic images at each scale are scanned to obtain the geometric characteristics and boundary characteristics of the rock specimen at different scales.

3. The rock multi-scale 3D reconstruction method based on deep learning according to claim 2 is characterized in that: The step of segmenting each image in the structural image sequence of the rock specimen at each scale to obtain the rock structural characteristic image at each scale specifically includes: Inputting the structural image sequence of the rock specimen at each scale into a pre-trained deep learning segmentation model to obtain a rock structural feature image at each scale output by the deep learning segmentation model; The deep learning segmentation model is trained by using a U-shaped convolutional neural network as a basic framework and constructing a loss function based on the percentage loss, geometric loss and size loss of the rock specimen at each scale.

4. The rock multi-scale 3D reconstruction method based on deep learning according to claim 2, characterized in that: The step of segmenting each image in the structural image sequence of the rock specimen at each scale to obtain the rock structural characteristic image at each scale specifically includes: Segmenting each image in the structural image sequence of the rock specimen at each scale, and labeling the segmentation results to obtain a rock structural characteristic image with labeling at each scale; The step of scanning the rock structure characteristic image at each scale to obtain the geometric characteristics and boundary characteristics of the rock specimen at different scales specifically includes: Scanning the rock structure feature images at each scale using a pre-built feature extraction template to obtain the geometric features and boundary features of the rock specimen at different scales; Before the step of training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, the method further includes: Based on the scanning results, a feature classification vectorizer for each rock structure feature at a scale is constructed, and based on the feature classification vectorizer for each rock structure feature, a feature classification fusion device that characterizes the structure features of the rock sample at all scales is constructed.

5. The rock multi-scale 3D reconstruction method based on deep learning according to claim 4 is characterized in that: The step of training a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales specifically includes: Establishing a generator using the generative adversarial network based on the feature classification fusion device; The discriminator of the generative adversarial network is constructed based on the true value and predicted value of the rock sample at each scale.

6. The rock multi-scale 3D reconstruction method based on deep learning according to any one of claims 1 to 5, characterized in that: The step of obtaining a multi-scale three-dimensional reconstruction model of rock based on the structural image sequence of each scale specifically includes: Based on the structural image sequence of each scale, a three-dimensional voxel dataset of rocks at different scales is formed in a three-dimensional voxel space; The three-dimensional voxel data sets of different scales of the rock are visualized and rendered to obtain a multi-scale three-dimensional reconstruction model of the rock.

7. A rock multi-scale three-dimensional reconstruction device based on deep learning, characterized in that: include: An acquisition module is used to obtain rock geometric characteristics and boundary characteristics of rock specimens at different scales; A training module is used to train a generative adversarial network based on the geometric features and boundary features of the rock specimen at different scales, and to generate a structural image sequence of each scale in order from small to large scale based on the trained generative adversarial network; A construction module is used to obtain a multi-scale three-dimensional reconstruction model of rock based on the structural image sequence of each scale when the generated structural image sequence of each scale reaches the corresponding target scale.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-scale three-dimensional reconstruction method of rock based on deep learning as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-scale three-dimensional reconstruction method of rock based on deep learning as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-scale three-dimensional reconstruction method of rock based on deep learning as described in any one of claims 1 to 6 is implemented.

Images