Rock fib-sem serial image multi-phase registration segmentation method and system

Abstract

The application belongs to the technical field of image processing, and provides a rock FIB-SEM sequence image multi-phase registration segmentation method and system, a technical scheme of which is based on a constructed detail-preserving non-rigid registration subnetwork, and through affine correction and non-rigid registration, a registered image, a displacement vector field and a Jacobian determinant thereof are output; based on a constructed deformation-aware multi-phase segmentation subnetwork, the registered image is segmented to obtain a plurality of channel probability maps; in order to achieve registration accuracy and segmentation accuracy, the subnetworks are jointly trained by using a constructed loss function to obtain parameter-optimized subnetworks; and based on the parameter-optimized subnetworks, a target sequence image is processed to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned therewith. The method can simultaneously complete high-precision image registration and semantic segmentation while preserving high-frequency details and noise features with geological significance in the original image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and system for multi-phase registration and segmentation of rock FIB-SEM sequence images. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Focused ion beam scanning electron microscopy (FIB-SEM) technology can acquire a continuous two-dimensional cross-sectional image sequence at the nanoscale inside rock samples by alternately executing ion beam layer-by-layer cutting and electron beam high-resolution imaging. This allows for the establishment of high-fidelity digital core models to accurately quantify pore structure, mineral distribution, connectivity, and interface characteristics, playing an irreplaceable role in deep energy engineering such as oil and gas extraction, underground hydrogen storage, and carbon dioxide geological sequestration.

[0004] During the continuous slicing process of FIB-SEM equipment, which can last for hours or even days, samples are affected by factors such as ion beam bombardment, thermal drift, mechanical vibration, and differences in shrinkage rates among different mineral phases. This results in complex non-rigid deformations between adjacent slices. Direct stacking and reconstruction will introduce misalignment, tearing, or blurring artifacts. Existing traditional rigid / affine registration methods based on cross-correlation or phase correlation cannot effectively handle large local deformations. Although the deep learning unsupervised registration methods that have emerged in recent years can estimate dense deformation fields, their optimization objective is oriented towards maximizing global gray-level similarity and is supplemented by strong smoothing regularization. This can lead to excessive smoothing or even erasure of high-frequency components such as mineral grain boundaries, microcracks, and nanopore edges. This loss of key information, while macroscopically aligned, results in the loss of true texture, and the loss of detail directly weakens the realism of digital cores.

[0005] On the other hand, minerals and various pores in rock FIB-SEM images often exhibit characteristics such as gray-level overlap, diffuse boundaries, and low signal-to-noise ratio, making traditional segmentation methods (such as Otsu thresholding, watershed, and Canny edge detection) difficult to apply. The application of deep semantic segmentation networks is limited by the high cost of manual annotation and subjective differences. In addition, existing segmentation models assume that the input image is detailed and precisely aligned, and the segmentation and registration stages are separated. In reality, since high-frequency discriminative cues have been filtered out in the preprocessing step, the segmentation network will make decisions on images with poor information, further aggravating the accumulation of geometric distortion and semantic errors. Summary of the Invention

[0006] To address at least one of the technical problems mentioned above, this invention provides a method and system for multi-phase registration and segmentation of rock FIB-SEM sequence images. This method can simultaneously complete high-precision non-rigid image registration and pixel-level semantic segmentation of multiple micro-phases such as minerals and pores, while preserving the geologically significant high-frequency details and noise features in the original FIB-SEM image. This provides key technical support for constructing high-fidelity digital cores.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] The first aspect of the present invention provides a method for multi-phase registration and segmentation of rock FIB-SEM sequence images, comprising the following steps:

[0009] Data preprocessing was performed on the acquired rock FIB-SEM sequence images;

[0010] Affine correction is performed on the preprocessed floating image to obtain the affine-corrected floating image. The affine-corrected floating image and the reference image are then combined and non-rigidly registered through a detail-preserving non-rigid registration sub-network to obtain the registered image and its corresponding displacement vector field and Jacobian determinant.

[0011] The deformation-aware multi-phase segmentation sub-network constructed based on the displacement vector field and its Jacobian determinant is used to segment the registered image, resulting in probability maps of multiple channels, which correspond to various phases of the actual rock sample.

[0012] With registration accuracy and segmentation accuracy as objectives, the detail-preserving non-rigid registration subnetwork and the deformation-aware multi-phase segmentation subnetwork are jointly trained using the constructed loss function to obtain the subnetworks after parameter optimization.

[0013] Based on the parameter-optimized sub-networks, the target FIB-SEM sequence images are processed to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data.

[0014] Furthermore, affine correction is performed on the preprocessed floating image, including concatenating the single-channel floating image with the reference image by channel to obtain an input tensor, which is then input into a 3-layer convolutional layer to extract abstract features from edge texture to global deformation. The output feature map of the 3rd convolutional layer is flattened into a one-dimensional vector, and an affine transformation matrix is ​​constructed through a fully connected layer. Finally, the affine transformation matrix and the corresponding PyTorch function are combined to perform an affine transformation on the floating image based on the reference image to obtain the affine-corrected floating image.

[0015] Furthermore, the floating image after affine correction and the reference image are non-rigidly registered through a detail-preserving non-rigid registration sub-network, including:

[0016] Affine-corrected floating image and reference image The images are stitched together by channel and fed into the U-Net network, where an affine-corrected floating image is obtained through a decoder-encoder structure. The displacement vector field of pixel coordinates is used to generate an image through inverse mapping and bilinear interpolation, thus completing non-rigid registration and obtaining a non-rigid registered image. The Jacobian determinant is calculated based on the partial derivatives of the displacement vector field, reflecting local volume changes. Further, the deformation-sensing multi-phase segmentation sub-network constructed based on the displacement vector field and its Jacobian determinant segments the registered image, obtaining probability maps for multiple channels, including:

[0017] Feature extraction is performed on the registered images to obtain multi-scale main feature maps;

[0018] The displacement vector field and its Jacobian determinant output by the detail-preserving non-rigid registration subnetwork are concatenated into a tensor, which is then encoded to obtain a multi-scale deformation feature map of the same size as the multi-scale master feature map.

[0019] Multi-scale feature maps and deformation feature maps are concatenated one by one according to scale, and then CBAM attention weighting is introduced to obtain multi-scale fused feature maps.

[0020] The multi-scale fused feature map is mapped through 1×1 convolution channels and activated by Softmax to output a probability map with multiple channels.

[0021] Furthermore, the loss function is a weighted sum of the registration loss and the segmentation loss, expressed as:

[0022] ,

[0023] ,

[0024] ,

[0025] in, To register the loss weights, To register the subnetwork loss, The loss is the dynamic combination loss of the segmented subnetwork. , and This represents the weight coefficient of the corresponding item; The dynamic weights are adjusted based on the phase proportion and prediction accuracy. Represents image similarity loss. This represents the smoothing regularization term of the deformation field. This indicates a detail-preserving gradient loss. This indicates Dice's loss. This indicates Focal loss.

[0026] Furthermore, the registration subnetwork loss In the first term, image similarity loss is... The normalized cross-correlation measure is used to assess the structural alignment between the registered image and the reference image. The second term is the deformation field smoothing regularization term. The second derivative of the displacement vector field is constrained by bending energy. The third term is the detail-preserving gradient loss, which is defined as the L1 norm distance between the registered image and the reference image in the Sobel gradient domain.

[0027] Furthermore, the registration subnetwork loss The calculation also includes extracting phase boundary masks using Canny edge detection after the initial segmentation result is generated, and reintegrating them as soft constraints into the registration loss function to guide the deformation field to perform fine alignment in key boundary regions. A second aspect of the invention provides a multi-phase segmentation system for rock FIB-SEM sequence images, comprising:

[0028] The image preprocessing module is used to preprocess the acquired rock FIB-SEM sequence images.

[0029] The image registration module performs affine correction on the preprocessed floating image to obtain an affine-corrected floating image. The affine-corrected floating image and the reference image are then combined using a detail-preserving non-rigid registration sub-network for non-rigid registration, resulting in a registered image and its corresponding displacement vector field and Jacobian determinant. The image segmentation module segments the registered image based on the displacement vector field, its Jacobian determinant, and a deformation-aware multi-phase segmentation sub-network, obtaining probability maps for multiple channels, each corresponding to a different phase of the actual rock sample.

[0030] The joint training module is used to jointly train the detail-preserving non-rigid registration sub-network and the deformation-aware multi-phase segmentation sub-network with the goal of registration accuracy and segmentation accuracy, and obtain the sub-networks after parameter optimization.

[0031] The results output module is used to process the target FIB-SEM sequence image based on the parameter-optimized sub-networks to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data.

[0032] A third aspect of the present invention provides a computer-readable storage medium.

[0033] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described above.

[0034] A fourth aspect of the present invention provides a computer device.

[0035] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described above.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This invention achieves deep collaborative optimization of the two tasks by constructing a closed-loop feedback mechanism for registration and segmentation. By injecting the displacement vector field and its Jacobian determinant output by the registration sub-network as deformation-aware priors into the segmentation network and performing feature fusion at multi-scale jump connections, the segmentation model can perceive the spatial deformation history and local geometric state experienced by each pixel. This significantly enhances the ability to distinguish gray-scale overlapping and blurred boundary regions (such as the boundary between organic matter and micropores). It can simultaneously complete high-precision non-rigid image registration and pixel-level semantic segmentation of multiple micro-phases such as minerals and pores while preserving the high-frequency details and noise features with geological significance in the original FIB-SEM image.

[0038] This invention employs a dynamic combined segmentation loss strategy, adaptively adjusting the weights of Dice and Focal losses during training based on the class distribution and prediction confidence of the current batch of samples. This flexibly balances global class uniformity and local boundary accuracy at different training stages, alleviating the naturally occurring area imbalance between rock phases and enhancing the learning ability for difficult-to-classify samples, ensuring a simultaneous improvement in segmentation recall and boundary clarity for all phases.

[0039] This invention effectively avoids the loss of geological details caused by excessive smoothing by introducing a gradient consistency preservation term into the registration loss function. While achieving geometric alignment of the image, it fully preserves high-frequency texture and structural information with important geological interpretation value, such as mineral grain boundaries, microcracks, and nanopore edges, significantly improving the realism and scientific credibility of the reconstructed 3D digital core.

[0040] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0041] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0042] Figure 1 This is a flowchart of the multi-phase registration and segmentation method for rock FIB-SEM sequence images provided in this embodiment of the invention;

[0043] Figure 2 This is a schematic diagram of the registration sub-network provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the deformation-sensing multi-phase segmentation subnetwork provided in an embodiment of the present invention. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0047] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] Example 1

[0049] like Figure 1 As shown, this embodiment provides a method for multi-phase registration and segmentation of rock FIB-SEM sequence images, including the following steps:

[0050] Step 1: Preprocess the acquired rock FIB-SEM sequence images;

[0051] In this embodiment, the format of the acquired rock FIB-SEM sequence images is standardized to TIFF format to retain the original grayscale information. At the same time, the naming rules and image resolution are standardized to avoid compression distortion and inconsistent sizes.

[0052] Selecting the middle frame as the global reference image And make the corresponding labels, with the remaining images as floating images. ;

[0053] If outliers are present in a frame of an image, they need to be removed before normalization is performed.

[0054] To eliminate brightness differences between different frames and further avoid grayscale shift, image normalization is required. The formula is as follows:

[0055] ,

[0056] in, The original image is represented by pixel coordinates. grayscale value, , Represents the maximum and minimum pixel values ​​of the current frame, and represents the normalized value, mapped to the interval [0,1].

[0057] The model simulates the true deformation of FIB-SEM images and records the deformation parameters. Augmentation is applied only to the training set to diversify the training samples, thereby improving the model's robustness and preventing overfitting. The validation and test sets are not subjected to this process.

[0058] The processed image sequences are divided into three sets: training set, validation set, and test set. During this process, the continuity of FIB-SEM sequence slices must be considered to avoid data leakage and overlap. Furthermore, a unified storage format is required, and the original sequence numbers must be preserved for easy traceability.

[0059] Step 2: Based on the preprocessed floating image, the reference image, and the image segmentation model, obtain the initial image segmentation result;

[0060] The image segmentation model is trained based on the preprocessed floating image and the reference image to obtain the trained image segmentation network.

[0061] In this embodiment, the image segmentation model includes two highly coupled detail-preserving non-rigid registration subnetworks and a deformation-aware multi-phase segmentation subnetwork;

[0062] The construction of the image segmentation model specifically includes the following steps:

[0063] Step 201: Perform affine correction on the preprocessed floating image to obtain the affine-corrected floating image. Combine the affine-corrected floating image and the reference image to perform non-rigid registration using a detail-preserving non-rigid registration sub-network to obtain the registered image and its corresponding displacement vector field and Jacobian determinant; for example... Figure 2 As shown, the specific steps include the following:

[0064] Step 2011: Perform affine correction on the preprocessed floating image to obtain the affine-corrected floating image;

[0065] In this embodiment, a convolutional neural network is used for affine correction. The convolutional neural network includes three convolutional layers, one fully connected layer, and a no-pooling layer. The no-pooling layer is not a separate layer, but rather refers to the absence of pooling operations between the three convolutional layers.

[0066] The single-channel floating image is concatenated with the reference image channel by channel to obtain a 2×512×512 input tensor, which is then fed into three convolutional layers to extract abstract features from edge textures to global deformations. The final output feature map from the third convolutional layer has dimensions of 256×512×512. The flattened feature map is then transformed into a one-dimensional vector with dimensions of 256×512×512 = 67,108,864. An affine transformation matrix is ​​constructed using fully connected layers. The principle of affine transformation is as follows:

[0067] ,

[0068] in, These are the homogeneous coordinates of the original pixels. These are the homogeneous coordinates of the transformed pixels.

[0069] An affine transformation matrix consists of a linear transformation matrix and a translation vector, and can be expressed as:

[0070] ,

[0071] in, These are linear transformation parameters, corresponding to image rotation, scaling, and shearing transformations. Represents the translation parameter, representing the image in... x , y The translation direction, with the third row containing fixed values, serves to ensure the effectiveness of the homogeneous coordinate transformation, preventing any fundamental change in the two-dimensional coordinates. These two sets of parameters form a 6-dimensional affine parameter vector, which is then used as the output of the fully connected layer.

[0072] Finally, the corresponding PyTorch function (grid_sample) is used to perform a radiometric transformation on the floating image based on the reference image to obtain the corrected image. ).

[0073] Step 2012: Based on the affine-corrected floating image and the reference image, a dense two-dimensional displacement vector field is predicted. The floating image is then subjected to pixel-level elastic deformation correction based on the two-dimensional displacement vector field to obtain a non-rigid registration and assembly image.

[0074] In this embodiment, a deep network based on the U-Net structure is used as a reference image. and the floating image after affine correction As input, predict a dense two-dimensional displacement vector field;

[0075] The specific prediction process includes the following steps:

[0076] The corrected affine-corrected floating image Compared with reference image The data is concatenated by channel and fed into the U-Net network. Then, multi-scale features are extracted through a decoder-encoder structure, represented as follows:

[0077] ,

[0078] in, For pixel coordinates displacement vector field, For this pixel in x Displacement in the direction, For this pixel in y directional offset;

[0079] As a further implementation method, based on a two-dimensional displacement vector field U The corresponding function is called to perform deformation correction based on the displacement vector field. U Image generation through inverse mapping and bilinear interpolation Complete the non-rigid registration to obtain a non-rigid registered image;

[0080] To avoid pixel holes that may occur with forward mapping, this invention uses reverse mapping coordinate calculation, the formula of which is as follows:

[0081] ,

[0082] in, for Mid-pixel coordinates for The corresponding source coordinates in for Medium pixels Displacement in the direction, for Medium pixels directional offset;

[0083] For non-integer pixel coordinates, bilinear interpolation is used to calculate the pixel value, and the calculation formula is as follows:

[0084] ,

[0085] in, Integer coordinates Pixel value at that location, for , The surrounding four integer pixel coordinates are as follows:

[0086] ,

[0087] ,

[0088] ,

[0089] ,

[0090] in, The bilinear interpolation weights are calculated using the following formula:

[0091] ,

[0092] ,

[0093] ,

[0094] ,

[0095] ,

[0096] ,

[0097] When the source coordinates are not in the image range (height is...) , width is Within the range of ), a "mirror fill" strategy is used for boundary handling, the specific expression of which is:

[0098] ,

[0099] ,

[0100] ,

[0101] Finally based on partial derivatives Calculate the Jacobian determinant This reflects local volume changes. The calculation formula is shown below:

[0102] ,

[0103] Step 2013: Calculate the L1 norm distance between the non-rigidly registered assembly image and the reference image in the Sobel gradient domain;

[0104] In the detail preservation mechanism section, a Sobel gradient extraction module is added to the output of the registration sub-network. Specifically, it uses a Sobel convolution kernel to calculate the gradients of the registered images. Compared with reference image The gradient plots in the x and y directions, and compared with... , , They are fed into the loss function calculation module together to force the network to retain high-frequency features such as grain boundaries and pore edges during optimization.

[0105] The calculation formula is as follows:

[0106] Sobel kernel calculation formula:

[0107] x-direction:

[0108] ,

[0109] y direction:

[0110] ,

[0111] Image gradient calculation formula:

[0112] x-direction:

[0113] ,

[0114] y direction:

[0115] ,

[0116] Gradient loss formula (implementing high-frequency detail constraints):

[0117] This formula calculates the L1 norm distance between the non-rigidly registered image and the reference image in the Sobel gradient domain. , The image dimensions are all 512.

[0118] The registration process of this invention does not simply pursue grayscale similarity, but introduces a specially designed detail-preserving regularization term. This regularization term calculates the L1 norm distance between the registered image and the reference image in the Sobel gradient domain, forcing the network to prioritize the protection of high-frequency geological features such as edges, grain boundaries, microcracks and nanopores when optimizing the deformation field, thereby effectively avoiding the loss of valuable "noise" information caused by excessive smoothing.

[0119] Step 202: Based on the displacement vector field and its Jacobian determinant, the deformation-aware multi-phase segmentation sub-network is constructed to segment the registered image, obtaining probability maps of multiple channels, which correspond to various phases of the actual rock sample respectively;

[0120] like Figure 3 As shown, the specific steps include the following:

[0121] Step 2021: Input the registered image into a two-layer convolutional encoder to extract multi-scale main feature maps and corresponding multi-scale deformation feature maps;

[0122] In this embodiment, the two-layer convolutional encoder specifically includes a main feature extraction encoder and a deformation feature encoder.

[0123] The first part is the main feature extraction encoder, which can extract geological structural features and shares features with the established registration sub-network. Its specific structure is divided into shared layers and independent layers. Its input is the registered image, and its output is a multi-scale main feature map with 4 levels. The second part is the deformation feature encoding module. Its input is the displacement vector field output by the registration sub-network and the tensor formed by concatenating the Jacobian determinants. Its encoding layer consists of 2 layers of convolutional encoders and 4 layers of downsampling convolutions. Its output is a deformation feature map with 4 levels of the same size as the main feature map.

[0124] Specifically, the main processing module in the multi-scale master feature map extraction process consists of four levels of downsampling convolutional blocks. The first three levels share the encoder with the registration sub-network, and the parameters are updated synchronously. The fourth level is an independent convolutional block, each level containing: 3×3 convolution, BatchNorm, ReLU, and downsampling with a stride of 2. There are no pooling layers in the process, the number of channels doubles at each level, and the size is halved at each level, finally outputting a four-scale master feature map.

[0125] The main processing module in the deformation feature map extraction process consists of two basic convolutional layers and four levels of downsampling. First, two 3×3 convolutional layers with a stride of 1 and padding of 1 are used to map 3 channels to 64 channels. Then, four levels of downsampling convolutional blocks are used to achieve the size being halved step by step and the number of channels being adapted to the main feature. Finally, a 4-scale deformation feature map with the same size and number of channels as the main feature map is output.

[0126] Step 2022: Fuse the multi-scale feature map and the deformation feature map to obtain a multi-scale fused feature map. Based on the multi-scale fused feature map, obtain the probability maps of multiple channels, which correspond to various phases of the actual rock sample.

[0127] The output multi-scale main feature map and deformation feature map are concatenated one by one according to scale, and then CBAM attention weighting is introduced to obtain a multi-scale attention fusion feature map.

[0128] The bottleneck layer performs a global abstraction of the deepest features of the main feature encoder, forming a global abstract feature map. This map is then input into the decoder, passing through four decoder layers. Each decoder layer performs four processes sequentially: transposed convolutional upsampling, skip connection concatenation, CBAM attention fusion, and feature compression. The final output is a high-resolution fused feature map. It's important to note that the skip connection concatenation performs channel-wise concatenation between the upsampled feature map and the corresponding-scale attention fusion feature map generated in the previous step, supplementing details and deformation information at that scale.

[0129] The high-resolution fused feature map is processed through a 1×1 convolution operation to map the number of channels to K channels, which corresponds to the number of phase categories in the actual rock sample, generating a K-channel phase feature map. Then, softmax activation is performed on the channel dimension to output multiple channel probability maps. These maps can correspond to various phases in the actual rock sample, such as quartz minerals, clay minerals, organic matter, and pores, with the specific categories determined based on the actual rock sample.

[0130] This mechanism enables the segmentation network to perceive the spatial deformation history and local geometric state of each pixel during the decoding stage, significantly improving its ability to distinguish gray-scale overlapping areas (such as the boundary between organic matter and micropores). The segmentation sub-network ultimately outputs probability maps for multiple channels, corresponding to various phases in the actual rock sample, such as quartz minerals, clay minerals, organic matter, and pores, with the specific category determined based on the actual rock sample.

[0131] Step 3: Combine the registration accuracy and segmentation accuracy as objectives to construct a loss function, and jointly train the detail-preserving non-rigid registration sub-network and the deformation-aware multi-phase segmentation sub-network to obtain the parameters of the optimized sub-network;

[0132] In this embodiment, the entire network adopts an end-to-end joint training strategy, and its total loss function is composed of a weighted sum of registration loss and segmentation loss, expressed as:

[0133] ,

[0134] in, To register the loss weights, To register the subnetwork loss, The loss is the dynamic combination loss of the segmented subnetwork.

[0135] Among them, registration loss It contains three explicit items, represented as:

[0136] ,

[0137] in, , and This represents the weight coefficient of the corresponding item;

[0138] The first term is image similarity loss. Normalized cross-correlation (NCC) is used to measure the structural alignment between the registered image and the reference image, expressed as:

[0139] ,

[0140] in, , These represent the pixel coordinates of the registered image and the global reference image, respectively. grayscale value at that location , This represents the mean of a 3×3 local neighborhood, ensuring image structure alignment.

[0141] The second term is the deformation field smoothing regularization term. The second derivative of the displacement vector field is constrained by bending energy to prevent non-physical, severe distortion, as expressed in:

[0142] ,

[0143] in, For the Laplace operator, and These are the directional components of the displacement field in the x and y directions;

[0144] The third term is the detail-preserving gradient loss. , defined as the L1 norm distance between the registered image and the reference image in the Sobel gradient domain, is used to force the preservation of geologically significant high-frequency details such as mineral grain boundaries and micropore edges, and is expressed as:

[0145] ,

[0146] in, , These are image dimensions, all 512. Represents reference image Sobel gradient plot in the x-direction, Indicates the registered image Sobel gradient plot in the y-direction. Represents reference image Sobel gradient plot in the x-direction, Represents reference image Sobel gradient plot in the y-direction.

[0147] The dynamic segmentation loss, which is a dynamically weighted sum of the Dice loss and the Focal loss, is expressed as:

[0148] ,

[0149] in, The dynamic weights are adjusted according to the phase ratio and prediction accuracy to achieve adaptive optimization.

[0150] The Dice loss is used to mitigate the sample imbalance caused by significant area differences between different phase categories. It is calculated by subtracting the Dice coefficient between the predicted probability map and the true label, expressed as:

[0151] ,

[0152] in, The category of phase. These are genuine labels marked by humans. To predict the probability map, .

[0153] Focal loss is used to enhance the model's ability to learn difficult-to-classify samples (such as gray-level overlapping regions or blurred boundaries). It reduces the weight of easily classified samples by introducing a modulation factor into the standard cross-entropy loss, making the network pay more attention to pixels that are difficult to classify. It is expressed as:

[0154] ,

[0155] in, To balance the weights for each category, For focusing parameters.

[0156] The joint training process can be summarized as "forward propagation - loss calculation - backpropagation - parameter update - loop feedback". The training set is input into the previously constructed joint network, and the result is a non-rigidly registered output image. Displacement vector field U Jacobian determinant The probability distribution plot and gradient plot are used to implement forward propagation. Based on the results of forward propagation, the weighted total loss is calculated. .

[0157] The weighting coefficients of the two losses are not fixed values, but are dynamically adjusted during training based on the distribution of various phases in the current batch and the prediction confidence. When a certain phase accounts for a small proportion in the current batch or the prediction accuracy is low, the system automatically increases the weight of the Dice loss to strengthen the recall ability of that category; conversely, the weight of the Focal loss is appropriately increased to optimize boundary accuracy. The hyperparameters within the Focal loss are fixed to ensure effective focusing on difficult examples. Through this dynamic weighting mechanism, the segmentation loss can adaptively balance global class balance and local boundary accuracy at different training stages, thereby achieving stable and high-precision multi-phase segmentation.

[0158] To further improve collaborative performance, this embodiment introduces a cyclic consistency feedback mechanism: after the initial segmentation result is generated, the system can extract the phase boundary mask through Canny edge detection and reintegrate it as a soft constraint into the registration loss function, especially strengthening the alignment accuracy of the boundary region and preventing misalignment or adhesion across phases.

[0159] The updated registration loss formula is as follows:

[0160] ,

[0161] in, The weights of the boundary-aligned loss term are expressed as follows:

[0162] ,

[0163] in, For boundary pixels, Register the offset distance for the boundary pixels.

[0164] By extracting phase boundary masks from the preliminary segmentation results and reintegrating them as soft constraints into the registration process, the deformation field is guided to be finely aligned in the key boundary region, thus forming a positive reinforcement cycle of "registration preserves details - segmentation is more accurate - boundary guides re-registration", which effectively suppresses the problem of error accumulation and amplification in serial processing.

[0165] This process can be executed iteratively until the registration and segmentation results reach a stable optimum. It is worth noting that even in real-world application scenarios lacking manually labeled data, the registration subnetwork can still operate in a completely unsupervised manner, while the segmentation subnetwork can be initialized through self-supervised pre-training, transfer learning, or using a small number of weak labels, and then gradually optimized with the aforementioned feedback mechanism, thus possessing good practicality and generalization ability.

[0166] Step 4: Based on the optimized parameters of each sub-network, process the target FIB-SEM sequence image to obtain the 3D image volume data and the aligned multi-phase semantic segmentation 3D labels;

[0167] During the inference phase, the system sequentially executes the aforementioned joint processing flow on the entire FIB-SEM sequence, outputting two sets of core results: first, 3D image volume data that has undergone high-precision non-rigid registration while retaining original details; and second, 3D tag volumes with strictly aligned multi-phase semantic segmentation. Based on this, key reservoir parameters such as the volume fraction of various phases, pore connectivity, specific surface area, and throat distribution can be automatically calculated, providing a high-fidelity, high-precision data foundation for quantitative analysis of the microstructure of tight rocks such as shale and digital core modeling.

[0168] As a further embodiment, based on the output results, two different types of metrics are introduced for verification:

[0169] (1) Registration effect

[0170] Root Mean Square Error (RMSE):

[0171] ,

[0172] Structural Similarity (SSIM):

[0173] ,

[0174] in, , It is a constant. Standard deviation, For covariance.

[0175] (2) Segmentation effect

[0176] Intersection over Union (IoU):

[0177] ,

[0178] in, Representing category C ,average For all categories The mean, Represents a logical OR operation.

[0179] Dice coefficient:

[0180] ,

[0181] This coefficient can quantify segment recall.

[0182] The registration accuracy, detail preservation, and segmentation accuracy of the invention are further compared and verified based on the calculation results of the above coefficients.

[0183] The registration module of this invention is completely unsupervised, and the segmentation module supports transfer learning and weakly supervised initialization. The overall framework has low dependence on manual annotation. Therefore, it has good practicality and generalization ability, and can run stably in real-world engineering scenarios lacking a large number of pixel-level labels, significantly reducing the technical threshold and application cost of intelligent processing of high-precision FIB-SEM images.

[0184] Example 2

[0185] This embodiment provides a multi-phase registration and segmentation system for rock FIB-SEM sequence images, including:

[0186] The image preprocessing module is used to preprocess the acquired rock FIB-SEM sequence images.

[0187] The image registration module is used to output the registered image, displacement vector field and its Jacobian determinant based on the preprocessed floating image and reference image, and the constructed detail-preserving non-rigid registration sub-network, through affine correction and non-rigid registration.

[0188] The image segmentation module is used to segment the registered image based on the displacement vector field and its Jacobian determinant and the constructed deformation-aware multi-phase segmentation sub-network to obtain probability maps of multiple channels, which correspond to various phases of the actual rock sample.

[0189] The joint training module is used to jointly train the detail-preserving non-rigid registration sub-network and the deformation-aware multi-phase segmentation sub-network with the goal of registration accuracy and segmentation accuracy, and obtain the sub-networks after parameter optimization.

[0190] The results output module is used to process the target FIB-SEM sequence image based on the parameter-optimized sub-networks to obtain three-dimensional image volume data and aligned multi-phase semantic segmentation three-dimensional labels.

[0191] It should be noted that the specific implementation of the multi-phase segmentation system for rock FIB-SEM sequence images in this embodiment of the invention is similar to the specific implementation of the multi-phase registration and segmentation method for rock FIB-SEM sequence images in this embodiment of the invention. For details, please refer to the description in the method section. To reduce redundancy, it will not be repeated here.

[0192] Example 3

[0193] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described above.

[0194] Example 4

[0195] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described above.

[0196] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0197] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0198] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0200] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0201] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for multi-phase registration and segmentation of rock FIB-SEM sequence images, characterized in that, Includes the following steps: Data preprocessing was performed on the acquired rock FIB-SEM sequence images; Affine correction is performed on the preprocessed floating image to obtain the affine-corrected floating image. The affine-corrected floating image and the reference image are then combined and non-rigidly registered through a detail-preserving non-rigid registration sub-network to obtain the registered image and its corresponding displacement vector field and Jacobian determinant. The floating image after affine correction and the reference image are non-rigidly registered through a detail-preserving non-rigid registration sub-network, including: Affine-corrected floating image and reference image The images are stitched together by channel and fed into the U-Net network, where an affine-corrected floating image is obtained through a decoder-encoder structure. The displacement vector field of pixel coordinates is used to generate an image through inverse mapping and bilinear interpolation, thus completing non-rigid registration and obtaining a non-rigid registered image. The Jacobian determinant is calculated based on the partial derivatives of the displacement vector field, reflecting local volume changes. Detail preservation involves introducing a Sobel gradient extraction module at the output of the registration sub-network to calculate the non-rigidly registered image. Compared with reference image L1 norm distance over the Sobel gradient domain; The deformation-aware multi-phase segmentation sub-network constructed based on the displacement vector field and its Jacobian determinant is used to segment the registered image, resulting in probability maps of multiple channels, which correspond to various phases of the actual rock sample. The deformation-aware multi-phase segmentation sub-network constructed based on the displacement vector field and its Jacobian determinant segments the registered image, obtaining probability maps for multiple channels, including: Feature extraction is performed on the registered images to obtain multi-scale main feature maps; The displacement vector field and its Jacobian determinant output by the detail-preserving non-rigid registration subnetwork are concatenated into a tensor, which is then encoded to obtain a multi-scale deformation feature map of the same size as the multi-scale master feature map. Multi-scale feature maps and deformation feature maps are concatenated one by one according to scale, and then CBAM attention weighting is introduced to obtain multi-scale fused feature maps. The multi-scale fused feature map is mapped through 1×1 convolution channels and activated by Softmax to output a probability map with multiple channels; With registration accuracy and segmentation accuracy as objectives, the detail-preserving non-rigid registration subnetwork and the deformation-aware multi-phase segmentation subnetwork are jointly trained using the constructed loss function to obtain the subnetworks after parameter optimization. The loss function consists of a weighted average of the registration loss and the segmentation loss, expressed as: ， ， ， in, To register the loss weights, To register the subnetwork loss, The loss is the dynamic combination loss of the segmented subnetwork. , and This represents the weight coefficient of the corresponding item; The dynamic weights are adjusted based on the phase proportion and prediction accuracy. Represents image similarity loss. This represents the smoothing regularization term of the deformation field. This indicates a detail-preserving gradient loss. This indicates Dice's loss. Indicates Focal loss; Registration subnetwork loss In the first term, image similarity loss is... The normalized cross-correlation measure is used to assess the structural alignment between the registered image and the reference image. The second term is the deformation field smoothing regularization term. The second derivative of the displacement vector field is constrained by bending energy. The third term is the detail-preserving gradient loss, which is defined as the L1 norm distance between the registered image and the reference image in the Sobel gradient domain. Based on the parameter-optimized sub-networks, the target FIB-SEM sequence images are processed to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data.

2. The method for multi-phase registration and segmentation of rock FIB-SEM sequence images as described in claim 1, characterized in that, Affine correction is performed on the preprocessed floating image, which involves concatenating the single-channel floating image with the reference image channel by channel to obtain the input tensor, which is then fed into three convolutional layers to extract abstract features from edge textures to global deformation. The third convolutional layer outputs a feature map, which is then flattened into a one-dimensional vector. An affine transformation matrix is ​​constructed through a fully connected layer. Finally, the affine transformation matrix and the corresponding PyTorch function are combined to perform an affine transformation on the floating image based on the reference image, resulting in the affine-corrected floating image.

3. The method for multi-phase registration and segmentation of rock FIB-SEM sequence images as described in claim 1, characterized in that, Registration subnetwork loss The calculation also includes extracting phase boundary masks through Canny edge detection after the initial segmentation results are generated, and reintegrating them as soft constraints into the registration loss function to guide the deformation field to perform fine alignment in the key boundary regions.

4. A multi-phase registration and segmentation system for rock FIB-SEM sequence images, employing the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described in any one of claims 1-3, characterized in that, include: The image preprocessing module is used to preprocess the acquired rock FIB-SEM sequence images. The image registration module performs affine correction on the preprocessed floating image to obtain an affine-corrected floating image. The affine-corrected floating image and the reference image are then combined using a detail-preserving non-rigid registration sub-network for non-rigid registration, resulting in a registered image and its corresponding displacement vector field and Jacobian determinant. The image segmentation module segments the registered image based on the displacement vector field, its Jacobian determinant, and a deformation-aware multi-phase segmentation sub-network, obtaining probability maps for multiple channels, each corresponding to a different phase of the actual rock sample. The joint training module is used to jointly train the detail-preserving non-rigid registration sub-network and the deformation-aware multi-phase segmentation sub-network with the goal of registration accuracy and segmentation accuracy, and obtain the sub-networks after parameter optimization. The results output module is used to process the target FIB-SEM sequence image based on the parameter-optimized sub-networks to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described in any one of claims 1-3.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the multi-phase registration and segmentation method for rock FIB-SEM sequence images as described in any one of claims 1-3.

Images