A cross-device stylized enhancement method and system for cement hydration microstructure images

Abstract

The application discloses a cement hydration microstructure image cross-device stylized enhancement method and system, and belongs to the technical field of digital image processing. The method comprises the following steps: acquiring a first cement hydration microstructure image and a second cement hydration microstructure image, wherein the first cement hydration microstructure image is collected by a CT device with low configuration, and the second cement hydration microstructure image is collected by a CT device with high configuration; inputting the first cement hydration microstructure image and the second cement hydration microstructure image into a cross-device image enhancement model for processing, calculating feature position coding, feature extraction, fusion and up-sampling restoration of the two images through an input layer, a representation layer and an output layer, and finally obtaining a cement hydration microstructure image after cross-device stylized enhancement. The cement hydration microstructure image with imaging quality close to that of the CT device with high configuration can be obtained, and the problem of cement hydration microstructure image cross-device enhancement is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital image processing, in particular to a cement hydration microstructure image cross-device stylized enhancement method and system. BACKGROUND

[0002] The statements in this section merely refer to the background of the present application and do not necessarily constitute the prior art.

[0003] The microstructure of cement can be characterized in various ways, and only a micro-CT scanner can obtain a three-dimensional CT image of the cement hydration microstructure without damaging the physical structure of the cement block itself. Research based on three-dimensional CT images can help researchers observe the changes of various substances in the cement hydration process intuitively and fundamentally understand the cement hydration mechanism.

[0004] The quality of CT images is affected by the advancement of CT devices. Three-dimensional high-quality images obtained by high-configuration CT devices can provide more reliable and scientific data support for cement hydration research. High-configuration CT devices use more advanced imaging accessories, reconstruction algorithms, etc., to obtain higher quality CT images; however, the cost of such high-configuration CT devices, whether in terms of equipment cost or imaging cost, is much higher than that of low-configuration CT devices. For cement hydration research, the equipment available in the cement plant laboratory is usually low-configuration, and the quality of the obtained CT images needs to be improved.

[0005] Traditional methods of using artificial intelligence technology for CT image enhancement, such as image denoising and image super-resolution, can enhance CT images to some extent, but cannot achieve cross-device image enhancement. Image denoising is usually used to simulate the image quality enhancement from low-dose to high-dose scanning by the same CT device, which belongs to the enhancement within the same device image domain. Similarly, image super-resolution only focuses on expanding the granularity of CT images from the perspective of resolution, and cannot achieve enhancement between image domains from low-configuration devices to high-configuration devices.

[0006] Image enhancement based on stylization proposed in recent years can learn the style of the target image domain and convert the style of the source image domain to the style of the target image domain, but is more commonly applied to image artification and has not achieved cross-device stylized enhancement. At the same time, three-dimensional cement CT images are typical homogeneous images, and during the process of cross-device image stylized enhancement, the size, shape, and position of cement particles may change, details and textures may be lost, and patterns may collapse during training, making normal training impossible. SUMMARY

[0007] In order to solve the problems in the prior art, the present application provides a cement hydration microstructure image cross-device stylized enhancement method, system, electronic device, computer storage medium and computer program product, which obtains physical imaging quality close to that of a high-configuration CT device through artificial intelligence assisted computational imaging, realizes cross-device image enhancement, improves the usability of a low-configuration CT device, and greatly reduces imaging cost.

[0008] In a first aspect, the present application provides a cement hydration microstructure image cross-device stylized enhancement method.

[0009] A cement hydration microstructure image cross-device stylized enhancement method comprises the following steps.

[0010] Obtaining a first cement hydration microstructure image and a second cement hydration microstructure image, wherein the first cement hydration microstructure image and the second cement hydration microstructure image are collected by different CT devices.

[0011] Inputting the first cement hydration microstructure image and the second cement hydration microstructure image into a trained cross-device image enhancement model for processing to obtain a cement hydration microstructure image after cross-device stylized enhancement.

[0012] The cross-device image enhancement model encodes the first cement hydration microstructure image and the second cement hydration microstructure image through an input layer, respectively, generates corresponding skeleton position encodings and adds them to the first cement hydration microstructure image, and generates corresponding style position encodings and adds them to the second cement hydration microstructure image.

[0013] In some embodiments, the encoding of the first cement hydration microstructure image through the input layer to generate corresponding skeleton position encodings and add them to the first cement hydration microstructure image comprises:

[0014] Converting the first cement hydration microstructure image into a first image feature vector through an embedding layer, performing maximum pooling and average pooling on the first image feature vector in parallel, and performing convolution processing on the difference between the maximum pooling processing result and the average pooling processing result.

[0015] Restoring the size of the first image feature vector after convolution processing through an interpolation algorithm to generate skeleton position encodings and add them to the first image feature vector.

[0016] In some embodiments, the encoding of the second cement hydration microstructure image through the input layer to generate style position encodings and add them to the second cement hydration microstructure image comprises:

[0017] convert the second cement hydration microstructure image into a second image feature vector through an embedding layer;

[0018] perform convolution processing on the second image feature vector, generate style position encoding and add to the second image feature vector.

[0019] In some embodiments, the cross-device image enhancement model comprises an input layer, a representation layer and an output layer connected in sequence, the input layer comprises an embedding layer and a position encoding layer connected in sequence, the representation layer comprises a Transformer encoder module and a Transformer decoder module connected in sequence, and the output layer comprises an up-sampling module.

[0020] Wherein, the number of Transformer encoder modules is multiple and is arranged in parallel.

[0021] In some embodiments, the Transformer encoder modules are respectively used for skeleton information extraction on the first image feature vector added with skeleton position encoding and style information extraction on the second image feature vector added with style position encoding, to obtain corresponding skeleton information features and style information features.

[0022] The Transformer decoder module is used for decoding and fusing the skeleton information features and the style information features to generate a fusion feature vector.

[0023] The up-sampling module is used for decoding the fusion feature vector to convert the fusion feature vector into a cross-device stylized and enhanced cement hydration microstructure image.

[0024] In some embodiments, the cross-device image enhancement model is pre-trained using non-homogeneous images by transfer learning, and is trained for the purpose of minimizing skeleton loss, style loss and double-layer cycle consistency loss.

[0025] In a second aspect, the present application provides a cross-device stylized enhancement system for cement hydration microstructure images.

[0026] A cross-device stylized enhancement system for cement hydration microstructure images comprises:

[0027] An acquisition module is configured to acquire a first cement hydration microstructure image and a second cement hydration microstructure image, wherein the first cement hydration microstructure image and the second cement hydration microstructure image are collected by different CT devices.

[0028] The cross-device stylized enhancement module is configured to input the first cement hydration microstructure image and the second cement hydration microstructure image into a trained cross-device image enhancement model for processing to obtain a cross-device stylized enhanced cement hydration microstructure image.

[0029] The cross-device image enhancement model encodes the first cement hydration microstructure image and the second cement hydration microstructure image through an input layer to generate corresponding skeleton position encodings and add them to the first cement hydration microstructure image, and generate corresponding style position encodings and add them to the second cement hydration microstructure image.

[0030] In a third aspect, the present application provides an electronic device.

[0031] An electronic device includes a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the cross-device stylized enhancement method of the cement hydration microstructure image.

[0032] In a fourth aspect, the present application provides a computer readable storage medium.

[0033] A computer readable storage medium has a computer program / instruction stored thereon, and the computer program / instruction is executed by a processor to implement the steps of the cross-device stylized enhancement method of the cement hydration microstructure image.

[0034] In a fifth aspect, the present application provides a computer program product.

[0035] A computer program product includes a computer program / instruction, and the computer program / instruction is executed by a processor to implement the steps of the cross-device stylized enhancement method of the cement hydration microstructure image.

[0036] Compared with the prior art, the present application has the following advantages:

[0037] 1. The technical solution provided by the present application proposes a cross-device enhancement idea for cement hydration microstructure images, describes the difference in image quality between a cement hydration microstructure image imaged by a high-configuration CT device and a cement hydration microstructure image imaged by a low-configuration CT device as a difference in style, and realizes cross-device enhancement of cement hydration microstructure images by stylizing the cement hydration microstructure image imaged by the low-configuration CT device.

[0038] 2. The technical scheme provided by the application, a cross-device image enhancement model based on a Transformer is designed, a new position encoding calculation method is proposed for homogeneous images such as cement hydration microstructure images, and skeleton position encoding and style position encoding are obtained by processing according to the characteristics of skeleton features and style features; two parallel Transformer encoder modules are used to extract the style of cement hydration microstructure images collected by high-configuration CT devices and extract the skeleton of cement hydration microstructure images collected by low-configuration CT devices; a Transformer decoder module is used to fuse the extracted style and skeleton information; and an up-sampling module is used to generate the fused features into cement hydration microstructure images with low-configuration CT device skeleton information and high-configuration CT device style information, thereby realizing cross-device stylization enhancement of cement hydration microstructure images.

[0039] 3. The technical scheme provided by the application, in order to solve the problem of unstable training of the cross-device image enhancement model on homogeneous images such as cement, the cross-device image enhancement model is pre-trained on non-homogeneous images, and the parameters are used as the initialization parameters of the model to stabilize the training of the model on homogeneous images such as cement hydration microstructure images.

[0040] 4. The technical scheme provided by the application, in order to better maintain the skeleton of the original image unchanged, a double-layer cyclic consistency architecture is used to train the cross-device image enhancement model. BRIEF DESCRIPTION OF DRAWINGS

[0041] The drawings accompanying the specification of the application form part of the application and serve to provide a further understanding of the application, the illustrative embodiments of the application and their description serve to explain the application without constituting an improper limitation of the application.

[0042] Figure 1 The flowchart of the cross-device stylization enhancement method of the cement hydration microstructure image provided by the embodiment of the application is shown in the figure.

[0043] Figure 2 The training flowchart of the cross-device image enhancement model provided by the embodiment of the application is shown in the figure.

[0044] Figure 3 The flowchart of the cross-device image enhancement model training based on double-layer cyclic consistency loss provided by the embodiment of the application is shown in the figure.

[0045] Figure 4 The test result example provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0046] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0047] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0048] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0049] Embodiment one

[0050] The three-dimensional high-quality images formed by high-configuration CT devices can provide more reliable and scientific data support for cement hydration research, but the CT devices in the current cement plant laboratory are usually low-configuration. In the traditional method of using artificial intelligence technology for CT image enhancement, only stylized enhancement can learn the style of the target image domain, but more is applied to the artistic image, and cross-device stylized enhancement is not realized; and due to the complex structure and homogeneity characteristics of cement hydration microstructure images, it is difficult to ensure that the skeleton of the original image remains unchanged during the cross-device stylized enhancement process, and it is easy to fall into pattern collapse.

[0051] Therefore, the present application provides a cross-device stylized enhancement method for cement hydration microstructure images, which describes the difference in image quality between high-configuration CT devices and low-configuration CT devices as a style difference, and uses a method of stylizing cement images imaged by low-configuration CT devices to realize cross-device level image enhancement.

[0052] Next, combined with Figures 1-4 A cross-device stylized enhancement method for cement hydration microstructure images disclosed in the present embodiment will be described in detail. The cross-device stylized enhancement method for cement hydration microstructure images comprises the following steps:

[0053] S1, obtaining a first cement hydration microstructure image and a second cement hydration microstructure image.

[0054] In this embodiment, the first cement hydration microstructure image is a cement hydration microstructure image collected by a low-configuration CT device, and the second cement hydration microstructure image is a cement hydration microstructure image collected by a high-configuration CT device. The image quality of the first cement hydration microstructure image is poorer than that of the second cement hydration microstructure image.

[0055] The image quality of the cement hydration microstructure image collected by the low-configuration CT device is poorer than that of the cement hydration microstructure image collected by the high-configuration CT device. The difference between the first cement hydration microstructure image quality and the second cement hydration microstructure image quality is described as a difference in image style. Specifically, there are differences in imaging accessories, reconstruction algorithms, stability of the X-ray source, etc. of different CT devices, resulting in differences in imaging characteristics, gray scale performance, and detail presentation methods, which give the images unique style characteristics.

[0056] S2, input the first cement hydration microstructure image and the second cement hydration microstructure image into the trained cross-device image enhancement model for processing to obtain a cross-device stylized and enhanced cement hydration microstructure image.

[0057] Further, the cross-device image enhancement model includes an input layer, a representation layer, and an output layer connected in sequence. The input layer includes an embedding layer and a position encoding layer connected in sequence. The representation layer includes a Transformer encoder module and a Transformer decoder module connected in sequence. The output layer includes three upsampling modules and an output unit connected in sequence. The number of Transformer encoder modules is 2 and is arranged in parallel. The number of Transformer decoder modules is 1. The Transformer encoder module includes three Transformer encoder layers connected in sequence. The Transformer decoder module includes three Transformer decoder layers. The upsampling module includes a padding layer, a convolution layer, an activation layer, and an upsampling layer connected in sequence. The output unit includes a padding layer, a convolution layer, and an activation layer connected in sequence.

[0058] As an implementation manner, S2 specifically includes:

[0059] S201, position encode the first cement hydration microstructure image and the second cement hydration microstructure image through the input layer to generate corresponding skeleton position encodings and add them to the first cement hydration microstructure image, and generate corresponding style position encodings and add them to the second cement hydration microstructure image.

[0060] Under the premise of processing cement hydration microstructure images with complex structure and homogeneity, it is necessary to approach the imaging style of the high-configuration CT device as much as possible, and at the same time, it is necessary to ensure that the skeleton of the original image remains unchanged.

[0061] The skeleton, as reflected in an image, refers to the boundaries, contours, and other features of each element in the image. When the original image undergoes rigid transformations such as rotation, flipping, or translation, or when the grayscale values ​​of the original image are transformed linearly with a slope of 1 or -1, such as inverting the grayscale or adding a specific value to all grayscale values, the skeleton information at the corresponding positions should remain unchanged.

[0062] The style is more hidden in the colors of the image itself, and is expressed as grayscale values ​​in the first and second cement hydration microstructure images.

[0063] In order to convert grayscale images into feature vectors and perform positional encoding on the feature vectors, this embodiment constructs an input layer in a cross-device image enhancement model. The input layer includes an embedding layer, a skeleton positional encoding, and a style positional encoding. The embedding layer is a linear layer containing a hidden layer.

[0064] To better extract skeleton and style information from homogeneous images such as cement hydration microstructure images, and to minimize the impact of noise in these images, the specific process in S201 is as follows:

[0065] S2011. The first cement hydration microstructure image is converted into a first image feature vector through the embedding layer. Max pooling and average pooling are performed in parallel on the first image feature vector. The difference between the max pooling result and the average pooling result is convolved. The size of the first image feature vector after convolution is restored by an interpolation algorithm, and a skeleton position code is generated and added to the first image feature vector.

[0066] The first image feature vector is represented as:

[0067] ;

[0068] ;

[0069] Skeleton position encoding is represented as:

[0070] ;

[0071] The first image feature vector with added skeleton position encoding is represented as:

[0072] ;

[0073] In the formula, Indicates the parameters of the embedding layer. The image represents the first cement hydration microstructure segmentation into the second... There are n image blocks in total; ,in, ; Indicates the skeleton position code. c Indicates the first subscript, This represents an interpolation algorithm that restores the first image feature vector after convolution to its initial size. express convolution, This indicates a max pooling operation. This indicates the average pooling operation. This indicates the size after the pooling operation.

[0074] Here, before inputting the embedding layer, the first cement hydration microstructure image and the second cement hydration microstructure image are segmented into image blocks of the same size, and the image blocks are sent into the embedding layer.

[0075] S2012. The second cement hydration microstructure image is converted into a second image feature vector through an embedding layer; the second image feature vector is convolved to generate a style location code and added to the second image feature vector.

[0076] The second image feature vector is represented as follows:

[0077] ;

[0078] ;

[0079] Style location coding is represented as:

[0080] ;

[0081] The second image feature vector with added stylized positional encoding is represented as follows:

[0082] ;

[0083] In the formula, The image represents the second cement hydration microstructure segmentation into the first... There are n image blocks in total. ,in, ; Indicates style location code, s Indicates the second subscript. Indicates a property with one boundary extension. The convolution.

[0084] Skeleton location encoding and style location encoding can help the multi-head attention mechanism in the representation layer better preserve the dependencies between neighboring regions, making cross-device image augmentation models more accurate in recognizing image edges and textures.

[0085] S202, processing the first image feature vector added with the skeleton position coding and the second image feature vector added with the style position coding through the presentation layer to generate a fusion feature vector.

[0086] The cement hydration microstructure image is a typical homogeneous image. From a macroscopic point of view, there are a large number of unhydrated cement particles, cement hydration products, bubbles and other contents in each local part. However, from a microscopic point of view, the size, shape and structure formed in the hydration process of each substance are different, and the whole has high complexity. Therefore, in the process of cement hydration microstructure image stylization, problems such as changes in the size, shape and position of cement particles and loss of detail texture may occur.

[0087] In order to extract and fuse the skeleton information and style information in the cement hydration microstructure image, specifically, S202 includes:

[0088] S2021, input the first image feature vector added with the skeleton position coding and the second image feature vector added with the style position coding into the parallelly arranged Transformer encoder modules, extract the skeleton information from the first image feature vector added with the skeleton position coding through one Transformer encoder module to obtain skeleton information features, and extract the style information from the second image feature vector added with the style position coding through another Transformer encoder module to obtain style information features.

[0089] S2022, decode and fuse the skeleton information features and the style information features through the Transformer decoder module to generate a fusion feature vector.

[0090] Exemplarily, the multi-head attention mechanism calculation method in the Transformer encoder layer and the Transformer decoder layer is as follows:

[0091] ;

[0092] ;

[0093] In the formula, respectively represent the query vector, the key vector and the value vector when calculating attention, is the dimension of the query vector, k represents the third subscript, represents arranging into a matrix in order, represents the matrix of linear transformation, O represents the fourth subscript, and N represents the number of attention mechanism calculations, that is, the number of heads in the multi-head attention mechanism.

[0094] For example, the data processing flow of the Transformer encoder layer is as follows:

[0095]

[0096] ;

[0097] ;

[0098] ;

[0099] In the formula, in the first layer, Represented as the first image feature vector or the second image feature vector, in the second and third layers, pos represents the output vector of the previous layer, and pos represents the positional encoding. When extracting the skeleton information of the first image feature vector, pos is the skeleton positional encoding. When extracting the style information of the second image feature vector, pos is the style positional encoding. These are the generation steps performed when the attention mechanism is computed for the i-th time on a vector. The required training linear transformation matrix is ​​FFN, which is a linear layer with two hidden layers, interspersed with ReLU activation functions, and LN represents layer normalization. This is the output of a Transformer encoder layer.

[0100] For example, the data processing flow of the Transformer decoder layer is as follows:

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] In the first layer... This represents the skeleton information features output by the first Transformer encoder. In the second and third layers, This indicates the output of the previous layer. This represents the style information features output by the second Transformer encoder, which are the same for every Transformer decoder layer. Indicates style location code, Skeleton position encoding, FFN is a linear layer with 2 hidden layers, Relu activation function is added in the middle, LN represents layer normalization, is the output of a Transformer decoder layer.

[0107] S203, decode the fusion feature vector through the output layer, and convert the fusion feature vector into a cement hydration microstructure image enhanced by cross-device stylization.

[0108] Due to the highly complex internal structure of the cement hydration microstructure image, which is a homogeneous single-channel grayscale image, model training is very difficult, which is manifested as the loss function when training the model using the gradient descent algorithm will oscillate, and the final model will generate a gray pure color image, which is a typical mode collapse.

[0109] In order to stabilize the training of the model, as an embodiment, when training the cross-device image enhancement model, the specific process is as follows:

[0110] (1) Obtain a non-homogeneous image to construct a pre-training set, and pre-train the cross-device image enhancement model through the pre-training set. The parameters obtained by pre-training are used as the initialization parameters of the cross-device image enhancement model.

[0111] In this step, the input of the cross-device image enhancement model is the pre-training set, and the output is the enhanced image.

[0112] There will be no repeated structure in the non-homogeneous image. In this embodiment, the non-homogeneous image is a landscape.

[0113] (2) Obtain cement hydration microstructure images collected by different CT devices and perform data preprocessing to construct a training set. That is, a three-dimensional cement hydration microstructure image is sliced according to three different directions, and the sliced image is rotated by 90°, 180°, and 270° and saved.

[0114] In this embodiment, the image is enhanced by learning the style, so it is not necessary to ensure that the obtained data is of the same cement block. As long as any two cement blocks are collected by different CT devices.

[0115] The cement hydration microstructure image contains three-dimensional cement block information. Considering that the cross-device image enhancement model needs a large amount of data for training to ensure the effect of the model, and for cement images of the same type, the mode collapse problem will occur during training. Therefore, the cement hydration microstructure image obtained in this step is preprocessed.

[0116] (3) Train the cross-device image enhancement model to minimize the skeleton loss, style loss, and double-layer cycle consistency loss.

[0117] In this embodiment, features of the original image input to the cross-device image enhancement model and the generated image output by the cross-device image enhancement model are extracted by a pre-trained VGG neural network. The difference in features between the generated image and the input original image is calculated as the loss of the model and used to train the cross-device image model.

[0118] The cement hydration microstructure image obtained by a low-configuration CT device is denoted as... The cement image obtained by a high-configuration CT device is denoted as The cross-device image enhancement model output will Stylization The styled image is denoted as Similarly, if we take and To exchange, Extract style information and perform style information extraction. By extracting skeleton information, it is possible to generate... Stylization Style images Here, the generated and This is equivalent to exchanging the styles of cement hydration microstructure images generated by two different CT devices.

[0119] The generated and The images are then fed back as input to the cross-device image enhancement model. The process of swapping the styles of the two cement hydration microstructure images is repeated. After this style swap, a result matching the original image should be generated. Figure 1 The cement hydration microstructure images generated by the two cycles can be compared with the original image to form a two-layer recurrent consistency loss for network training.

[0120] For example:

[0121] ;

[0122] ;

[0123] ;

[0124] ;

[0125] ;

[0126] ;

[0127] ;

[0128] In the formula, This represents a cross-device image enhancement model. This represents the image obtained after stylizing the first cement hydration microstructure image into the style of the second cement hydration microstructure image, where cs represents the fifth subscript. This represents the image obtained by stylizing the second cement hydration microstructure image to the style of the first cement hydration microstructure image. sc represents the sixth superscript. Indicates will Stylization The image obtained after styling, cyc-c indicates the seventh subscript. Indicates will Stylization The image obtained after styling, cyc-s indicates the eighth subscript. This represents the VGG neural network. This indicates that the data was extracted through the VGG network. Feature information, g Indicates the eighth subscript. This indicates that the data was extracted through the VGG network. Feature information, This indicates that the data was extracted through the VGG network. The characteristic information.

[0129] The skeleton loss function is then expressed as:

[0130] ;

[0131] ;

[0132] In the formula, This indicates the number of layers used in the VGG neural network for output, typically 5. Let represent the mean squared error function, and j represent the j-th feature output by the VGG network.

[0133] The style loss function is then expressed as:

[0134] ;

[0135] In the formula, Represents the variance of the data. This represents the mean of the data.

[0136] The two-level circular consistency loss function is then expressed as:

[0137] .

[0138] The overall training loss function is expressed as:

[0139] ;

[0140] in which, 、 、 represents a weight parameter, and all represents a ninth subscript.

[0141] Embodiment Two

[0142] The embodiment discloses a cross-device stylized enhancement system for cement hydration microstructure images, comprising:

[0143] An acquisition module is configured to acquire a first cement hydration microstructure image and a second cement hydration microstructure image, wherein the first cement hydration microstructure image and the second cement hydration microstructure image are collected by different CT devices.

[0144] A cross-device stylized enhancement module is configured to input the first cement hydration microstructure image and the second cement hydration microstructure image into a trained cross-device image enhancement model for processing to obtain a cross-device stylized enhanced cement hydration microstructure image.

[0145] The cross-device image enhancement model encodes the first cement hydration microstructure image and the second cement hydration microstructure image by an input layer, respectively, generates corresponding skeleton position encodings and adds them to the first cement hydration microstructure image, and generates corresponding style position encodings and adds them to the second cement hydration microstructure image.

[0146] It should be noted that the above acquisition module and cross-device image enhancement module correspond to the steps in Embodiment One, and the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment One. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0147] Embodiment Three

[0148] Embodiment Three of the present application provides an electronic device comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the steps of the above cross-device stylized enhancement method for cement hydration microstructure images are completed.

[0149] Embodiment Four

[0150] Embodiment Four of the present application provides a computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the steps of the above cross-device stylized enhancement method for cement hydration microstructure images are completed.

[0151] Embodiment Five

[0152] Embodiment five of the present application provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the above-mentioned method for cross-device stylized enhancement of cement hydration microstructure images.

[0153] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagrams, as well as combinations of flows and / or blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowchart and / or block diagrams of the flowchart and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flow or flows and / or block or blocks.

[0154] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufacture product including instruction means, which implement the functions specified in the flowchart and / or block diagrams of the flowchart and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flow or flows and / or block or blocks.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to execute a series of operation steps to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowchart and / or block diagrams of the flowchart and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flow or flows and / or block or blocks.

[0156] The description of each embodiment in the above embodiments is focused on each embodiment, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0157] The above only describes the preferred embodiments of the present application and is not used to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A cross-device stylization enhancement method for cement hydration microstructure images, characterized in that, include: A first cement hydration microstructure image and a second cement hydration microstructure image are acquired, wherein the first cement hydration microstructure image and the second cement hydration microstructure image are acquired by different CT devices; the image quality of the first cement hydration microstructure image is inferior to that of the second cement hydration microstructure image; wherein the first cement hydration microstructure image and the second cement hydration microstructure image acquired by the CT devices are from different cement blocks or the same cement block. The first cement hydration microstructure image and the second cement hydration microstructure image are input into a trained cross-device image enhancement model for processing to obtain a cross-device stylized enhanced cement hydration microstructure image. The cross-device image enhancement model converts a first cement hydration microstructure image into a first image feature vector through an embedding layer. Max pooling and average pooling are then performed on the first image feature vector in parallel. The difference between the max pooling and average pooling results is then convolved. An interpolation algorithm is used to restore the size of the convolved first image feature vector, generating a skeleton position code which is added to the first image feature vector. A second cement hydration microstructure image is then converted into a second image feature vector through an embedding layer. The second image feature vector is then convolved to generate a style position code which is added to the second image feature vector. The cross-device image enhancement model includes an input layer, a representation layer, and an output layer connected in sequence. The input layer includes an embedding layer and a position encoding layer connected in sequence. The representation layer includes a Transformer encoder module and a Transformer decoder module connected in sequence. The output layer includes an upsampling module. The Transformer encoder modules are multiple and configured in parallel. The Transformer encoder module is used to extract skeleton information from the first image feature vector with added skeleton position encoding and to extract style information from the second image feature vector with added style position encoding, so as to obtain the corresponding skeleton information features and style information features. The Transformer decoder module is used to decode and fuse the skeleton information features and the style information features to generate a fused feature vector. The upsampling module is used to decode the fused feature vector and convert the fused feature vector into a cross-device stylized enhanced image of cement hydration microstructure. Skeleton position encoding is represented as: ； In the formula, Indicates the skeleton position code. c Indicates the first subscript, This represents an interpolation algorithm that restores the first image feature vector after convolution to its initial size. express convolution, This indicates a max pooling operation. This indicates the average pooling operation. Indicates the size after the pooling operation; Style location coding is represented as: ； In the formula, Indicates style location code, s Indicates the second subscript. Indicates a property with one boundary extension. Convolution; Transfer learning is used to pre-train a cross-device image augmentation model using non-homogeneous images, and the model is trained with the aim of minimizing skeleton loss, style loss, and bi-layer cycle consistency loss.

2. A cross-device stylization enhancement system for cement hydration microstructure images, characterized in that, include: The acquisition module is configured to acquire a first cement hydration microstructure image and a second cement hydration microstructure image, wherein the first cement hydration microstructure image and the second cement hydration microstructure image are acquired by different CT devices; the image quality of the first cement hydration microstructure image is inferior to that of the second cement hydration microstructure image; wherein the first cement hydration microstructure image and the second cement hydration microstructure image acquired by the CT devices come from different cement blocks or the same cement block. The cross-device stylization enhancement module is configured to: input the first cement hydration microstructure image and the second cement hydration microstructure image into the trained cross-device image enhancement model for processing, and obtain the cement hydration microstructure image after cross-device stylization enhancement; The cross-device image enhancement model converts a first cement hydration microstructure image into a first image feature vector through an embedding layer. Max pooling and average pooling are then performed on the first image feature vector in parallel. The difference between the max pooling and average pooling results is then convolved. An interpolation algorithm is used to restore the size of the convolved first image feature vector, generating a skeleton position code which is added to the first image feature vector. A second cement hydration microstructure image is then converted into a second image feature vector through an embedding layer. The second image feature vector is then convolved to generate a style position code which is added to the second image feature vector. The cross-device image enhancement model includes an input layer, a representation layer, and an output layer connected in sequence. The input layer includes an embedding layer and a position encoding layer connected in sequence. The representation layer includes a Transformer encoder module and a Transformer decoder module connected in sequence. The output layer includes an upsampling module. The Transformer encoder modules are multiple and configured in parallel. The Transformer encoder module is used to extract skeleton information from the first image feature vector with added skeleton position encoding and to extract style information from the second image feature vector with added style position encoding, so as to obtain the corresponding skeleton information features and style information features. The Transformer decoder module is used to decode and fuse the skeleton information features and the style information features to generate a fused feature vector. The upsampling module is used to decode the fused feature vector and convert the fused feature vector into a cross-device stylized enhanced image of cement hydration microstructure. Skeleton position encoding is represented as: ； In the formula, Indicates the skeleton position code. c Indicates the first subscript, This represents an interpolation algorithm that restores the first image feature vector after convolution to its initial size. express convolution, This indicates a max pooling operation. This indicates the average pooling operation. Indicates the size after the pooling operation; Style location coding is represented as: ； In the formula, Indicates style location code, s Indicates the second subscript. Indicates a property with one boundary extension. Convolution; Transfer learning is used to pre-train a cross-device image augmentation model using non-homogeneous images, and the model is trained with the aim of minimizing skeleton loss, style loss, and bi-layer cycle consistency loss.

3. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the cross-device stylization enhancement method for cement hydration microstructure images as described in claim 1.

4. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the cross-device stylization enhancement method for cement hydration microstructure images as described in claim 1.

5. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the cross-device stylization enhancement method for cement hydration microstructure images as described in claim 1.

Images