Molecular sieve image restoration method and restoration system
By using a generative adversarial network model to repair molecular sieve images, the problem of particle size information not being extracted due to image occlusion in molecular sieves was solved, achieving efficient and accurate image repair and particle size information extraction.
Patent Information
- Application Number
- CN202310814087.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-04
- Publication Date
- 2025-01-10
AI Technical Summary
During the synthesis of molecular sieve catalysts, the clustered distribution and high overlap can cause image occlusion, making it impossible to accurately extract particle size information. Existing technologies cannot effectively repair occluded molecular sieve images.
An image inpainting model based on generative adversarial networks is adopted. Through the game between the generator and the discriminator, the generator is trained to generate realistic molecular sieve images. The adversarial loss and reconstruction loss of the generative adversarial network are used to optimize the model and realize the inpainting of occluded images.
It enables accurate restoration of obscured molecular sieve images, improving the accuracy and efficiency of extracting characterization information such as molecular sieve particle size.
Smart Images
Figure CN119295345A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a molecular sieve image repair method and repair system. Background Art
[0002] In the field of petrochemicals, molecular sieve catalyst materials are widely used. At present, the synthesis process of molecular sieve catalysts is gradually becoming intelligent and unmanned. In the intelligent synthesis experiment of molecular sieve catalysts, it is very important to extract characterization information such as particle size from unstructured molecular sieve images. However, the clustered distribution and high overlap characteristics of molecular sieve catalysts during the synthesis process will cause the molecular sieve particles to be obscured, making it impossible to accurately extract the particle size information of the molecular sieve. Therefore, repairing the obscured molecular sieve SEM images is an indispensable part of improving the efficiency of intelligent experiments. Summary of the invention
[0003] The object of the present invention is to provide a molecular sieve image repair method and repair system, which can accurately repair the blocked molecular sieve image, so as to obtain the corresponding particle size information through the complete outline of the repaired molecular sieve image.
[0004] In order to achieve the above-mentioned objectives, the present invention provides a method for repairing a molecular sieve image on one hand, the method comprising: obtaining a molecular sieve image to be repaired; and repairing the molecular sieve image to be repaired using a trained image repair model based on a generative adversarial network.
[0005] Preferably, the image restoration model based on the generative adversarial network includes: a generator, which includes: an encoder for generating a high-dimensional feature vector of the molecular sieve image to be repaired, wherein the encoder is a residual module and a convolution module; and a decoder for mapping the high-dimensional feature vector into an RGB image to obtain a repaired molecular sieve image, and a discriminator for outputting the probability of whether the repaired molecular sieve image is complete.
[0006] Preferably, the restoration method also includes: obtaining a training set of molecular sieve images, wherein the training set includes multiple occluded molecular sieve images and multiple complete molecular sieve images, and the multiple occluded molecular sieve images correspond one-to-one to the multiple complete molecular sieve images; and using the training set to train the image restoration model based on the generative adversarial network until the loss function meets a preset threshold.
[0007] Preferably, the loss function is expressed as:
[0008]
[0009] Among them, G is the generator, D is the discriminator, and Ioccluded is the obscured molecular sieve image, minE[log(1-D(G(I occluded ))) is the adversarial loss of the generative adversarial network; I gt is the complete molecular sieve image corresponding to the blocked molecular sieve image, I prd The repaired molecular sieve image generated by the generator, is the reconstruction loss; w is the loss weight.
[0010] Preferably, the use of the training set to train the image restoration model based on the generative adversarial network includes: when the loss function does not meet the preset threshold, using the stochastic gradient descent method to optimize the feature parameters of the image restoration model based on the generative adversarial network; and using the training set to train the optimized image restoration model based on the generative adversarial network.
[0011] Preferably, the multiple obscured molecular sieve images are acquired in the following manner: marking and segmenting the complete molecular sieve particles in the SEM image; placing the complete molecular sieve particles in a completely black canvas; selecting spheres, cubes and prisms of molecular sieves as foreground molecular sieve particles, respectively, and using the complete molecular sieve particles as background molecular sieve particles; placing the background molecular sieve particles in a new canvas, and placing the foreground molecular sieve at different positions of the new canvas to acquire the obscured background molecular sieve particles; and using the obscured background molecular sieve particles as the obscured molecular sieve images.
[0012] Preferably, the area ratio of the plurality of blocked molecular sieve images to the corresponding complete molecular sieve images is 50%-80%.
[0013] Through the above technical scheme, the molecular sieve image to be repaired is first obtained, and then the trained image repair model based on the generative adversarial network is used to repair the molecular sieve image to be repaired. Thus, the intelligent repair of the occluded molecular sieve image can be realized, so that the outline and content of the molecular sieve image are complete, thereby greatly improving the accuracy and efficiency of extracting characterization information such as the molecular sieve particle size.
[0014] A second aspect of the present invention provides a molecular sieve image repair system, which may include: a first image acquisition device for acquiring a molecular sieve image to be repaired; and a repair device for repairing the molecular sieve image to be repaired using a trained image repair model based on a generative adversarial network.
[0015] Preferably, the image restoration model based on the generative adversarial network includes: a generator, which includes: an encoder for generating a high-dimensional feature vector of the molecular sieve image to be repaired, wherein the encoder includes a residual module and a convolution module; and a decoder for mapping the high-dimensional feature vector into an RGB image to obtain a repaired molecular sieve image, and a discriminator for outputting the probability of whether the repaired molecular sieve image is complete.
[0016] Preferably, the restoration system also includes: a sample acquisition device for acquiring a training set of molecular sieve images, wherein the training set includes multiple occluded molecular sieve images and multiple complete molecular sieve images, and the multiple occluded molecular sieve images correspond one-to-one to the multiple complete molecular sieve images; and a training device for using the training set to train the image restoration model based on the generative adversarial network until the loss function meets a preset threshold.
[0017] Preferably, the loss function is expressed as:
[0018]
[0019] Among them, G is the generator, D is the discriminator, and I occluded is the obscured molecular sieve image, minE[log(1-D(G(I occluded ))) is the adversarial loss of the generative adversarial network; I gt is the complete molecular sieve image corresponding to the blocked molecular sieve image, I prd The repaired molecular sieve image generated by the generator, is the reconstruction loss; w is the loss weight.
[0020] Preferably, the training device includes: an optimization module, used to optimize the feature parameters of the image restoration model based on the generative adversarial network by using the stochastic gradient descent method when the loss function does not meet the preset threshold; and a training module, used to train the optimized image restoration model based on the generative adversarial network by using the training set.
[0021] Preferably, the repair system also includes: a second image acquisition device, used to acquire the multiple obscured molecular sieve images in the following manner: marking and segmenting the complete molecular sieve particles in the SEM image; placing the complete molecular sieve particles in a completely black canvas; selecting spheres, cubes and prisms of molecular sieves as foreground molecular sieve particles, respectively, and using the complete molecular sieve particles as background molecular sieve particles; placing the background molecular sieve particles in a new canvas, and placing the foreground molecular sieve at different positions of the new canvas to acquire the obscured background molecular sieve particles; and using the obscured background molecular sieve particles as the obscured molecular sieve images.
[0022] Preferably, the area ratio of the plurality of blocked molecular sieve images to the corresponding complete molecular sieve images is 50%-80%.
[0023] The specific details and benefits of the molecular sieve image repair system provided by the embodiment of the present invention can be found in the description of the molecular sieve image repair method mentioned above, which will not be repeated here.
[0024] A third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the molecular sieve image repair method is implemented.
[0025] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:
[0027] Figure 1 It is a schematic flow chart of a method for repairing a molecular sieve image based on a generative adversarial network provided by one embodiment of the present invention;
[0028] Figure 2 It is a model structure of a molecular sieve image repair system based on a generative adversarial network provided by an embodiment of the present invention;
[0029] Figure 3A and Figure 3B is an original scanning electron microscope (SEM) image of a molecular sieve provided by an embodiment of the present invention;
[0030] Figure 4 is a complete molecular sieve image provided by an embodiment of the present invention;
[0031] Figure 5 An embodiment of the present invention provides Figure 4 The corresponding obscured molecular sieve image;
[0032] Figure 6 This is an example diagram of a restoration result obtained by a molecular sieve SEM image restoration method based on a generative adversarial network provided by an embodiment of the present invention;
[0033] Figure 7 is a system block diagram of an electronic device provided by an embodiment of the present invention; and
[0034] Figure 8 It is a comparison diagram of the repair effect provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0035] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.
[0036] Figure 1 FIG. 1 is a flow chart of a method for repairing a molecular sieve image provided by an embodiment of the present invention. Figure 1 As shown, the repair method may include step S101, obtaining a molecular sieve image to be repaired; and step S102, using a trained image repair model based on a generative adversarial network to repair the molecular sieve image to be repaired.
[0037] This embodiment provides a method for repairing a molecular sieve image (e.g., a scanning electron microscope image, i.e., a SEM image) based on a generative adversarial network, which can accurately repair the occluded molecular sieve image, thereby obtaining the corresponding particle size information through the complete outline of the repaired molecular sieve image.
[0038] The above steps S101 - S102 are explained and illustrated below.
[0039] Step S101, obtaining an image of the molecular sieve to be repaired.
[0040] A scanning electron microscope (SEM) image of the to-be-repaired samples in a certain experimental batch is obtained, for example, a SEM grayscale image (ie, an original SEM image).
[0041] Step S102: Use the trained image restoration model based on the generative adversarial network to restore the molecular sieve image to be restored.
[0042] The image restoration model based on the generative adversarial network includes: a generator 10, the generator includes: an encoder 100, which is used to generate a high-dimensional feature vector of the molecular sieve image to be restored, wherein the encoder 100 includes a convolution module 101 and a residual module 102; and a decoder 110, which is used to map the high-dimensional feature vector to an RGB image to obtain a restored molecular sieve image, and a discriminator 20, which is used to output the probability of whether the restored molecular sieve image is complete, such as Figure 2 shown.
[0043] like Figure 2 As shown, the image restoration model based on the generative adversarial network includes a generator 10 and a discriminator 20.
[0044] First, the specific structure of the generator 10 is described.
[0045] like Figure 2 As shown, the generator 10 includes an encoder 100 and a decoder 110, and the high-dimensional feature representation of the input image is obtained by the encoder 100, and the high-dimensional feature representation is mapped back to the RGB image by the decoder 110. Among them, the encoder 100 is a feature extraction network. In this embodiment, the feature extraction network selects the structure of the convolution module 101 (for example, 3 convolution layers) and the residual module 102 (for example, 8 residual blocks). Among them, the decoder 110 selects the structure of 3 deconvolution layers and sets the output channel 256 dimensions. The input of the generator is the occluded molecular sieve image, and the output is the predicted complete molecular sieve image.
[0046] Specifically, the residual connection adds a skip connection on the basis of the convolution operation. The occluded molecular sieve image x0 is output through the convolution layer F(x0); and then output through the residual block Output n =F(x n )+x n : The first residual block: Substitute the convolutional layer output F(x0) as x1 into the Output formula to obtain Output1=F(x1)+x1; The second residual block: Substitute Output1 as x2 into the Output formula to obtain Output2, and so on, the eighth residual block, substitute Output7 as x8 into the Output formula to obtain Output8, that is, the high-dimensional features of the input image. The residual connection converts the output into a linear superposition of the input and the nonlinear transformation of the input, further avoiding the problem of gradient dissipation when training the repair model. Since the encoder uses a residual connection, this embodiment can ensure a sufficiently large receptive field to achieve the repair of larger missing areas of the molecular sieve image.
[0047] Next, the specific structure of the discriminator 20 will be described.
[0048] like Figure 2 As shown, the discriminator 20 is a feature extraction network. In this embodiment, the discriminator 20 selects a structure of 5 convolutional layers. The input of the discriminator 20 is the complete molecular sieve image and the complete molecular sieve image predicted by the generator, and the output is a probability value for judging true or false.
[0049] The discriminator 20 maps the input image into a matrix of size N*N, that is, divides the input image into N*N blocks for evaluation, so as to increase the attention paid to the molecular sieve missing area during the repair process and further improve the repair effect.
[0050] In the above embodiment, the activation functions in the generator 10 and the discriminator 20 are both ReLU functions, and spectral normalization is used after each convolution layer to avoid the gradient vanishing problem during the training process.
[0051] In the training process of the image restoration model described below, the generator and the discriminator compete with each other: the discriminator is used to distinguish the real complete molecular sieve image from the complete molecular sieve image predicted by the generator, while the generator is used to generate an image that is as close as possible to the real complete molecular sieve image, so as to restore the occluded molecular sieve image.
[0052] The following explains and illustrates the training process of the image restoration model based on the generative adversarial network.
[0053] In one embodiment, the restoration method may further include: obtaining a training set of molecular sieve images, wherein the training set includes multiple occluded molecular sieve images and multiple complete molecular sieve images, and the multiple occluded molecular sieve images correspond one-to-one to the multiple complete molecular sieve images; and using the training set to train the image restoration model based on the generative adversarial network until the loss function meets a preset threshold.
[0054] Among them, the multiple obscured molecular sieve images can be obtained in the following manner: marking and segmenting the complete molecular sieve particles in the SEM image; placing the complete molecular sieve particles in a completely black canvas; selecting spheres, cubes and prisms of molecular sieves as foreground molecular sieve particles, respectively, and using the complete molecular sieve particles as background molecular sieve particles; placing the background molecular sieve particles in a new canvas, and placing the foreground molecular sieve at different positions of the new canvas to obtain the obscured background molecular sieve particles; and using the obscured background molecular sieve particles as the obscured molecular sieve images.
[0055] The area ratio of the plurality of blocked molecular sieve images to the corresponding complete molecular sieve image is 50%-80%.
[0056] The following explains and illustrates the acquisition of the training set and the training process of the model.
[0057] First, the process of obtaining a training set of molecular sieve images is explained and illustrated.
[0058] For example, Figure 3A or Figure 3B As shown, a scanning electron microscope (SEM) image of a certain experimental batch is obtained, and the original SEM image is processed to construct a molecular sieve SEM image dataset. Among them, the SEM image is usually a SEM grayscale image, but in some application scenarios, a color SEM image can also be included in the SEM image dataset.
[0059] Specifically, the process of obtaining a complete molecular sieve image is: marking and segmenting the complete molecular sieve particles in the original SEM image to obtain a molecular sieve image of a single complete particle (such as Figure 4 As shown), the marked image is presented in the form of a json file.
[0060] The marking and segmentation of the complete molecular sieve particles in the original SEM image are specifically as follows: marking and segmenting the complete molecular sieve particles based on the contour, wherein the molecular sieve can be divided into cubic molecular sieves, prismatic molecular sieves and spherical molecular sieves according to their contour shapes. The process of obtaining the obscured molecular sieve image is specifically as follows: mutually obscuring the complete molecular sieve images to obtain the obscured molecular sieve image that conforms to the actual obstruction situation (such as Figure 5 As shown in FIG. 1 ), the mutual occlusion is that the foreground molecular sieve particles occlude the background molecular sieve particles, and cause the destruction of the outline and content of the background molecular sieve particles. There is a one-to-one correspondence between the occluded molecular sieve image and the complete molecular sieve image.
[0061] The blocked molecular sieve image is specifically obtained through the following steps.
[0062] Step 1: Mark the complete molecular sieve particles in the SEM image, segment the complete molecular sieve particles by using instance segmentation technology, and place them in a completely black canvas.
[0063] Step 2: Select spherical, cubic and prism (eg hexagonal) molecular sieves as foreground molecular sieve particles respectively, and use the molecular sieve particles obtained in step 1 as background molecular sieve particles.
[0064] Step 3: Place the background molecular sieve particles in a new canvas, and place the foreground molecular sieve at different positions of the new canvas to achieve multi-angle shielding of the background molecular sieve particles.
[0065] The blocked molecular sieve images include cubic molecular sieve blocking cubic molecular sieve, prismatic molecular sieve blocking prismatic molecular sieve, spherical molecular sieve blocking spherical molecular sieve, and mutual blocking between the three molecular sieves.
[0066] Step 4: retain the remaining portion of the background molecular sieve particles after being blocked as the blocked molecular sieve image.
[0067] Secondly, the process of training the image restoration model based on the generative adversarial network is explained and illustrated.
[0068] The use of the training set to train the image restoration model based on the generative adversarial network may include: when the loss function does not meet the preset threshold, using the stochastic gradient descent method to optimize the feature parameters of the image restoration model based on the generative adversarial network; and using the training set to train the optimized image restoration model based on the generative adversarial network.
[0069] In one embodiment, the loss function is expressed as:
[0070]
[0071] Among them, G is the generator, D is the discriminator, and I occluded is the obscured molecular sieve image, minE[log(1-D(G(I occluded ))) is the adversarial loss of the generative adversarial network; I gt is the complete molecular sieve image corresponding to the blocked molecular sieve image, I prd The repaired molecular sieve image generated by the generator, is the reconstruction loss; w is the loss weight.
[0072] In the expression of the above loss function, the first term is the adversarial loss of the generative adversarial network, which guides the model to generate images close to the real complete molecular sieve image. The second term is the reconstruction loss, which aims to narrow the generation space of the generator and guide the generator to generate images that are closer to the real complete molecular sieve in terms of contour and content. The reconstruction loss can effectively alleviate the problem of oscillation and difficulty in convergence of the generative adversarial network during training.
[0073] Specifically, the test set can be obtained by a method similar to the above-mentioned method of obtaining the training set, or the sample set can be obtained by a method similar to the above-mentioned method of obtaining the training set, and the sample set is randomly divided into a training set and a test set according to a preset ratio (for example, the training set contains 8,000 samples, and the test set contains 2,000 samples); then, the image restoration model is trained through the following steps 1 to 4.
[0074] Step 1: Construct an image restoration model based on a generative adversarial network, wherein the image restoration model includes the above-mentioned generator and the above-mentioned discriminator.
[0075] Step 2: Setting training parameters in the image restoration model training process, wherein the training parameters include the size of batch training, the size of learning rate and the weight size w of training loss.
[0076] Step 3: Train the image restoration model based on the one-to-one corresponding occluded molecular sieve images and complete molecular sieve images in the training set.
[0077] During each round of training, the training set is input into the image restoration model to obtain the loss function. If the loss function does not meet the preset threshold, the stochastic gradient descent method is used to optimize the feature parameters of the image restoration model based on the generative adversarial network, and then the training set is used to train the optimized image restoration model. If the loss function meets the preset threshold, it indicates that the model has been trained, and thus the model can be determined to be a trained model.
[0078] Step 4: Verify the trained image restoration model using the occluded molecular sieve images in the test set to obtain better training results.
[0079] After the trained image restoration model is obtained through the above process, the molecular sieve image to be restored may be input into the trained image restoration model to obtain a complete molecular sieve image corresponding to the molecular sieve image to be restored.
[0080] The above embodiments can construct and train an image restoration model based on a generative adversarial network based on a SEM image dataset, and train the model by optimizing a loss function to achieve intelligent restoration of the obscured molecular sieve image (the restoration result is as follows: Figure 6 As shown), the outline and content of the molecular sieve image are complete and the repair accuracy is high (as shown Figure 8 As shown in the figure, the accuracy and efficiency of extracting characterization information such as molecular sieve particle size are greatly improved.
[0081] In summary, the present invention creatively obtains the molecular sieve image to be repaired, and then uses the trained image repair model based on the generative adversarial network to repair the molecular sieve image to be repaired. Thus, the intelligent repair of the occluded molecular sieve image can be achieved, so that the outline and content of the molecular sieve image are complete, thereby greatly improving the accuracy and efficiency of extracting characterization information such as the molecular sieve particle size.
[0082] An embodiment of the present invention provides a molecular sieve image repair system, which may include: a first image acquisition device, used to acquire the molecular sieve image to be repaired; and a repair device, used to repair the molecular sieve image to be repaired using a trained image repair model based on a generative adversarial network.
[0083] Preferably, the image restoration model based on the generative adversarial network includes: a generator, which includes: an encoder for generating a high-dimensional feature vector of the molecular sieve image to be repaired, wherein the encoder includes a residual module and a convolution module; and a decoder for mapping the high-dimensional feature vector into an RGB image to obtain a repaired molecular sieve image, and a discriminator for outputting the probability of whether the repaired molecular sieve image is complete.
[0084] Preferably, the restoration system also includes: a sample acquisition device for acquiring a training set of molecular sieve images, wherein the training set includes multiple occluded molecular sieve images and multiple complete molecular sieve images, and the multiple occluded molecular sieve images correspond one-to-one to the multiple complete molecular sieve images; and a training device for using the training set to train the image restoration model based on the generative adversarial network until the loss function meets a preset threshold.
[0085] Preferably, the loss function is expressed as:
[0086]
[0087] Among them, G is the generator, D is the discriminator, and I occluded is the obscured molecular sieve image, minE[log(1-D(G(I occluded ))) is the adversarial loss of the generative adversarial network; I gt is the complete molecular sieve image corresponding to the blocked molecular sieve image, I prd The repaired molecular sieve image generated by the generator, is the reconstruction loss; w is the loss weight.
[0088] Preferably, the training device includes: an optimization module, used to optimize the feature parameters of the image restoration model based on the generative adversarial network by using the stochastic gradient descent method when the loss function does not meet the preset threshold; and a training module, used to train the optimized image restoration model based on the generative adversarial network by using the training set.
[0089] Preferably, the repair system also includes: a second image acquisition device, used to acquire the multiple obscured molecular sieve images in the following manner: marking and segmenting the complete molecular sieve particles in the SEM image; placing the complete molecular sieve particles in a completely black canvas; selecting spheres, cubes and prisms of molecular sieves as foreground molecular sieve particles, respectively, and using the complete molecular sieve particles as background molecular sieve particles; placing the background molecular sieve particles in a new canvas, and placing the foreground molecular sieve at different positions of the new canvas to acquire the obscured background molecular sieve particles; and using the obscured background molecular sieve particles as the obscured molecular sieve images.
[0090] Preferably, the area ratio of the plurality of blocked molecular sieve images to the corresponding complete molecular sieve images is 50%-80%.
[0091] The specific details and benefits of the molecular sieve image repair system provided by the embodiment of the present invention can be found in the description of the molecular sieve image repair method mentioned above, which will not be repeated here.
[0092] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the molecular sieve image repair method is implemented.
[0093] Figure 7 is a structural diagram of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, the electronic device includes a bus 70, a processor 71, and a memory 72, wherein the processor 71 and the memory 72 communicate with each other via the bus 70; the processor 71 is used to call program instructions in the memory 72 to execute the molecular sieve image repair method.
[0094] On the basis of the above embodiment, the electronic device further comprises: a communication port 73 for realizing data communication between the electronic device and the outside.
[0095] The bus 70 is used to realize data communication between various components. The processor 71 may include one or more processors to execute program instructions, make judgments and issue prompts to realize the main part of the method, and the processing results of the processor 71 are transmitted to the user device through the communication port 73, and then displayed on the user interface. The electronic device can send and receive information / data from the network through the communication port 35.
[0096] The memory 72 may include a hard disk, a read-only memory (ROM) and a random access memory (RAM) to store various data files used for computer processing and / or communication and program instructions executed by the processor 71 .
[0097] In this embodiment, the outline and content of the molecular sieve image can be accurately restored, and the visual effect can be very close to the real complete molecular sieve image.
[0098] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.
[0099] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.
[0100] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0101] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.
Claims
1. A method for repairing a molecular sieve image, characterized in that: The repair method comprises: Acquiring an image of the molecular sieve to be repaired; and The molecular sieve image to be repaired is repaired using a trained image repair model based on a generative adversarial network.
2. The repair method according to claim 1, characterized in that: The image restoration model based on the generative adversarial network includes: A generator, the generator comprising: An encoder, used to generate a high-dimensional feature vector of the molecular sieve image to be repaired, wherein the encoder is a residual module and a convolution module; and A decoder, for mapping the high-dimensional feature vector into an RGB image to obtain a repaired molecular sieve image, and The discriminator is used to output the probability of whether the repaired molecular sieve image is complete.
3. The repair method according to claim 2, characterized in that: The repair method further comprises: Acquire a training set of molecular sieve images, wherein the training set includes a plurality of blocked molecular sieve images and a plurality of complete molecular sieve images, and the plurality of blocked molecular sieve images correspond one to one to the plurality of complete molecular sieve images; and The training set is used to train the image restoration model based on the generative adversarial network until the loss function meets a preset threshold.
4. The repair method according to claim 3, characterized in that: The expression of the loss function is: Among them, G is the generator, D is the discriminator, and I occluded is the obscured molecular sieve image, minE[(1-D(( occluded ))) is the adversarial loss of the generative adversarial network; I gt is the complete molecular sieve image corresponding to the blocked molecular sieve image, I prd The repaired molecular sieve image generated by the generator, is the reconstruction loss; w is the loss weight.
5. The repair method according to claim 3, characterized in that: The adopting the training set to train the image restoration model based on the generative adversarial network comprises: When the loss function does not meet the preset threshold, a stochastic gradient descent method is used to optimize the feature parameters of the image restoration model based on the generative adversarial network; and The training set is used to train the optimized image restoration model based on the generative adversarial network.
6. The repair method according to claim 3, characterized in that: The plurality of obscured molecular sieve images are obtained by: Mark and segment intact molecular sieve particles in SEM images; Placing the intact molecular sieve particles in a completely black canvas; Selecting spheres, cubes and prisms of molecular sieves as foreground molecular sieve particles, respectively, and using the complete molecular sieve particles as background molecular sieve particles; Put the background molecular sieve particles in a new canvas, and place the foreground molecular sieve at different positions of the new canvas to obtain the obscured background molecular sieve particles; as well as The blocked background molecular sieve particles are used as the blocked molecular sieve image.
7. The repair method according to claim 3, characterized in that: The area ratio of the multiple blocked molecular sieve images to the corresponding complete molecular sieve image is 50%-80%.
8. A molecular sieve image repair system, characterized in that: The repair system comprises: A first image acquisition device, used to acquire an image of the molecular sieve to be repaired; and A repair device is used to repair the molecular sieve image to be repaired by using a trained image repair model based on a generative adversarial network.
9. The repair system according to claim 8, characterized in that: The image restoration model based on the generative adversarial network includes: A generator, the generator comprising: An encoder, used to generate a high-dimensional feature vector of the molecular sieve image to be repaired, wherein the encoder includes a residual module and a convolution module; and A decoder, for mapping the high-dimensional feature vector into an RGB image to obtain a repaired molecular sieve image, and The discriminator is used to output the probability of whether the repaired molecular sieve image is complete.
10. The repair system according to claim 9, characterized in that: The repair system also includes: A sample acquisition device, used to acquire a training set of molecular sieve images, wherein the training set includes a plurality of blocked molecular sieve images and a plurality of complete molecular sieve images, and the plurality of blocked molecular sieve images correspond one to one to the plurality of complete molecular sieve images; and A training device is used to train the image restoration model based on the generative adversarial network using the training set until the loss function meets a preset threshold.
11. The repair system according to claim 10, characterized in that: The expression of the loss function is: Among them, G is the generator, D is the discriminator, and I occluded is the obscured molecular sieve image, minE[(1-D(( occluded ))) is the adversarial loss of the generative adversarial network; I gt is the complete molecular sieve image corresponding to the blocked molecular sieve image, I prd The repaired molecular sieve image generated by the generator, is the reconstruction loss; w is the loss weight.
12. The repair system according to claim 11, characterized in that: The training device comprises: an optimization module, configured to optimize the feature parameters of the image restoration model based on the generative adversarial network by using a stochastic gradient descent method when the loss function does not meet the preset threshold; and A training module is used to train the optimized image restoration model based on the generative adversarial network using the training set.
13. The repair system according to claim 10, characterized in that: The repair system further comprises: a second image acquisition device, configured to acquire the plurality of obscured molecular sieve images in the following manner: Mark and segment intact molecular sieve particles in SEM images; Placing the intact molecular sieve particles in a completely black canvas; Selecting spheres, cubes and prisms of molecular sieves as foreground molecular sieve particles, respectively, and using the complete molecular sieve particles as background molecular sieve particles; Placing the background molecular sieve particles in a new canvas, and placing the foreground molecular sieve particles in different positions of the new canvas to obtain the obscured background molecular sieve particles; and The blocked background molecular sieve particles are used as the blocked molecular sieve image.
14. The repair system according to claim 10, characterized in that: The area ratio of the multiple blocked molecular sieve images to the corresponding complete molecular sieve image is 50%-80%.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for repairing a molecular sieve image according to any one of claims 1 to 7.