Microstructured multilayer material recovery method and system

Abstract

The application discloses a microstructure multilayer material recovery method and system, comprising the following steps: constructing a multilayer microstructure material model based on SpongeCake, regarding each layer of material as a volume medium, and there is no interface between layers; the optical properties of each layer of medium are determined by a phase function; taking the material parameters and maps of each layer of medium as input values of the multilayer microstructure material model based on SpongeCake; rendering the multilayer microstructure material model based on SpongeCake, and the multilayer microstructure material model based on SpongeCake outputs a three-dimensional rendering image; calculating a loss function between the three-dimensional rendering image and a camera shooting image in the rendering process, using an optimizer to update the material parameters of each layer of medium according to the loss function value, gradually approximating the real material parameters, and outputting the material parameters of each layer of medium when a set number of iterations is reached, so that a simulation material model of a real multilayer microstructure material sample is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer graphics, in particular to a microstructure multi-layer material restoration method and system. BACKGROUND

[0002] The SpongeCake model is a physics-based material model. The SpongeCake model compares the surface of an object in nature to a stack of media. These media are uniformly distributed, and there is no interface between layers, which makes the refraction between layers be ignored. According to the position-free theory proposed by Guo et al. in 2018, the single scattering formula of each layer of media in the SpongeCake model can be obtained by accumulating single scattering light rays:

[0003]

[0004] is the Fresnel term, is the attenuation term, represents a normal distribution term derived from the phase function of each layer of material, and the commonly used phase function here is the microflake-based SGGX phase function or the Henyey-Greenstein phase function.

[0005] The Henyey-Greenstein phase function is a function used to simulate the scattering direction of light in participating media (such as atmosphere, smoke, cloud layer, etc.), which uses the parameter to control the directionality of scattering:

[0006]

[0007] The microflake-based SGGX phase function is a model used to simulate the scattering properties of complex materials (such as hair, skin, fabric, etc.), which assumes that the material in nature is composed of tiny mirror flakes, and uses various defined phase functions to describe the scattering direction of light after interacting with these micro flakes, among which the SGGX method is the most common. The SGGX method simulates the scattering behavior of micro flakes in the material, and defines the scattering direction of each micro flake through a probability density function, so as to describe how light scatters on these micro structures. The SGGX phase function uses an anisotropic Gaussian distribution (or ellipsoidal Gaussian distribution) to represent the distribution orientation of the micro flakes. The normal direction of each micro flake is randomly distributed, and its direction can be dynamically calculated according to the direction of incident light and outgoing light:

[0008]

[0009] where is the projection area of the micro flake in the direction, Item is a normal distribution item defined by the roughness of SGGX.

[0010] Differentiable rendering is a technique that establishes a differentiable relationship between the parameters in the rendering process (such as material, lighting, shape, etc.) and the generation result of the image, so that the rendering process can be backpropagated error through gradient optimization. The traditional rendering process is to generate an image from scene parameters, while in differentiable rendering, scene parameters can be adjusted and optimized according to the error of image generation. This technology is of great significance to deep learning and computer vision, because it allows the model to optimize directly in the image space, thereby helping to solve inverse problems such as 3D reconstruction, lighting estimation and material optimization, etc. SUMMARY

[0011] In order to solve the problems of the prior art, the present application provides a microstructure multi-layer material recovery method and system; a multi-layer material recovery system with microstructure is constructed, and the recovered multi-layer material parameters can render realistic microstructure multi-layer material images, while realizing visual matching with the real material images taken.

[0012] In one aspect, a microstructure multi-layer material recovery method is provided, comprising:

[0013] obtaining a camera taken image and a microscope taken image of a real multi-layer microstructure material sample;

[0014] performing image processing on the microscope taken image to extract the minimum repeating unit of each layer of medium, and storing the minimum repeating unit as a map;

[0015] constructing a multi-layer microstructure material model based on SpongeCake, regarding each layer of material as a volume medium, and there is no interface between layers; the optical properties of each layer of medium are determined by a phase function;

[0016] taking the material parameters and the map of each layer of medium as input values of the multi-layer microstructure material model based on SpongeCake; rendering the multi-layer microstructure material model based on SpongeCake, and the multi-layer microstructure material model based on SpongeCake outputs a three-dimensional rendering image;

[0017] calculating a loss function between the three-dimensional rendering image and the camera taken image during the rendering process, using an optimizer to update the material parameters of each layer of medium according to the loss function value, and gradually approaching the real material parameters; when the set number of iterations is reached, the material parameters of each layer of medium are output, and a simulation material model of the real multi-layer microstructure material sample is obtained.

[0018] In another aspect, a microstructure multi-layer material recovery system is provided, comprising:

[0019] an acquisition module configured to acquire a camera shot image and a microscope shot image of a real multi-layer microstructure material sample;

[0020] an image processing module configured to perform image processing on the microscope shot image, extract a minimum repeating unit of each layer of medium, and store the minimum repeating unit as a map;

[0021] a model construction module configured to construct a SpongeCake-based multi-layer microstructure material model, treat each layer of material as a volume medium, and there is no interface between layers; and determine optical properties of each layer of medium by a phase function;

[0022] a rendering module configured to take material parameters and the map of each layer of medium as input values of the SpongeCake-based multi-layer microstructure material model, perform rendering on the SpongeCake-based multi-layer microstructure material model, and output a three-dimensional rendering image based on the SpongeCake-based multi-layer microstructure material model;

[0023] an output module configured to calculate a loss function between the three-dimensional rendering image and the camera shot image during the rendering process, use an optimizer to update material parameters of each layer of medium based on the loss function value, and gradually approach real material parameters; and output the material parameters of each layer of medium when a set number of iterations is reached, to obtain a simulation material model of the real multi-layer microstructure material sample.

[0024] In another aspect, an electronic device is also provided, comprising:

[0025] a memory for non-transiently storing computer readable instructions; and

[0026] a processor for running the computer readable instructions,

[0027] wherein the computer readable instructions, when run by the processor, perform the method of the first aspect.

[0028] In another aspect, a storage medium is also provided, which non-transiently stores computer readable instructions, wherein when the non-transient computer readable instructions are executed by a computer, the method of the first aspect is performed.

[0029] In another aspect, a computer program product is also provided, which includes a computer program for implementing the method of the first aspect when run on one or more processors.

[0030] The above technical solution has the following advantages or beneficial effects:

[0031] The application is a multilayer microstructure material recovery system based on a SpongeCake model, and the rendered multilayer material image is delicate and real.

[0032] The application captures the multi-scale material by cooperation of a camera and a microscope, and obtains the real material microstructure minimum repeating unit by combining various image processing technologies, so that the rendering of the microstructure multilayer material becomes possible.

[0033] The application adopts the micro-rendering, realizes the automatic recovery of the parameters of the collected image of the real microstructure multilayer material by the parameter gradient back transmission, and improves the reality and convenience of the rendering of the multilayer microstructure. DETAILED DESCRIPTION

[0034] The drawings accompanying the specification of the application form a part of the application and serve to provide further understanding of the application, and the illustrative embodiments of the application and their descriptions serve to explain the application, and do not constitute improper limitations on the application.

[0035] Figure 1 It is a flowchart of the microstructure multilayer material recovery system based on the SpongeCake model of the application.

[0036] Fig. 2 (a) and Fig. 2 (b) are schematic diagrams of the camera and microscope collection device in step one of the method of the application.

[0037] Figure 3 It is a refinement flowchart of the minimum repeating unit generated by the microscope shooting data processing in step two of the method of the application.

[0038] Fig. 4 (a) and Fig. 4 (b) are schematic diagrams of the microscope shooting image and the minimum repeating unit in the method of the application.

[0039] Figure 5 It is a schematic diagram of the SpongeCake model layered material design in the method of the application.

[0040] Figure 6 It is a refinement flowchart of the micro-rendering recovery of the multilayer material parameters in the method of the application. DETAILED DESCRIPTION

[0041] It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the application belongs.

[0042] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments of the present application. As used herein, the terms "comprises", "comprising", "includes", "including" and the like are specifically intended to be open-ended. These terms are, specifically, intended to mean "comprising but not limited to", "including but not limited to" or "consisting of and the like. It will be appreciated that variations to the embodiments disclosed herein can be made and will be apparent to those skilled in the art, whose purpose it is to construct a novel, useful, and nonobvious implementation of the present application. Such variations are considered to be within the spirit and scope of the present application. The specification and drawings are, accordingly, to be regarded as illustrative and not restrictive.

[0043] Embodiment one, as shown in the figure, provides a microstructure multilayer material recovery method, comprising: Figure 1

[0044] S101: Obtain camera shooting images and microscope shooting images of a real multilayer microstructure material sample;

[0045] S102: Perform image processing on the microscope shooting images, extract the minimum repeating unit of each layer of medium, and store the minimum repeating unit as a map;

[0046] S103: Construct a multilayer microstructure material model based on SpongeCake, treat each layer of material as a volume medium, and there is no interface between layers; the optical properties of each layer of medium are determined by a phase function;

[0047] S104: Take the material parameters and the map of each layer of medium as input values of the multilayer microstructure material model based on SpongeCake; perform rendering on the multilayer microstructure material model based on SpongeCake, and the multilayer microstructure material model based on SpongeCake outputs a three-dimensional rendering image;

[0048] S105: Calculate a loss function between the three-dimensional rendering image and the camera shooting image during the rendering process, use an optimizer to update the material parameters of each layer of medium according to the loss function value, and gradually approach the real material parameters; when a set number of iterations is reached, output the material parameters of each layer of medium, and obtain a simulation material model of the real multilayer microstructure material sample.

[0049] Further, the S101: obtaining camera shooting images and microscope shooting images of a real multilayer microstructure material sample, comprises:

[0050] S101-1: Using a camera, shooting the real multilayer microstructure material sample to obtain camera shooting images;

[0051] S101-2: Using a microscope, shooting the real multilayer microstructure material sample to obtain microscope shooting images.

[0052] ​Exemplarily, as shown in FIG. 2(a) and FIG. 2(b), the sample size is between 20cm*20cm, and is placed on a 15cm high shelf. The camera and the microscope are alternately photographed, the camera is placed 5cm above the sample using a bracket, the camera sensitivity is 400, the exposure time is 1 / 400, and the shooting resolution is 4096*3072 pixels; the microscope is placed 2cm above the sample using a bracket, the magnification is 500 times, and the shooting resolution is 1535*2048 pixels. The camera photographing obtains a picture that can clearly distinguish the sample material color and smoothness, and the microscope photographing obtains a picture that can clearly distinguish the shape of the microstructure of each layer of material.

[0053] Exemplarily, the real multi-layer microstructure material sample includes: a mobile phone shell, a woven cloth, etc.

[0054] Further, as Figure 3 shown, the S102: image processing on the microscope photographing image, extracting the minimum repeating unit of each layer of medium, and storing the minimum repeating unit as a map, specifically including:

[0055] S102-1: Gaussian filtering processing is performed on the microscope photographing image to obtain an image without noise interference;

[0056] S102-2: contrast enhancement is performed on the image without noise interference to obtain a contrast-enhanced image;

[0057] S102-3: grayscale processing is performed on the contrast-enhanced image to obtain a grayscale image, and threshold segmentation processing is performed on the grayscale image to obtain a binary image;

[0058] S102-4: edge detection is performed on the binary image to obtain an edge detection image;

[0059] S102-5: frequency domain conversion is performed on the edge detection image to obtain a frequency domain image;

[0060] S102-6: according to the size and position of the minimum repeating unit in the frequency domain image, the microscope photographing image is segmented to obtain a segmented minimum repeating unit, and the segmented minimum repeating unit is stored as a map.

[0061] Further, the S102-2: contrast enhancement is performed on the image without noise interference to obtain a contrast-enhanced image, including:

[0062] An adaptive histogram equalization method is used to enhance the contrast of the image without noise interference to obtain a contrast-enhanced image.

[0063] The adaptive histogram equalization method is used to enhance the contrast of the image, so that the details of the microstructure are more clearly visible.

[0064] It should be understood that S102-3: the contrast-enhanced image is grayed to obtain a grayed image, and the grayed image is threshold segmented to obtain a binary image, which is converted to a gray image, and is binarized by threshold segmentation, so as to analyze the structure edge and shape feature in the subsequent.

[0065] Further, S102-4: edge detection is performed on the binary image to obtain an edge detection image, including: using a Canny edge detection algorithm to capture edge information of the microstructure in the image.

[0066] Further, S102-5: the edge detection image is converted to a frequency domain to obtain a frequency domain image, including: applying a fast Fourier transform (FFT) to the edge detection image to convert the image to a frequency domain, and identifying the periodic characteristics of the microstructure by observing the peak value distribution in the frequency spectrum. The distance between the position of the edge detection image corresponding to the peak frequency in the frequency spectrum and the center point of the edge detection image is the period, and the distance between the position of the edge detection image corresponding to the peak frequency in the frequency spectrum and the center point of the edge detection image is the size of the smallest repeating unit. At the same time, the phase information corresponding to the frequency component is obtained from the result of the Fourier transform, and the phase information corresponding to the frequency component is the position of the smallest repeating unit.

[0067] The smallest repeating unit refers to a smallest part of the periodic structure of the microstructure pattern of the material, which is a basic module for generating the entire microstructure pattern by tiling and repeating. For example, the microscope image of a certain woven fabric and its smallest repeating unit are shown in FIG. 4(a) and FIG. 4(b).

[0068] Further, S102-6: according to the size and position of the smallest repeating unit in the frequency domain image, the microscope image is segmented to obtain a segmented smallest repeating unit, and the segmented smallest repeating unit is stored as a texture map.

[0069] It should be understood that according to the size and position information obtained by frequency domain analysis, the original image is segmented to extract the repeating unit structure, and finally a binary structure image containing only the smallest repeating unit is generated, so as to perform subsequent material restoration and multi-layer structure analysis.

[0070] Further, S103: a multi-layer microstructure material model based on SpongeCake is constructed, each layer of material is regarded as a volume medium, and there is no interface between layers, wherein the volume medium refers to: a medium that fills a three-dimensional space and can interact with light. When the light passes through the medium, it is absorbed, scattered or emitted.

[0071] There is no interface between layers, which means that there may be an interface between layers of multi-layer materials, which can cause reflection or refraction when light passes from one layer of volume medium to another layer of medium, causing the direction of light to deviate. In the SpongeCake material model, it is considered that there is no additional interface between layers, which makes the light continue to propagate along the original path without changing direction when passing between two layers of medium.

[0072] The construction of a multi-layer microstructure material model based on SpongeCake includes: specifying the phase function, minimum repeating unit map, and reflectivity, roughness, thickness, and normal scaling strength for each layer of material.

[0073] SpongeCake refers to a material model for multi-layer material rendering, which can be composed of any number of volume media stacks, from Wang et al. 2022 paper "SpongeCake: A Layered Microflake Surface Appearance Model".

[0074] As Figure 5 shown, the multi-layer microstructure material model based on SpongeCake is composed of volume media layers connected from top to bottom, and each volume medium has its own material parameters.

[0075] Further, the S103: the optical properties of each layer of medium are determined by the phase function, including: using the microflake-based SGGX phase function or the Henyey-Greenstein phase function to define the scattering direction of light in each layer of medium. The microflake-based SGGX phase function is:

[0076]

[0077] where is the projected area of the microflake in the direction, the term is the normal distribution term defined by the roughness of SGGX and the input normal direction , which is:

[0078]

[0079]

[0080] The parameter controls the directionality of scattering, and the Henyey-Greenstein phase function is:

[0081]

[0082] wherein is the included angle between the incident light and the light scattering direction.

[0083] Further, the S104: taking the material parameters and the map of each layer of medium as the input values of the multi-layer microstructure material model based on SpongeCake; rendering the multi-layer microstructure material model based on SpongeCake, and the multi-layer microstructure material model based on SpongeCake outputs a three-dimensional rendering image; wherein the material parameters of each layer of medium include reflectivity, roughness, thickness and normal scaling intensity.

[0084] The reflectivity is used to describe the ability of the material to reflect light, which describes the proportion of light in the incident light that is reflected, and usually has three components corresponding to the RGB values in the image color respectively.

[0085] The roughness is used to describe the roughness of the material. If the SGGX phase function based on microsheet is used, it is a parameter ; if the Henyey-Greenstein phase function is used, it is a parameter .

[0086] The normal scaling intensity is used to control the parameter of the influence of the normal map on the surface of the object. The greater the intensity, the stronger the concave-convex sense of the object surface caused by the normal map. The normal direction after the influence of the normal intensity is:

[0087]

[0088]

[0089] wherein is the normal direction, are the xyz coordinate values of the normal direction obtained from the normal map respectively.

[0090] As the input values of the multi-layer microstructure material model based on SpongeCake, the material parameters of each layer of medium in the first round are in a set range, and random numbers are used as the input values. For example, when initializing the roughness of each layer of medium, a random number is generated in a reasonable range of 0.1 to 0.8 for initialization.

[0091] Further, as shown in Figure 6 , S105: calculating the loss function between the three-dimensional rendering image and the camera shooting image in the rendering process, using an optimizer to update the material parameters of each layer of medium according to the loss function value, gradually approaching the real material parameters, and outputting the material parameters of each layer of medium when the set number of iterations is reached, to obtain the simulation material model of the real multi-layer microstructure material sample; wherein the loss function has the specific expression:​

[0092] Loss function: Perceptual Loss and Mean Squared Error (MSE):

[0093]

[0094] is the perceptual loss, which calculates the feature difference between the rendered image and the camera image by extracting features from a pre-trained VGG-19 deep convolutional network:

[0095]

[0096] where is the result obtained by calculating the Gram matrix (eccentric covariance matrix) on the features obtained by inputting the corresponding image into the VGG-19 deep convolutional network.

[0097] is the mean squared error, which measures the difference between the rendered image and the real image at the pixel level:

[0098] .

[0099] VGG-19 deep convolutional network, whose network structure contains 19 hidden layers (16 convolutional layers and 3 fully connected layers), all convolutional kernels use 3*3 size, pooling kernels use 2*2 size, and Max pooling with stride=1, padding=0. The depth of the convolutional layer is 64, 128, 256, 512, and 512.

[0100] The pre-training goal of VGG-19 is to let the network learn the image classification task on the ImageNet dataset, so as to obtain features that can be transferred to other tasks. Loss function: Use cross-entropy loss function to optimize classification performance. Optimizer: Use stochastic gradient descent (SGD) with momentum term (usually set to 0.9) to accelerate convergence. Learning rate strategy: Usually use learning rate decay strategy, such as reducing learning rate during training to improve model stability. Regularization: To prevent overfitting, use L2 regularization and Dropout (on fully connected layers). Data augmentation: Data preprocessing includes random cropping, horizontal flipping, normalization, etc. to enhance the diversity of data.

[0101] Each convolutional layer of VGG-19 is stacked on the basis of the previous layer by a small convolutional kernel, gradually capturing more complex features, and the deeper the network, the more abstract the learned features. Low-level features are simple edges, corner points, color blocks, etc. in the image, and the corresponding high-level features are more abstract features, such as two pictures composed of repeated units tiled multiple times, the shape of the repeated unit and the number of repeated tiles, etc. are high-level features. When calculating the feature difference, the rendered image and the camera image are input into the VGG-19 network to extract features, and then the Gram matrix of the network output is calculated and the difference between the two is calculated as the feature difference.

[0102] The perceptual loss can capture the more sensitive high-level features of the human visual system, such as texture and detail level, while the mean square error provides pixel-level accuracy, and the combination of the two ensures that the rendered image is visually close to the real image. The perceptual loss can capture the more sensitive high-level features of the human visual system, such as texture and detail level, while the mean square error provides pixel-level accuracy, and the combination of the two ensures that the rendered image is visually close to the real image.

[0103] The randomly initialized parameters are rendered using the SpongeCake model, and the input parameters (optical properties of each layer of the microstructure multi-layer material) are calculated during the rendering process. Based on the back propagation algorithm, gradient information is provided during the optimization process, so that the rendering system can calculate the gradient of the output image with respect to the input material parameters, thereby realizing the optimization of the material parameters. The results obtained by rendering are compared with the images actually taken by the camera, and the loss function is calculated to measure the difference between the two.

[0104] The Adam optimizer is selected to optimize the loss function in step four. Through continuous iteration, the Adam optimizer updates the material parameters according to the feedback of the loss function, and gradually approaches the real material properties. The number of iterations is set to 250-300 times, and finally, the material parameters of each layer are output after optimization to obtain a highly fitted material model.

[0105] A microstructure multi-layer material recovery system based on the SpongeCake model is proposed. On the one hand, a collection device is designed to capture the minimum repeating unit of each layer of the multi-layer material through image processing methods; on the other hand, the SpongeCake model material model is used to express the microstructure multi-layer material and generate images through a renderer, which has a delicate and realistic multi-layer material appearance; finally, using differentiable rendering, manual appearance parameter matching is not required according to the captured image, and the material parameters of each layer can be automatically recovered according to the microstructure multi-layer material image.

[0106] Embodiment two

[0107] The embodiment provides a microstructure multilayer material recovery system, which comprises:

[0108] An acquisition module configured to acquire a camera shot image and a microscope shot image of a real multilayer microstructure material sample;

[0109] An image processing module configured to perform image processing on the microscope shot image, extract a minimum repeating unit of each layer of medium, and store the minimum repeating unit as a map;

[0110] A model construction module configured to construct a SpongeCake-based multilayer microstructure material model, take each layer of material as a volume medium, and there is no interface between layers; and determine the optical properties of each layer of medium by a phase function;

[0111] A rendering module configured to take the material parameters and the map of each layer of medium as input values of the SpongeCake-based multilayer microstructure material model; perform rendering on the SpongeCake-based multilayer microstructure material model, and output a three-dimensional rendering image based on the SpongeCake-based multilayer microstructure material model;

[0112] An output module configured to calculate a loss function between the three-dimensional rendering image and the camera shot image in the rendering process, use an optimizer to update the material parameters of each layer of medium according to the loss function value, and gradually approach the real material parameters; and output the material parameters of each layer of medium when a set number of iterations is reached, to obtain a simulation material model of the real multilayer microstructure material sample.

[0113] It should be noted that the above acquisition module, image processing module, model construction module, rendering module and output module correspond to steps S101 to S105 in the first embodiment, and the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above first embodiment. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0114] The description of each embodiment in the above embodiments has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0115] The proposed system can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the above modules is only a logical function division, and in actual implementation, there can be another division method, for example, a plurality of modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0116] Embodiment three

[0117] The embodiment also provides an electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected with the memory, the one or more computer programs are stored in the memory, and the processor executes the one or more computer programs stored in the memory to enable the electronic device to execute the method in the embodiment one.

[0118] It should be understood that, in the embodiment, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0119] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0120] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software.

[0121] The method in the embodiment one can be directly embodied as hardware processor execution completion, or executed by hardware and software modules in the processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0122] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software mode depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0123] Embodiment four

[0124] The embodiment also provides a computer readable storage medium for storing computer instructions, which are executed by a processor to complete the method in the embodiment one.

[0125] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. A method for recovering a microstructured multilayer material, characterized by, The application relates to a method for simulating a real multi-layer microstructure material sample. The method comprises the following steps: acquiring a camera shooting image and a microscope shooting image of the real multi-layer microstructure material sample; performing image processing on the microscope shooting image to extract a minimum repeating unit of each layer of medium, and storing the minimum repeating unit as a map; constructing a multi-layer microstructure material model based on SpongeCake, regarding each layer of material as a volume medium, and regarding the layers as not having interfaces; the optical properties of each layer of medium are determined by a phase function; taking the material parameters and the map of each layer of medium as input values of the multi-layer microstructure material model based on SpongeCake; performing rendering on the multi-layer microstructure material model based on SpongeCake, and outputting a three-dimensional rendering image based on the multi-layer microstructure material model based on SpongeCake; wherein the material parameters of each layer of medium include reflectivity, roughness, thickness and normal scaling intensity; calculating a loss function between the three-dimensional rendering image and the camera shooting image during the rendering process, and using an optimizer to update the material parameters of each layer of medium according to the loss function value, so as to gradually approach the real material parameters; when a set number of iterations is reached, the material parameters of each layer of medium are output, and a simulation material model of the real multi-layer microstructure material sample is obtained; wherein the loss function is specifically expressed as: is the perceptual loss, which computes the feature difference between the rendered image and the camera image using features extracted by a pre-trained VGG-19 deep convolutional network: wherein, to obtain a corresponding image After inputting the VGG-19 deep convolutional network to obtain the feature, the result obtained by calculating the Gram matrix of the feature; is the mean squared error, used to measure the difference between the rendered image and the real image at the pixel level: 。 2. The method of claim 1, wherein the microstructure multilayer material is a metal or an alloy. the loss function selects a perceptual loss and a mean square error (MSE); performing image processing on the microscope shooting image to extract a minimum repeating unit of each layer of medium, and storing the minimum repeating unit as a map, specifically comprising the following steps: performing Gaussian filtering processing on the microscope shooting image to obtain a noise interference-removed image; performing contrast enhancement on the noise interference-removed image to obtain a contrast-enhanced image; performing grayscale processing on the contrast-enhanced image to obtain a grayscale image, and performing threshold segmentation processing on the grayscale image to obtain a binary image; performing edge detection on the binary image to obtain an edge detection image; performing frequency domain conversion on the edge detection image to obtain a frequency domain image; 3. The method of claim 2, wherein the microstructure multilayer material is a metal or an alloy.

3. The method of claim 2, wherein the microstructure multilayer material is a metal or an alloy. segmenting the microscope shooting image according to the size and position of the minimum repeating unit in the frequency domain image to obtain segmented minimum repeating units, and storing the segmented minimum repeating units as a map. performing contrast enhancement on the noise interference-removed image to obtain a contrast-enhanced image, comprising the following steps: adopting an adaptive histogram equalization method to perform contrast enhancement on the noise interference-removed image to obtain a contrast-enhanced image; performing edge detection on the binary image to obtain an edge detection image, comprising the following steps: using a Canny edge detection algorithm to capture edge information of the microstructure in the image.

4. The method of claim 2, wherein the microstructure multilayer material is a metal or an alloy.

5. The method of claim 2, wherein the microstructure multilayer material is a metal or an alloy. The edge detection image is subjected to frequency domain conversion to obtain a frequency domain image, including: applying fast Fourier transform to the edge detection image to convert the image to the frequency domain, and identifying the periodic characteristics of the microstructure by observing the peak value distribution in the frequency spectrum, the distance between the position of the edge detection image corresponding to the peak frequency in the frequency spectrum and the center point of the edge detection image being the period, and the distance between the position of the edge detection image corresponding to the peak frequency in the frequency spectrum and the center point of the edge detection image being the size of the smallest repeating unit. Meanwhile, the phase information corresponding to the frequency component is obtained from the result of the Fourier transform, and the phase information corresponding to the frequency component is the position of the smallest repeating unit.

5. The method of claim 1, wherein the microstructure multilayer material is a metal or an alloy.

5. The method of claim 1, wherein the microstructure multilayer material is a metal or an alloy. The optical properties of each layer of medium are determined by a phase function, including: using a micro-particle-based SGGX phase function or a Henyey-Greenstein phase function to define the scattering direction of a light ray in each layer of medium.

6. A microstructured multilayer material recovery system, characterized by, The microstructure multilayer material recovery method according to any one of claims 1-5 comprises: An acquisition module configured to acquire a camera shot image and a microscope shot image of a real multilayer microstructure material sample; An image processing module configured to perform image processing on the microscope shot image, extract the smallest repeating unit of each layer of medium, and store the smallest repeating unit as a texture map; A model construction module configured to construct a multilayer microstructure material model based on SpongeCake, treat each layer of material as a volume medium, and there is no interface between layers; A rendering module configured to take the material parameters and the texture map of each layer of medium as input values of the multilayer microstructure material model based on SpongeCake, render the multilayer microstructure material model based on SpongeCake, and output a three-dimensional rendered image based on the multilayer microstructure material model based on SpongeCake; An output module configured to calculate a loss function between the three-dimensional rendered image and the camera shot image during rendering, use an optimizer to update the material parameters of each layer of medium based on the loss function value, and gradually approximate the real material parameters, and output the material parameters of each layer of medium when a set number of iterations is reached, to obtain a simulated material model of the real multilayer microstructure material sample.

7. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; and a processor for running the computer readable instructions, wherein the computer readable instructions, when executed by the processor, perform the method of any one of claims 1-5.

8. A storage medium characterized by, Non-transitory computer readable instructions, wherein when the non-transitory computer readable instructions are executed by a computer, the method of any one of claims 1-5 is performed.

Images