Microstructure multilayer material recovery method and system

Abstract

The invention discloses a microstructure multilayer material recovery method and system, and the method comprises the steps: constructing a SpongCage-based multilayer microstructure material model, taking each layer of material as a volume medium, and enabling no interface to exist between layers; the optical property of each layer of medium is determined through a phase function; the material parameters and the chartlet of each layer of medium are used as input values of a multilayer microstructure material model based on SpongCage; the multi-layer microstructure material model based on the SpongCage is rendered, and the multi-layer microstructure material model based on the SpongCage outputs a three-dimensional rendered image; calculating a loss function between the three-dimensional rendering image and the image shot by the camera in the rendering process, updating the material parameter of each layer of medium by using an optimizer according to the loss function value, gradually approaching the real material parameter, and outputting the material parameter of each layer of medium when the set iteration frequency is reached, and obtaining a simulation material model of the real multi-layer microstructure material sample.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics, and in particular to a method and system for restoring microstructured multi-layer materials. Background Art

[0002] The SpongeCake model is a physically based material model. The SpongeCake model compares the surface of an object in nature to a layer of medium. These media are evenly distributed, and there is no interface between the layers, which makes the refraction between the layers negligible. According to the position-free theory proposed by Guo et al. in 2018, the single scattered light in each layer of the medium in the SpongeCake model is accumulated to obtain the analytical single scattering formula:

[0003]

[0004] F is the Fresnel term, G is the attenuation term, and D represents the normal distribution term derived from the phase function of each layer of material. The commonly used phase function here is the micro-chip-based SGGX phase function or the Henyey-Greenstein phase function.

[0005] The Henyey-Greenstein phase function is a function used to simulate the scattering direction of light in participating media (such as atmosphere, smoke, clouds, etc.), which uses the parameter g to control the directionality of the scattering:

[0006]

[0007] The micro-flake-based SGGX phase function is a model used to simulate the scattering properties of complex materials (such as hair, skin, fabrics, etc.). It assumes that matter in nature is composed of tiny specular flakes, and uses various defined phase functions to describe the scattering direction of light after interacting with these micro-flakes, among which the SGGX method is the most common. The SGGX method simulates the scattering behavior of micro-flakes in the material, and defines the scattering direction of each micro-flake by a probability density function, thereby describing how light is scattered on these microstructures. The SGGX phase function uses an anisotropic Gaussian distribution (or ellipsoidal Gaussian distribution) to represent the distribution orientation of the micro-flakes. The normal direction of each micro-flake is randomly distributed, and its direction can be dynamically calculated based on the direction of the incident and outgoing light:

[0008]

[0009] Where σ(ω i ) is the microchip in ω i The projected area in the direction, and the D term is the normal distribution term defined by the roughness of SGGX.

[0010] Differentiable rendering is a technique that establishes a differentiable relationship between the parameters in the rendering process (such as materials, lighting, shapes, etc.) and the generated results of the image, so that the rendering process can back-propagate errors through gradient optimization. The traditional rendering process generates images from scene parameters, while in differentiable rendering, the scene parameters can be adjusted and optimized based on the errors generated by the image. This technology is of great significance to deep learning and computer vision because it allows the model to be directly optimized in the image space, thereby helping to solve inverse problems such as 3D reconstruction, lighting estimation, and material optimization. Summary of the invention

[0011] In order to address the deficiencies in the prior art, the present invention provides a method and system for restoring microstructured multi-layered materials; a multi-layered material restoration system with a microstructure is constructed, and the restored multi-layered material parameters can render realistic microstructured multi-layered material images, while achieving visual matching with the captured real material images.

[0012] On the one hand, a method for restoring a microstructured multilayer material is provided, comprising:

[0013] Obtain camera and microscope images of real multi-layer microstructure material samples;

[0014] Perform image processing on the images taken by the microscope, extract the minimum repeating unit of each layer of the medium, and store the minimum repeating unit as a map;

[0015] A multi-layer microstructure material model based on SpongCake is constructed, where each layer of material is regarded as a volume medium, and there is no interface between layers; the optical properties of each layer of medium are determined by the phase function;

[0016] The material parameters and textures of each layer of the medium are used as input values ​​of the multi-layer micro-structure material model based on SpongCake; the multi-layer micro-structure material model based on SpongCake is rendered, and a three-dimensional rendering image is output based on the multi-layer micro-structure material model based on SpongCake;

[0017] The loss function between the 3D rendered image and the camera image during the rendering process is calculated, and the material parameters of each layer of the medium are updated using the optimizer according to the loss function value, gradually approaching the real material parameters. When the set number of iterations is reached, the material parameters of each layer of the medium are output to obtain a simulated material model of the real multi-layer microstructure material sample.

[0018] On the other hand, a microstructure multi-layer material restoration system is provided, comprising:

[0019] An acquisition module, which is configured to: acquire a camera-photographed image and a microscope-photographed image of a real multi-layer microstructure material sample;

[0020] An image processing module is configured to: perform image processing on the image taken by the microscope, extract the minimum repeating unit of each layer of the medium, and store the minimum repeating unit as a map;

[0021] A model building module, which is configured to: build a multi-layer microstructure material model based on SpongCake, regard each layer of material as a volume medium, and there is no interface between layers; the optical properties of each layer of the medium are determined by a phase function;

[0022] A rendering module is configured to: use the material parameters and textures of each layer of the medium as input values ​​of the multi-layer micro-structure material model based on SpongCake; render the multi-layer micro-structure material model based on SpongCake, and output a three-dimensional rendering image based on the multi-layer micro-structure material model based on SpongCake;

[0023] The output module is configured to: calculate the loss function between the three-dimensional rendered image and the camera-captured image during the rendering process, use the optimizer to update the material parameters of each layer of the medium according to the loss function value, gradually approach the real material parameters, and when the set number of iterations is reached, output the material parameters of each layer of the medium to obtain a simulated material model of the real multi-layer microstructure material sample.

[0024] On the other hand, there is also provided an electronic device, comprising:

[0025] a memory for non-transitory storage of computer-readable instructions; and

[0026] a processor for executing the computer readable instructions,

[0027] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

[0028] On the other hand, a storage medium is provided, which non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the method described in the first aspect is executed.

[0029] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is used to implement the method described in the first aspect when running on one or more processors.

[0030] The above technical solution has the following advantages or beneficial effects:

[0031] The present invention is a multi-layer microstructure material restoration system based on the SpongeCake model, and the rendered multi-layer material image is delicate and realistic.

[0032] The present invention uses a camera and a microscope to capture multi-scale materials, and combines a variety of image processing technologies to obtain the minimum repeating unit of the real material microstructure, making it possible to render microstructure multi-layer materials.

[0033] The present invention adopts differentiable rendering and realizes automatic parameter recovery of the real microstructure multi-layer material acquisition image through parameter gradient return, which improves the realism and convenience of multi-layer microstructure resignation rendering. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0035] Figure 1 It is a schematic diagram of the process of the microstructure multi-layer material restoration system based on the SpongeCake model of the present invention.

[0036] FIG. 2( a ) and FIG. 2( b ) are schematic diagrams of the camera and microscope acquisition equipment described in step 1 of the method of the present invention.

[0037] Figure 3 A detailed flow chart of generating the minimum repeating unit for processing the microscope-photographed data described in step 2 of the method of the present invention.

[0038] FIG. 4( a ) and FIG. 4( b ) are schematic diagrams of images captured by a microscope and a minimum repeating unit in the method of the present invention.

[0039] Figure 5 Schematic diagram of layered material design for the SpongeCake model in the method of the present invention.

[0040] Figure 6 A detailed flowchart for recovering multi-layer material parameters for differentiable rendering in the method of the present invention. DETAILED DESCRIPTION

[0041] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0042] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments of the present invention. The terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] Embodiment 1, as Figure 1 As shown, this embodiment provides a method for restoring a microstructured multilayer material, comprising:

[0044] S101: Acquire a camera image and a microscope image of a real multi-layer microstructure material sample;

[0045] S102: performing image processing on the image taken by the microscope, extracting the minimum repeating unit of each layer of the medium, and storing the minimum repeating unit as a map;

[0046] S103: Construct a multi-layer microstructure material model based on SpongCake, treating each layer of material as a volume medium with no interface between layers; the optical properties of each layer of medium are determined by the phase function;

[0047] S104: using the material parameters and texture of each layer of the medium as input values ​​of the multi-layer micro-structure material model based on SpongCake; rendering the multi-layer micro-structure material model based on SpongCake, and outputting a three-dimensional rendering image based on the multi-layer micro-structure material model based on SpongCake;

[0048] S105: Calculate the loss function between the three-dimensional rendered image and the camera-captured image during the rendering process, use an optimizer to update the material parameters of each layer of the medium according to the loss function value, and gradually approach the real material parameters. When the set number of iterations is reached, output the material parameters of each layer of the medium to obtain a simulated material model of the real multi-layer microstructure material sample.

[0049] Furthermore, the S101: obtaining a camera image and a microscope image of a real multi-layer microstructure material sample includes:

[0050] S101-1: Use a camera to photograph a real multi-layer microstructure material sample to obtain a camera photographed image;

[0051] S101-2: Use a microscope to photograph a real multi-layer microstructure material sample to obtain a microscope photographed image.

[0052] For example, as shown in Figure 2(a) and Figure 2(b), the sample size is between 20cm*20cm and is placed on a 15cm high shelf. The camera and the microscope are alternately photographed. The camera is placed 5cm above the sample using a bracket. The camera sensitivity is 400, the exposure time is 1 / 400, and the shooting resolution is 4096*3072 pixels. The microscope is placed 2cm above the sample using a bracket, the magnification is 500 times, and the shooting resolution is 1535*2048 pixels. The camera can clearly distinguish the color and smoothness of the sample material, and the microscope can clearly distinguish the microstructure shape of each layer of material.

[0053] Exemplarily, the real multi-layer microstructure material samples include: mobile phone casing, woven fabric, etc.

[0054] Furthermore, if Figure 3 As shown, the S102: performing image processing on the image taken by the microscope, extracting the minimum repeating unit of each layer of the medium, and storing the minimum repeating unit as a map, specifically includes:

[0055] S102-1: Performing Gaussian filtering on the image captured by the microscope to obtain an image without noise interference;

[0056] S102-2: performing contrast enhancement on the image after noise interference is removed to obtain a contrast enhanced image;

[0057] S102-3: graying the contrast enhanced image to obtain a grayed image, and performing threshold segmentation on the grayed image to obtain a binary image;

[0058] S102-4: Perform edge detection on the binary image to obtain an edge detection image;

[0059] S102-5: Perform frequency domain conversion on the edge detection image to obtain a frequency domain image;

[0060] S102-6: Segment the image captured by the microscope according to the size and position of the minimum repeating unit in the frequency domain image to obtain the segmented minimum repeating unit, and store the segmented minimum repeating unit as a map.

[0061] Furthermore, the step S102-2: performing contrast enhancement on the image from which noise interference is removed to obtain a contrast enhanced image includes:

[0062] Adaptive histogram equalization is used to perform contrast enhancement on the image with noise removed to obtain a contrast enhanced image.

[0063] Adaptive histogram equalization method is used to enhance image contrast and make the details of tiny structures more clearly visible.

[0064] It should be understood that S102-3: graying the contrast enhanced image to obtain a grayscale image, and performing threshold segmentation on the grayscale image to obtain a binary image, is to convert the image into a grayscale image and binarize it by threshold segmentation to facilitate subsequent analysis of structural edge and shape features.

[0065] Furthermore, the S102-4: performing edge detection on the binary image to obtain an edge detected image includes: using a Canny edge detection algorithm to capture edge information of microstructures in the image.

[0066] Furthermore, the S102-5: performing frequency domain conversion on the edge detection image to obtain a frequency domain image, includes: applying fast Fourier transform (FFT) to the image after edge detection, converting the image to the frequency domain, and identifying the periodic characteristics of the microstructure by observing the peak distribution in the spectrum, the distance between the position of the edge detection image corresponding to the peak frequency in the spectrum and the center point of the edge detection image is the period, the distance between the position of the edge detection image corresponding to the peak frequency in the spectrum and the center point of the edge detection image is the size of the minimum repeating unit, and at the same time, from the result of Fourier transform, obtaining the phase information corresponding to the frequency component, the phase information corresponding to the frequency component is the position of the minimum repeating unit.

[0067] The minimum repeating unit refers to a minimum part of the periodic structure of the microstructure pattern of the material, and is the basic module for generating the entire microstructure pattern by tiling and repeating. Taking a woven fabric as an example, its microscope image and its minimum repeating unit are shown in Figures 4(a) and 4(b).

[0068] Further, S102-6: segment the image captured by the microscope according to the size and position of the minimum repeating unit in the frequency domain image to obtain the segmented minimum repeating unit, and store the segmented minimum repeating unit as a map.

[0069] It should be understood that the original image is segmented according to the size and position information obtained by frequency domain analysis, the repeated unit structure is extracted, and finally a binary structure image containing only the minimum repeated unit is generated to facilitate subsequent material restoration and multi-layer structure analysis.

[0070] Furthermore, the S103: constructs a multi-layer microstructure material model based on SpongCake, and regards each layer of material as a volume medium, and there is no interface between the layers, wherein the volume medium refers to: a medium that fills the three-dimensional space and can interact with light, and the light is absorbed, scattered or emitted when passing through the medium.

[0071] There is no interface between layers, which means that there may be interfaces between layers of multi-layer materials, causing light to reflect or refract when passing from one volume medium to another, causing the direction of the light to be deflected. In the SpongeCake material model, it is assumed that there is no additional interface between layers, so that light can continue to propagate along the original path without changing direction when passing between two layers of media.

[0072] Build a multi-layer microstructure material model based on SpongCake, including: specifying the phase function, minimum repeating unit map, reflectivity, roughness, thickness and normal scaling strength used by each layer of material.

[0073] SpongCake refers to a material model for multi-layer material rendering, which can be composed of any number of volume media stacks. It comes from Wang et al.'s 2022 paper "SpongeCake: A Layered Microflake Surface Appearance Model".

[0074] like Figure 5 As shown, the multi-layer microstructure material model based on SpongCake is composed of volume medium layers connected sequentially from top to bottom, and each layer of volume medium has its own material parameters.

[0075] Furthermore, the optical properties of each layer of the medium are determined by a phase function, including: using a microchip-based SGGX phase function or a Henyey-Greenstein phase function to define the scattering direction of light in each layer of the medium. The material parameters of each layer of the medium include reflectivity, roughness, thickness, and normal scaling intensity.

[0076] The microchip-based SGGX phase function is:

[0077]

[0078] Where σ(ω i ) is the microchip in ω i The projected area in the direction, D is the roughness σ of SGGX and the input normal direction ω m The normal distribution term is defined as:

[0079]

[0080] Using the parameter g to control the directionality of the scattering, the Henyey-Greenstein phase function is:

[0081]

[0082] Where θ is the angle between the incident light and the direction in which the light is scattered.

[0083] Furthermore, the S104: uses the material parameters and texture of each layer of the medium as input values ​​of the multi-layer microstructure material model based on SpongCake; renders the multi-layer microstructure material model based on SpongCake, and outputs a three-dimensional rendered image based on the multi-layer microstructure material model based on SpongCake; wherein the material parameters of each layer of the medium include: reflectivity, roughness, thickness and normal scaling strength.

[0084] The reflectivity is used to describe the ability of a material to reflect light. It describes the proportion of incident light that is reflected, and usually has three components, corresponding to the RGB values ​​in the image color.

[0085] The roughness is used to describe the roughness of the material. If the micro-sheet-based SGGX phase function is used, it is the parameter σ; if the Henyey-Greenstein phase function is used, it is the parameter g.

[0086] The normal scaling strength is a parameter used to control the effect of the normal map on the surface of the object. The greater the strength, the stronger the concave and convex feeling of the object surface made by the normal map. The normal direction after the influence of the normal strength β is:

[0087]

[0088] where ω n is the normal direction, n x n y n z They are the xyz coordinate values ​​of the normal direction obtained from the normal map.

[0089] As the input value of the multi-layer microstructure material model based on SpongCake, the material parameters of each layer of the medium in the first round are within the set range, and random numbers are used as input values. For example, when initializing the roughness of each layer of the medium, a random number is generated within a reasonable range of 0.1 to 0.8 for initialization.

[0090] Furthermore, if Figure 6 As shown, S105: calculate the loss function between the 3D rendered image and the camera captured image during the rendering process, use the optimizer to update the material parameters of each layer of the medium according to the loss function value, gradually approach the real material parameters, and when the set number of iterations is reached, output the material parameters of each layer of the medium to obtain a simulated material model of the real multi-layer microstructure material sample; wherein the loss function is specifically expressed as:

[0091] The loss function selects perceptual loss (Perceptual Loss) and mean square error (MSE):

[0092] L=L perceptual +L MSE

[0093] L perceptual is the perceptual loss, which is calculated by extracting features from the pre-trained VGG-19 deep convolutional network to render the image I rendered and camera image I real The characteristic differences between:

[0094] L perceptual =||Gram(I rendered )-Gram(I real )||,

[0095] Among them, Gram(I) is the result of calculating the Gram matrix (eccentric covariance matrix) for the features obtained by inputting the corresponding image I into the VGG-19 deep convolutional network.

[0096] L MSE is the mean square error, which is used to measure the rendered image I rendered and the real image I real Differences at the pixel level:

[0097]

[0098] The VGG-19 deep convolutional network has 19 hidden layers (16 convolutional layers and 3 fully connected layers). All convolution kernels are 3*3 in size, and all pooling kernels are 2*2 in size. Max pooling with stride=1 and padding=0 is used. The depths of the convolutional layers are 64, 128, 256, 512, and 512, respectively.

[0099] The pre-training goal of VGG-19 is to allow the network to learn image classification tasks on the ImageNet dataset, so as to obtain features that can be transferred to other tasks. Loss function: The cross entropy loss function is used to optimize the classification performance. Optimizer: Use stochastic gradient descent (SGD) with a momentum term (usually set to 0.9) to accelerate convergence. Learning rate strategy: A learning rate decay strategy is usually adopted, such as reducing the learning rate during training to improve the stability of the model. Regularization: To prevent overfitting, L2 regularization and Dropout (on the fully connected layer) are used. Data enhancement: Data preprocessing includes random cropping, horizontal flipping, normalization, etc. to enhance data diversity.

[0100] Each convolution layer of VGG-19 gradually captures more complex features by stacking small convolution kernels on the previous layer, and the deeper the network, the more abstract the features learned. Low-level features are simple edges, corners, color blocks, etc. in the image, and the corresponding high-level features are relatively abstract features. For example, both images are composed of repeated units tiled repeatedly, and the shape of the repeated units and the number of repeated tiles are all high-level features. When calculating feature differences, both the rendered image and the camera image are input into the VGG-19 network to extract features, and then the Gram matrix is ​​calculated using the eigenvalues ​​output by the network and the difference between the two is calculated as the feature difference.

[0101] The perceptual loss captures high-level features that the human visual system is more sensitive to, such as texture and level of detail, while the mean squared error provides pixel-level accuracy. The combination of the two ensures that the rendered image is visually close to the real image. The perceptual loss captures high-level features that the human visual system is more sensitive to, such as texture and level of detail, while the mean squared error provides pixel-level accuracy. The combination of the two ensures that the rendered image is visually close to the real image.

[0102] The SpongeCake model is used to perform differentiable rendering on randomly initialized parameters. During the rendering process, the input parameters (optical properties of each layer of the microstructure multi-layer material) are differentiated. Based on the back-propagation algorithm, gradient information is provided during the optimization process, so that the rendering system can calculate the gradient of the output image relative to the input material parameters, thereby optimizing the material parameters. The rendered results are compared with the images actually taken by the camera, and the loss function is calculated to measure the difference between the two.

[0103] The Adam optimizer is used to optimize the loss function in step 4. Through continuous iteration, the Adam optimizer updates the material parameters according to the feedback of the loss function, gradually approaching the real material properties. The number of iterations is set to 250-300 times. Finally, after the optimization is completed, the material parameters of each layer will be output to obtain a highly fitted material model.

[0104] A microstructure multi-layer material restoration system based on the SpongeCake model is proposed. On the one hand, the acquisition equipment is designed to capture the microstructure minimum repeating unit of each layer of the multi-layer material through image processing methods; on the other hand, the SpongeCake model material model is used to express the microstructure multi-layer material and the image is generated through the renderer, which has a delicate and realistic multi-layer material appearance; finally, differentiable rendering is used, and there is no need to manually match the appearance parameters according to the captured image. The material parameters of each layer can be automatically restored based on the microstructure multi-layer material image.

[0105] Embodiment 2

[0106] This embodiment provides a microstructure multi-layer material restoration system, including:

[0107] An acquisition module, which is configured to: acquire a camera-photographed image and a microscope-photographed image of a real multi-layer microstructure material sample;

[0108] An image processing module is configured to: perform image processing on the image taken by the microscope, extract the minimum repeating unit of each layer of the medium, and store the minimum repeating unit as a map;

[0109] A model building module, which is configured to: build a multi-layer microstructure material model based on SpongCake, regard each layer of material as a volume medium, and there is no interface between layers; the optical properties of each layer of the medium are determined by a phase function;

[0110] A rendering module is configured to: use the material parameters and textures of each layer of the medium as input values ​​of the multi-layer micro-structure material model based on SpongCake; render the multi-layer micro-structure material model based on SpongCake, and output a three-dimensional rendering image based on the multi-layer micro-structure material model based on SpongCake;

[0111] The output module is configured to: calculate the loss function between the three-dimensional rendered image and the camera-captured image during the rendering process, use the optimizer to update the material parameters of each layer of the medium according to the loss function value, gradually approach the real material parameters, and when the set number of iterations is reached, output the material parameters of each layer of the medium to obtain a simulated material model of the real multi-layer microstructure material sample.

[0112] It should be noted that the acquisition module, image processing module, model building module, rendering module and output module described above correspond to steps S101 to S105 in Embodiment 1, and the examples and application scenarios implemented by the modules and corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules described above as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0113] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0114] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0115] Embodiment 3

[0116] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory so that the electronic device executes the method described in the above embodiment one.

[0117] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0118] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0119] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0120] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0121] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0122] Embodiment 4

[0123] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is completed.

[0124] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for restoring microstructured multilayer materials, characterized in that: include: Obtain camera and microscope images of real multi-layer microstructure material samples; Perform image processing on the images taken by the microscope, extract the minimum repeating unit of each layer of the medium, and store the minimum repeating unit as a map; A multi-layer microstructure material model based on SpongCake is constructed, where each layer of material is regarded as a volume medium, and there is no interface between layers; the optical properties of each layer of medium are determined by the phase function; The material parameters and textures of each layer of the medium are used as input values ​​of the multi-layer micro-structure material model based on SpongCake; the multi-layer micro-structure material model based on SpongCake is rendered, and a three-dimensional rendering image is output based on the multi-layer micro-structure material model based on SpongCake; The loss function between the 3D rendered image and the camera image during the rendering process is calculated, and the material parameters of each layer of the medium are updated using the optimizer according to the loss function value, gradually approaching the real material parameters. When the set number of iterations is reached, the material parameters of each layer of the medium are output to obtain a simulated material model of the real multi-layer microstructure material sample.

2. The method for restoring microstructured multi-layered materials according to claim 1, characterized in that: Perform image processing on the images taken by the microscope, extract the minimum repeating unit of each layer of the medium, and store the minimum repeating unit as a map, including: Perform Gaussian filtering on the images taken by the microscope to obtain images without noise interference; Perform contrast enhancement on the image after removing noise interference to obtain a contrast enhanced image; Graying the contrast enhanced image to obtain a grayed image, and performing threshold segmentation on the grayed image to obtain a binary image; Perform edge detection on the binary image to obtain an edge detection image; Perform frequency domain conversion on the edge detection image to obtain a frequency domain image; According to the size and position of the minimum repeating unit in the frequency domain image, the image taken by the microscope is segmented to obtain the segmented minimum repeating unit, and the segmented minimum repeating unit is stored as a map.

3. The method for restoring microstructured multi-layered materials as claimed in claim 2, characterized in that: The image after noise interference is removed is contrast enhanced to obtain a contrast enhanced image, including: using an adaptive histogram equalization method to enhance the contrast of the image after noise interference is removed to obtain a contrast enhanced image; edge detection is performed on the binarized image to obtain an edge detected image, including: using a Canny edge detection algorithm to capture edge information of microstructures in the image.

4. The method for restoring microstructured multi-layered materials as claimed in claim 2, characterized in that: The edge detection image is converted into a frequency domain to obtain a frequency domain image, including: applying a fast Fourier transform to the image after edge detection, converting the image into the frequency domain, and identifying the periodic characteristics of the microstructure by observing the peak distribution in the spectrum, the distance between the position of the edge detection image corresponding to the peak frequency in the spectrum and the center point of the edge detection image is the period, the distance between the position of the edge detection image corresponding to the peak frequency in the spectrum and the center point of the edge detection image is the size of the minimum repeating unit, and at the same time, from the result of the Fourier transform, the phase information corresponding to the frequency component is obtained, and the phase information corresponding to the frequency component is the position of the minimum repeating unit.

5. The method for restoring microstructured multi-layered materials according to claim 1, characterized in that: The optical properties of each layer of the medium are determined by a phase function, including: using a microchip-based SGGX phase function or a Henyey-Greenstein phase function to define the scattering direction of light in each layer of the medium; the material parameters of each layer of the medium include: reflectivity, roughness, thickness, and normal scaling intensity.

6. The method for restoring microstructured multi-layered materials according to claim 1, characterized in that: The material parameters and textures of each layer of the medium are used as input values ​​of the multi-layer microstructure material model based on SpongCake; the multi-layer microstructure material model based on SpongCake is rendered, and a three-dimensional rendering image is output based on the multi-layer microstructure material model based on SpongCake; wherein the material parameters of each layer of the medium include: reflectivity, roughness, thickness and normal scaling strength.

7. The method for restoring microstructured multi-layered materials as claimed in claim 1, characterized in that: The loss function between the 3D rendered image and the camera-captured image is calculated during the rendering process. The material parameters of each layer of the medium are updated using the optimizer according to the loss function value, gradually approaching the real material parameters. When the set number of iterations is reached, the material parameters of each layer of the medium are output to obtain a simulated material model of the real multi-layer microstructure material sample. The loss function is specifically expressed as follows: The loss function selects perceptual loss and mean square error MSE: L=L perceptual +L MSE L perceptual is the perceptual loss, which is calculated by extracting features from the pre-trained VGG-19 deep convolutional network to render the image I rendered and camera image I real The characteristic differences between: L perceptual =||Gram(I rendered )-Gram(I real )||, Wherein, Gram(I) is the result obtained by calculating the Gram matrix for the features obtained by inputting the corresponding image I into the VGG-19 deep convolutional network; L MSE is the mean square error, which is used to measure the rendered image I rendered and the real image I real Differences at the pixel level:

8. A microstructured multi-layer material restoration system, characterized in that: include: An acquisition module, which is configured to: acquire a camera-photographed image and a microscope-photographed image of a real multi-layer microstructure material sample; An image processing module is configured to: perform image processing on the image taken by the microscope, extract the minimum repeating unit of each layer of the medium, and store the minimum repeating unit as a map; A model building module, which is configured to: build a multi-layer microstructure material model based on SpongCake, regard each layer of material as a volume medium, and there is no interface between layers; the optical properties of each layer of the medium are determined by a phase function; A rendering module is configured to: use the material parameters and textures of each layer of the medium as input values ​​of the multi-layer micro-structure material model based on SpongCake; render the multi-layer micro-structure material model based on SpongCake, and output a three-dimensional rendering image based on the multi-layer micro-structure material model based on SpongCake; The output module is configured to: calculate the loss function between the three-dimensional rendered image and the camera-captured image during the rendering process, use the optimizer to update the material parameters of each layer of the medium according to the loss function value, gradually approach the real material parameters, and when the set number of iterations is reached, output the material parameters of each layer of the medium to obtain a simulated material model of the real multi-layer microstructure material sample.

9. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; as well as a processor for executing the computer readable instructions, When the computer-readable instructions are executed by the processor, the method described in any one of claims 1 to 7 is executed.

10. A storage medium, characterized in that: The computer-readable instructions are non-transitory stored, wherein when the non-transitory computer-readable instructions are executed by a computer, the method according to any one of claims 1 to 7 is performed.

Images