Pottery fragment virtual repair method based on pattern diffusion and graphics 3D technology

Abstract

The invention discloses a pottery fragment virtual restoration method based on a pattern diffusion and graphics 3D technology, and relates to the technical field of pottery fragment virtual restoration, and the method comprises the steps: obtaining a pottery fragment image through a high-resolution scanning device, carrying out the preprocessing of the pottery fragment image, and obtaining a high-quality fragment image set; extracting pattern, color and material information from the high-quality fragment image set by using an image processing algorithm, establishing a pattern library, and comparing edges and patterns of fragments in the high-quality fragment image set by using a computer vision algorithm to obtain a relative position relationship of the fragments; based on the high-quality fragment image set, a diffusion model method is adopted to generate a two-dimensional image for the missing part of the damaged fragments in the high-quality fragment image set, and a two-dimensional pottery picture is obtained; the method comprises the following steps: converting a two-dimensional pottery picture into a three-dimensional geometric model by utilizing a Figure 3D technology NeRF, and mapping the pattern and color information of the pottery in the two-dimensional pottery picture to the surface of the three-dimensional geometric model through a texture mapping technology Texture Mapping;

Description

Technical Field

[0001] The invention relates to the technical field of virtual restoration of pottery fragments, in particular to a virtual restoration method for pottery fragments based on pattern diffusion and image-generated 3D technology. Background Art

[0002] Virtual restoration of pottery fragments is a method of digitally reconstructing broken or damaged pottery using advanced technologies such as modern computer vision, image processing, machine learning, and 3D modeling. The technology obtains detailed information about pottery fragments through high-resolution scanning, uses advanced algorithms to extract patterns, colors, and material features, and establishes the relative position relationship between fragments, and then uses a diffusion model to generate a two-dimensional image of the missing part. Finally, it uses image-generated 3D technology and texture mapping to integrate this information into a realistic three-dimensional model.

[0003] Traditional methods have difficulty in effectively removing noise while retaining subtle features of fragments when dealing with high-precision scanning and preprocessing. In the process of pattern extraction and fragment matching, existing technologies often face low robustness and inaccuracy in complex backgrounds, especially when dealing with pottery fragments with complex patterns and material properties. At the same time, for the repair of severely damaged or missing parts, traditional methods often cannot generate natural and high-quality supplementary images. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a virtual restoration method for pottery fragments based on pattern diffusion and image-generated 3D technology to solve the problem that traditional methods are difficult to effectively remove noise while retaining subtle features of fragments when dealing with high-precision scanning and preprocessing, and in the process of pattern extraction and fragment matching, the existing technology often faces low robustness and inaccuracy in complex backgrounds, especially poor performance when dealing with pottery fragments with complex patterns and material properties.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology, which comprises:

[0008] Using high-resolution scanning equipment, the images of pottery fragments are obtained, and the images of pottery fragments are preprocessed to obtain a set of high-quality fragment images;

[0009] Image processing algorithms are used to extract pattern, color and material information from high-quality fragment image sets, and a pattern library is established. Computer vision algorithms are used to compare the edges and patterns of fragments in the high-quality fragment image sets to obtain the relative position relationship of the fragments.

[0010] Based on the high-quality fragment image set, the diffusion model method is used to generate a two-dimensional image of the missing part of the broken fragments in the high-quality fragment image set to obtain a two-dimensional pottery picture;

[0011] Using NeRF, a 3D image processing technology, the two-dimensional pottery image is converted into a three-dimensional geometric model. Then, the pattern and color information of the pottery in the two-dimensional pottery image is mapped to the surface of the three-dimensional geometric model through texture mapping technology to obtain a three-dimensional pottery model.

[0012] The rendering engine Blender Cycles is used to render the pottery 3D model, compare the pottery 3D model with the pottery fragment image, dynamically adjust the 3D pottery model, and output the optimized 3D pottery model.

[0013] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology of the present invention, wherein: the pottery fragment image is obtained by a high-resolution scanning device, and the pottery fragment image is preprocessed to obtain a high-quality fragment image set, and the specific steps are:

[0014] The pottery fragments were scanned using a high-resolution laser scanner;

[0015] A high-resolution laser scanner emits a laser beam onto the surface of the debris and obtains point cloud data based on the time and angle of reflected light;

[0016] Based on the point cloud data, a filtering algorithm based on local plane fitting is used to remove noise;

[0017] Adaptive histogram equalization (AHE) technology is used to enhance the image and obtain a high-quality fragment image set.

[0018] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology described in the present invention, wherein: the image processing algorithm is used to extract pattern, color and material information from the high-quality fragment image set, and a pattern library is established, and the edges and patterns of the fragments in the high-quality fragment image set are compared by a computer vision algorithm to obtain the relative position relationship of the fragments. The specific steps are:

[0019] The edge detection algorithm Canny is used to identify the pattern boundaries on pottery fragments in the high-quality fragment image set;

[0020] The scale-invariant feature transform (SIFT) algorithm is used to extract key points and descriptors within the pattern boundary.

[0021] The K-means clustering algorithm is used to segment the image into several regions with similar colors, and the color histogram of each region is calculated as its color feature. Based on the gray-level co-occurrence matrix GLCM, the texture features reflecting the surface roughness and directional characteristics are extracted.

[0022] Gather the extracted patterns, colors and material information to form a pattern library;

[0023] The feature matching algorithm FLANN is used to compare the feature descriptors between two images, find the matching fragment combination, and determine the best matching object by minimizing the matching error.

[0024] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology described in the present invention, wherein: based on the high-quality fragment image set, a diffusion model method is used to generate a two-dimensional image of the missing part of the damaged fragment in the high-quality fragment image set to obtain a two-dimensional pottery picture, and the specific steps are:

[0025] Select the pre-trained diffusion model StableDiffusion;

[0026] Convert the pattern, color and material information extracted from the high-quality fragment image set into a form suitable for the diffusion model StableDiffusion processing;

[0027] To map the image features to the latent space, let the feature vector be F;

[0028] Use the diffusion model to gradually generate images of the missing parts according to the given conditions.

[0029] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology of the present invention, the diffusion model is used to gradually generate an image of the missing part according to given conditions, and the specific steps are as follows:

[0030] Assume the initial noise image is N 0 , the final image obtained after T steps of iteration is I t ;

[0031] Seamlessly stitch the generated missing part with the existing fragmented image, and use the image stitching algorithm SIFT to adjust the position and size of the generated image;

[0032] Image enhancement technology was applied to optimize the stitching effect and obtain a two-dimensional pottery picture.

[0033] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology of the present invention, wherein: the two-dimensional pottery picture is converted into a three-dimensional geometric model by using the image-generated 3D technology NeRF, and the pattern and color information of the pottery in the two-dimensional pottery picture is mapped to the surface of the three-dimensional geometric model by the texture mapping technology TextureMapping to obtain a three-dimensional pottery model, and the specific steps are:

[0034] Preprocess the two-dimensional pottery images, including resizing, cropping, and color correction, and record the shooting angle of each image;

[0035] Train the NeRF model using a preprocessed 2D image set;

[0036] Using the trained NeRF model, we generate pottery images from any viewing angle by integrating the volume representation of the scene.

[0037] Based on the volume representation generated by NeRF, the MarchingCubes surface extraction algorithm is used to extract the three-dimensional surface mesh model of the pottery from the volume data.

[0038] UV unfold the generated 3D model and map the 3D surface onto a 2D plane;

[0039] Use TextureMapping technology to map the pattern and color information on the two-dimensional pottery image to the surface of the three-dimensional model;

[0040] Through the material editing tool Substance Painter, according to the actual material properties of the pottery fragments, including roughness and glossiness, the corresponding material attributes are added to the surface of the 3D model to obtain a 3D pottery model.

[0041] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology of the present invention, the three-dimensional pottery model is rendered using the rendering engine Blender Cycles, the three-dimensional pottery model is compared with the pottery fragment image, the three-dimensional pottery model is dynamically adjusted, and the optimized three-dimensional pottery model is output. The specific steps are:

[0042] Import the generated 3D pottery model into the rendering engine Blender Cycles;

[0043] Create a scene in Blender suitable for simulating real-world lighting conditions, including adding different types of light sources, and adjusting the position and intensity of the light sources to match the lighting conditions in the original pottery fragment image;

[0044] Adopt physical lighting model and physically based rendering PBR to enhance the realism of the model and calculate the lighting effect;

[0045] The 3D pottery model was fully rendered using the Cycles rendering engine;

[0046] Comparing the rendered three-dimensional pottery model image with the original pottery fragment image;

[0047] Based on the comparison results, dynamically adjust the geometric structure, texture mapping or material properties of the 3D model;

[0048] After completing the adjustments, save and export the final optimized 3D pottery model file.

[0049] As a preferred solution of the pottery fragment virtual restoration method based on pattern diffusion and image-generated 3D technology of the present invention, the specific steps of dynamically adjusting the geometric structure, texture mapping or material properties of the three-dimensional model based on the comparison result are as follows:

[0050] The difference between two images is calculated using the mean squared error (MSE) as a metric.

[0051] For the identified geometric parts that need to be adjusted, a surface smoothing algorithm based on curvature flow is used to fine-tune the geometric structure of the model and define a smoothness metric function;

[0052] After completing the adjustment of the geometric structure, the improved texture mapping technology is used to remap the pattern and color information to the surface of the three-dimensional model;

[0053] Based on the material properties of actual pottery, especially roughness and glossiness, the material properties are adjusted using the Physical Base Rendering (PBR) method;

[0054] After each adjustment, the above rendering and comparison process was repeated until the MSE value between the 3D model and the original pottery fragment image dropped to an acceptable range;

[0055] After completing all necessary adjustments and confirming that everything is correct, save the optimized 3D pottery model file and export a high-quality rendering image.

[0056] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology as described in the first aspect of the present invention is implemented.

[0057] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology as described in the first aspect of the present invention is implemented.

[0058] The beneficial effects of the present invention are as follows: by acquiring images of pottery fragments through high-resolution scanning equipment and preprocessing them, the conversion from physical fragments to digital high-quality image sets is realized; by using image processing algorithms such as the edge detection algorithm Canny and the scale-invariant feature transform algorithm SIFT, key information is extracted from the high-quality fragment image set and a pattern library is established; at the same time, a computer vision algorithm is used to compare the edges and patterns of the fragments to determine the relative position relationship, so that the precise positioning and preliminary splicing of the pottery fragments can be achieved, which greatly improves the efficiency and accuracy of fragment reconstruction; by mapping image features to latent space and gradually generating images of missing parts, the problem that traditional methods are difficult to handle severely damaged or missing fragments is solved; not only the blank areas between the fragments are filled to form a complete two-dimensional pottery picture, but also the degree of automation of the overall restoration work is improved, the labor cost and time consumption are reduced, and the work efficiency is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0060] Figure 1 This is a flow chart of the method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology in Example 1. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0064] Example 1, reference Figure 1 , which is the first embodiment of the present invention, provides a method for virtual restoration of pottery fragments based on pattern diffusion and 3D graphics technology, including the following steps:

[0065] S1. Obtain pottery fragment images through high-resolution scanning equipment, pre-process the pottery fragment images, and obtain a high-quality fragment image set;

[0066] Going a step further, the pottery fragments were scanned using a high-resolution laser scanner;

[0067] A high-resolution laser scanner emits a laser beam onto the surface of the debris and obtains point cloud data based on the time and angle of reflected light;

[0068] Based on the point cloud data, a filtering algorithm based on local plane fitting is used to remove noise;

[0069] Adaptive histogram equalization (AHE) technology is used to enhance the image and obtain a high-quality fragment image set.

[0070] It should be noted that the selection of a high-resolution laser scanner is crucial to obtaining high-quality point cloud data. Its accuracy directly affects the effects of subsequent image processing and restoration. The use of a filtering algorithm based on local plane fitting to remove noise can not only effectively reduce the random errors generated during the scanning process, but also retain the key detail features of the fragment surface. The adaptive histogram equalization (AHE) technology significantly improves the accuracy of edge detection and feature extraction by enhancing image contrast, ensuring the generation of a high-quality fragment image set.

[0071] S2. Use image processing algorithms to extract pattern, color and material information from high-quality fragment image sets, and establish a pattern library. Use computer vision algorithms to compare the edges and patterns of fragments in the high-quality fragment image sets to obtain the relative position relationship of the fragments.

[0072] Furthermore, the edge detection algorithm Canny was used to identify the pattern boundaries on the pottery fragments in the high-quality fragment image set;

[0073] The scale-invariant feature transform algorithm SIFT is used to extract key points and descriptors within the pattern boundary. The expression is:

[0074]

[0075] Where D(x,y) represents the characteristic response intensity at the coordinate (x,y), G σ and G kσ They represent the kernel functions of the Gaussian function at different scales, I(x,y) is the input image, and k is the scale factor;

[0076] The K-means clustering algorithm is used to segment the image into several regions with similar colors, and the color histogram of each region is calculated as its color feature. Based on the gray-level co-occurrence matrix GLCM, the texture features reflecting the surface roughness and directional characteristics are extracted. The expression is:

[0077]

[0078] Where T(i,j) represents the sum of the products of pixel values ​​at displacement (i,j), f(x,y) represents the grayscale value at coordinate (x,y) in the image, and M and N are the width and height of the image respectively;

[0079] Gather the extracted patterns, colors and material information to form a pattern library;

[0080] The feature matching algorithm FLANN is used to compare the feature descriptors between the two images, find the matching fragment combination, and determine the best matching pair by minimizing the matching error. The expression is:

[0081]

[0082] Where E represents the matching error, d i represents the feature distance difference in the i-th dimension, and n is the number of dimensions of the feature vector;

[0083] It should be noted that the combination of Canny edge detection and SIFT feature extraction can accurately identify pattern boundaries and key points on pottery fragments under complex backgrounds, and can maintain high robustness and accuracy even in areas with complex textures. The K-means clustering algorithm effectively captures the color distribution characteristics of different regions by segmenting the image and calculating the color histogram, providing a solid foundation for subsequent color matching and restoration. The gray-level co-occurrence matrix GLCM is used to extract texture features that reflect surface roughness and directionality. This information is particularly important for restoring the true texture of pottery. Compared with traditional brute force matching methods, the FLANN algorithm is more efficient on large-scale data sets, can significantly reduce calculation time, and improve the speed and accuracy of fragment matching.

[0084] S3, based on the high-quality fragment image set, a diffusion model method is used to generate a two-dimensional image of the missing parts of the broken fragments in the high-quality fragment image set to obtain a two-dimensional pottery picture;

[0085] Going further, we select the pre-trained diffusion model StableDiffusion;

[0086] Convert the pattern, color and material information extracted from the high-quality fragment image set into a form suitable for the diffusion model StableDiffusion processing;

[0087] To map the image features to the latent space, let the feature vector be F, the expression is:

[0088]

[0089] Among them, L(F) represents the representation of the feature vector F in the latent space, K(xx 0 ) is the kernel function used to achieve a smooth transition from the original feature space to the latent space, x 0 represents the position of the center point, and x traverses the entire domain;

[0090] Use the diffusion model to gradually generate images of the missing parts according to the given conditions;

[0091] Assume the initial noise image is N 0 , the final image obtained after T steps of iteration is I t , the expression is:

[0092]

[0093] Where I represents the image after the t-th iteration, C is the conditional information, L(.) is the loss function, which measures the difference between the generated image and the condition, and α is the learning rate;

[0094] Seamlessly stitch the generated missing part with the existing fragmented image, and use the image stitching algorithm SIFT to adjust the position and size of the generated image;

[0095] Image enhancement technology was applied to optimize the stitching effect and obtain a two-dimensional pottery image;

[0096] It should be noted that the pre-trained diffusion model StableDiffusion was chosen because it performs well in generating high-quality images, especially in dealing with complex patterns. The process of mapping image features to latent space is a key step in generating realistic images. Reasonable kernel function design can achieve a smooth transition from the original feature space to the latent space, thereby ensuring the quality of the generated image. By gradually denoising the image of the missing part, the unnatural effect caused by direct generation can be effectively avoided. This method relies on the initial noise image and optimizes the final result through multiple iterations to ensure that the generated two-dimensional pottery image is as close to the real state as possible.

[0097] S4, using NeRF, a 3D image processing technology, to convert the two-dimensional pottery image into a three-dimensional geometric model, and using texture mapping technology, to map the pattern and color information of the pottery in the two-dimensional pottery image to the surface of the three-dimensional geometric model, to obtain a three-dimensional pottery model;

[0098] Furthermore, the two-dimensional pottery images were preprocessed, including resizing, cropping, and color correction, and the shooting angle corresponding to each image was recorded;

[0099] Train the NeRF model using a preprocessed 2D image set;

[0100] Using the trained NeRF model, we generate pottery images from any viewing angle by integrating the volume representation of the scene.

[0101] Based on the volume representation generated by NeRF, the MarchingCubes surface extraction algorithm is used to extract the three-dimensional surface mesh model of the pottery from the volume data.

[0102] UV unfold the generated 3D model and map the 3D surface onto a 2D plane;

[0103] Use TextureMapping technology to map the pattern and color information on the two-dimensional pottery image to the surface of the three-dimensional model;

[0104] Using the material editing tool SubstancePainter, according to the actual material properties of the pottery fragments, including roughness and glossiness, the corresponding material attributes are added to the surface of the 3D model to obtain a 3D pottery model.

[0105] It should be noted that the training of the NeRF model requires a large number of viewing angles and high-quality input images. By performing volumetric integral calculations on the scene, realistic pottery images can be generated from different angles, which is crucial for 3D reconstruction. The MarchingCubes algorithm is used to extract surface meshes from volumetric data, and its accuracy directly affects the quality of the 3D model. The algorithm can efficiently handle complex geometric structures and ensure that the generated 3D model is as close to the real object as possible. UV unfolding technology allows the 3D surface to be mapped to a 2D plane, facilitating the application of texture mapping technology, while material editing tools such as SubstancePainter allow corresponding attributes to be added according to the material properties of the actual pottery, further enhancing the realism of the model.

[0106] S5, using the rendering engine BlenderCycles to render the three-dimensional pottery model, comparing the three-dimensional pottery model with the image of the pottery fragments, dynamically adjusting the three-dimensional pottery model, and outputting an optimized three-dimensional pottery model;

[0107] Furthermore, the generated three-dimensional pottery model was imported into the rendering engine BlenderCycles;

[0108] Create a scene in Blender suitable for simulating real-world lighting conditions, including adding different types of light sources, and adjusting the position and intensity of the light sources to match the lighting conditions in the original pottery fragment image;

[0109] The physical lighting model is used, based on physical rendering PBR, to enhance the realism of the model and calculate the lighting effect. The expression is:

[0110] L o (p,ω o )=L e (p,ω o )+∫ Ω f r (p,ω i ,ω o )L i (p,ω i )(ω i ·n)dω i ;

[0111] Among them, L o is the light intensity emitted from surface point p, L e is the spontaneous luminescence intensity of the point, f r is the reflectivity function, L i is the incident light intensity, ω i and ω o are the incident and outgoing directions respectively, and n is the surface normal;

[0112] The 3D pottery model was fully rendered using the Cycles rendering engine;

[0113] Comparing the rendered three-dimensional pottery model image with the original pottery fragment image;

[0114] Based on the comparison results, dynamically adjust the geometric structure, texture mapping or material properties of the 3D model;

[0115] Using mean square error MSE as a metric, the difference between the two images is calculated as follows:

[0116]

[0117] Where m and n are the height and width of the image, respectively, I(i,j) is the pixel value at position (i,j) of the original image, and R(i,j) is the pixel value of the rendered image at the same position.

[0118] For the identified geometric parts that need to be adjusted, a surface smoothing algorithm based on curvature flow is used to fine-tune the geometric structure of the model, and a smoothness metric function is defined, which is expressed as:

[0119]

[0120] in, represents the geometric boundary that needs to be adjusted, κ(s) is the curvature of the curve at the arc length parameter s;

[0121] After completing the adjustment of the geometric structure, the improved texture mapping technology is used to remap the pattern and color information to the surface of the three-dimensional model;

[0122] Based on the material properties of actual pottery, especially roughness and glossiness, the material properties are adjusted using the Physical Base Rendering (PBR) method;

[0123] After each adjustment, the above rendering and comparison process was repeated until the MSE value between the 3D model and the original pottery fragment image dropped to an acceptable range;

[0124] After completing all necessary adjustments and confirming that everything is correct, save the optimized 3D pottery model file and export a high-quality rendering image;

[0125] It should be noted that BlenderCycles, as a physically based rendering engine, can generate highly realistic three-dimensional model images by simulating real lighting effects. The mean square error (MSE), as a commonly used image quality evaluation indicator, can quantify the difference between two images. By continuously adjusting the model parameters to reduce the MSE value, the ideal restoration effect is gradually approached. The surface smoothing algorithm based on curvature flow can not only improve the smoothness of the geometric structure, but also maintain important shape features. It is particularly suitable for processing complex geometric structures. The physical base rendering (PBR) method can more realistically reproduce the surface texture of pottery by considering the optical properties of actual materials. This step is crucial to enhancing the realism of the final model. Repeat the rendering and comparison process after each adjustment until the MSE value between the three-dimensional model and the original pottery fragment image drops to an acceptable range to ensure that the final output three-dimensional pottery model achieves the best effect.

[0126] This embodiment also provides a computer device, which is suitable for the case of a virtual restoration method for pottery fragments based on pattern diffusion and 3D graphics technology, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the virtual restoration method for pottery fragments based on pattern diffusion and 3D graphics technology as proposed in the above embodiment.

[0127] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through Wi-Fi, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0128] The present embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0129] In summary, the present invention obtains images of pottery fragments through high-resolution scanning equipment and preprocesses them, thereby realizing the conversion from physical fragments to digital high-quality image sets. Image processing algorithms such as edge detection algorithm Canny and scale-invariant feature transform algorithm SIFT are used to extract key information from the high-quality fragment image set and establish a pattern library. At the same time, computer vision algorithms are used to compare the edges and patterns of the fragments to determine the relative position relationship. The precise positioning and preliminary splicing of pottery fragments can be achieved, which greatly improves the efficiency and accuracy of fragment reconstruction. By mapping image features to latent space and gradually generating images of missing parts, the problem that traditional methods are difficult to handle severely damaged or missing fragments is solved. Not only the blank areas between the fragments are filled to form a complete two-dimensional pottery picture, but also the degree of automation of the overall restoration work is improved, the labor cost and time consumption are reduced, and the work efficiency is significantly improved.

[0130] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A virtual restoration method for pottery fragments based on pattern diffusion and image-generated 3D technology, characterized in that: include: Using high-resolution scanning equipment, the images of pottery fragments are obtained, and the images of pottery fragments are preprocessed to obtain a set of high-quality fragment images; Image processing algorithms are used to extract pattern, color and material information from high-quality fragment image sets, and a pattern library is established. Computer vision algorithms are used to compare the edges and patterns of fragments in the high-quality fragment image sets to obtain the relative position relationship of the fragments. Based on the high-quality fragment image set, the diffusion model method is used to generate a two-dimensional image of the missing part of the broken fragments in the high-quality fragment image set to obtain a two-dimensional pottery picture; Using NeRF, a 3D image processing technology, the two-dimensional pottery image is converted into a three-dimensional geometric model. Then, the pattern and color information of the pottery in the two-dimensional pottery image is mapped to the surface of the three-dimensional geometric model through texture mapping technology to obtain a three-dimensional pottery model. The rendering engine Blender Cycles is used to render the pottery 3D model, compare the pottery 3D model with the pottery fragment image, dynamically adjust the 3D pottery model, and output the optimized 3D pottery model.

2. The method for virtual restoration of pottery fragments based on pattern diffusion and image generation 3D technology as claimed in claim 1, characterized in that: The pottery fragment images are obtained by high-resolution scanning equipment, and the pottery fragment images are preprocessed to obtain a high-quality fragment image set. The specific steps are: The pottery fragments were scanned using a high-resolution laser scanner; A high-resolution laser scanner emits a laser beam onto the surface of the debris and obtains point cloud data based on the time and angle of reflected light; Based on the point cloud data, a filtering algorithm based on local plane fitting is used to remove noise; Adaptive histogram equalization (AHE) technology is used to enhance the image and obtain a high-quality fragment image set.

3. The method for virtual restoration of pottery fragments based on pattern diffusion and 3D graphics technology as claimed in claim 2, characterized in that: The image processing algorithm is used to extract pattern, color and material information from the high-quality fragment image set, and a pattern library is established. The edges and patterns of the fragments in the high-quality fragment image set are compared by a computer vision algorithm to obtain the relative position relationship of the fragments. The specific steps are as follows: The edge detection algorithm Canny is used to identify the pattern boundaries on pottery fragments in the high-quality fragment image set; The scale-invariant feature transform (SIFT) algorithm is used to extract key points and descriptors within the pattern boundary. Use the K-means clustering algorithm to segment the image into several regions with similar colors, and calculate the color histogram of each region as its color feature. Based on the gray level co-occurrence matrix GLCM, texture features reflecting surface roughness and directional characteristics are extracted; Gather the extracted patterns, colors and material information to form a pattern library; The feature matching algorithm FLANN is used to compare the feature descriptors between two images, find the matching fragment combination, and determine the best matching object by minimizing the matching error.

4. The method for virtual restoration of pottery fragments based on pattern diffusion and 3D graphics technology as claimed in claim 3, characterized in that: Based on the high-quality fragment image set, a diffusion model method is used to generate a two-dimensional image of the missing part of the broken fragments in the high-quality fragment image set to obtain a two-dimensional pottery picture. The specific steps are: Select the pre-trained diffusion model Stable Diffusion; Convert the pattern, color and material information extracted from the high-quality fragment image set into a form suitable for the diffusion model StableDiffusion processing; To map the image features to the latent space, let the feature vector be F; Use the diffusion model to gradually generate images of the missing parts according to the given conditions.

5. The method for virtual restoration of pottery fragments based on pattern diffusion and image generation 3D technology as claimed in claim 4, characterized in that: The diffusion model is used to gradually generate an image of the missing part according to given conditions, and the specific steps are: Assume that the initial noise image is N0, and the final image obtained after T steps of iteration is I t ; Seamlessly stitch the generated missing part with the existing fragmented image, and use the image stitching algorithm SIFT to adjust the position and size of the generated image; Image enhancement technology was applied to optimize the stitching effect and obtain a two-dimensional pottery picture.

6. The method for virtual restoration of pottery fragments based on pattern diffusion and image generation 3D technology as claimed in claim 5, characterized in that: The NeRF technology is used to convert a two-dimensional pottery picture into a three-dimensional geometric model, and the pattern and color information of the pottery in the two-dimensional pottery picture is mapped to the surface of the three-dimensional geometric model through the texture mapping technology to obtain a three-dimensional pottery model. The specific steps are as follows: Preprocess the two-dimensional pottery images, including resizing, cropping, and color correction, and record the shooting angle of each image; Train the NeRF model using a preprocessed 2D image set; Using the trained NeRF model, we generate pottery images from any viewing angle by integrating the volume representation of the scene. Based on the volume representation generated by NeRF, the MarchingCubes surface extraction algorithm is used to extract the three-dimensional surface mesh model of the pottery from the volume data. UV unfold the generated 3D model and map the 3D surface onto a 2D plane; Use Texture Mapping technology to map the pattern and color information on the two-dimensional pottery image to the three-dimensional model surface; Through the material editing tool Substance Painter, according to the actual material properties of the pottery fragments, including roughness and glossiness, the corresponding material attributes are added to the surface of the 3D model to obtain a 3D pottery model.

7. The method for virtual restoration of pottery fragments based on pattern diffusion and 3D graphics technology as claimed in claim 6, characterized in that: The method of using the rendering engine Blender Cycles to render the pottery 3D model, comparing the pottery 3D model with the pottery fragment image, dynamically adjusting the 3D pottery model, and outputting the optimized 3D pottery model specifically comprises the following steps: Import the generated 3D pottery model into the rendering engine Blender Cycles; Create a scene in Blender suitable for simulating real-world lighting conditions, including adding different types of light sources, and adjusting the position and intensity of the light sources to match the lighting conditions in the original pottery fragment image; Adopt physical lighting model and physically based rendering PBR to enhance the realism of the model and calculate the lighting effect; The 3D pottery model was fully rendered using the Cycles rendering engine; Comparing the rendered three-dimensional pottery model image with the original pottery fragment image; Based on the comparison results, dynamically adjust the geometric structure, texture mapping or material properties of the 3D model; After completing the adjustments, save and export the final optimized 3D pottery model file.

8. The method for virtual restoration of pottery fragments based on pattern diffusion and image generation 3D technology as claimed in claim 7, characterized in that: The specific steps of dynamically adjusting the geometric structure, texture map or material properties of the three-dimensional model based on the comparison result are as follows: The difference between two images is calculated using the mean squared error (MSE) as a metric. For the identified geometric parts that need to be adjusted, a surface smoothing algorithm based on curvature flow is used to fine-tune the geometric structure of the model and define a smoothness metric function; After completing the adjustment of the geometric structure, the improved texture mapping technology is used to remap the pattern and color information to the surface of the three-dimensional model; Based on the material properties of actual pottery, especially roughness and glossiness, the material properties are adjusted using the Physical Base Rendering (PBR) method; After each adjustment, the above rendering and comparison process was repeated until the MSE value between the 3D model and the original pottery fragment image dropped to an acceptable range; After completing all necessary adjustments and confirming that everything is correct, save the optimized 3D pottery model file and export a high-quality rendering image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for virtual restoration of pottery fragments based on pattern diffusion and image-generated 3D technology described in any one of claims 1 to 7 are implemented.

Images