A three-dimensional pattern retrieval method based on style consistency grid flattening

Abstract

This invention discloses a 3D pattern retrieval method based on style-consistent mesh flattening. The method includes the following steps: S1, obtaining a pattern dataset from 2D cultural relic images; S2, obtaining a 3D model through 3D reconstruction technology, followed by preprocessing of the 3D model; S3, preprocessing the 3D textures using style-consistent flattening technology to obtain a fully flattened 2D image of the 3D textures; S4, extracting features from the preprocessed flattened 3D image using Transformer and visual labeling methods; S5, calculating the feature distance between 3D textures based on the extracted features, using the feature distance as the similarity between 3D textures, and selecting the top N 3D models as the results. This invention achieves the expected accuracy in retrieving 3D patterns from cultural relics with different configurations, demonstrating good versatility and enabling the flattening and retrieval of 3D patterns from cultural relics, regardless of the format, lighting, material, or resolution of the 3D model.

Description

Technical Field

[0001] This invention relates to the field of computer graphics technology, specifically to a three-dimensional pattern retrieval method based on style-consistent grid flattening. Background Technology

[0002] China possesses an extremely rich reserve of cultural relics, with significant regional variations. These relics encompass a wide range of types, including pottery, porcelain, ancient architecture, bronzes, jade, and paintings, and their total quantity is on the rise. In the digital and intelligent era, the standards and technical requirements for the digitization of cultural relics are increasingly stringent. The research scope has expanded from the macroscopic overall form and image data of cultural relics to the microscopic details of patterns, color layers, and craftsmanship. This provides solid data support for deepening the interpretation of the value of cultural relics and innovating their utilization. The combination of cultural relic digitization and deep neural network technology in the management of three-dimensional cultural relics aims to promote the modernization of cultural relic information systems.

[0003] The shortcomings of existing technology:

[0004] Currently, when studying three-dimensional traditional patterns, it is necessary to extract meaningful patterns from the three-dimensional model, thereby promoting the construction of the three databases in the cultural big data system. Unlike image retrieval, three-dimensional pattern retrieval requires retrieval from three-dimensional to image to pattern image. However, the surface of most three-dimensional objects is curved, and how to retrieve three-dimensional patterns has not yet been solved. Summary of the Invention

[0005] The purpose of this invention is to provide a three-dimensional pattern retrieval method based on style-consistent grid flattening, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a three-dimensional pattern retrieval method based on style consistency grid flattening, characterized in that: the method includes the following steps:

[0007] S1. Obtain pattern datasets from two-dimensional cultural relic images;

[0008] S2. Obtain a 3D model through 3D reconstruction technology, and then perform preprocessing operations on the 3D model;

[0009] S3. The 3D texture is preprocessed using style consistency flattening technology to obtain a 2D image of the 3D texture that is completely flattened.

[0010] S4. Extract features from the preprocessed 3D flattened image using Transformer and visual labeling methods;

[0011] S5. Calculate the feature distance between 3D textures by extracting features, use the feature distance as the similarity between 3D textures, and select the top N 3D models as the results for display.

[0012] Preferably, the two-dimensional cultural relic image in step S1 can be any image with Chinese cultural patterns.

[0013] Preferably, step S3 includes the following steps:

[0014] a1. Use the boundary-first flattening algorithm to flatten the 3D mesh model and extract the flattened 3D model;

[0015] a2. Import the original 3D model and the 3D model obtained in step a1 using the Blender tool. Use the baking plugin to bake the original 3D model into the 3D model obtained in step a1. Obtain the flattened 3D texture through UV mapping.

[0016] a3. Use annotation software, such as LabelMe, to annotate the patterns in the flattened texture for feature calculation.

[0017] Preferably, step S4 includes the following steps:

[0018] b1. Use spatial attention to extract local information from the image, for local feature sets. L attention maps were generated. They are implemented by L 1×1 convolutional layers; the parameters representing the convolutional layers are... The attention module can adaptively discover special regions in an image, represented as pattern regions in 3D pattern retrieval. The attention graph is calculated as follows:

[0019]

[0020] in This is the i-th attention map, where w and h are the width and height of the local image, respectively;

[0021] b2. Visual tagging technology constructs visual tags by aggregating local features; local visual tags are calculated as follows:

[0022]

[0023] In the formula , ;

[0024] b3. Visual features T are used as input, and T is mapped to a query with three C×C dimensional learnable matrices. ,key Sum Attention is calculated as

[0025]

[0026] Among them, normalized similarity Different visual markers and Model the correlation between them;

[0027] b4. Calculate the similarity S using MHA. In MHA, different projection matrices for the query, key, and value are used for different headers. These matrices map the visual tokens to different subspaces. MHA then uses... Layer weights Connecting, fusing, and reducing the dimensionality of the outputs from different heads, in form,

[0028]

[0029] Where N is the number of heads. This is the output of the i-th head.

[0030] Preferably, in step S5:

[0031] c1. Calculate the features of all patterns in the library and save them to an npz file;

[0032] c2. Input the 3D model and its texture for query. Extract the query features according to steps S3 and S4, calculate the similarity with all patterns in the library, sort the calculation results, and select the top 10 as the results to be displayed.

[0033] Preferably, step b2 is equivalent to an abstract expression of the pattern of a specific region within the image; in form,

[0034]

[0035] Where N is the Gaussian mixture number, Let z be the local features of an image of dimension c, and z be the latent clustering assignment variable. and Let be the mean vector and variance of the j-th Gaussian distribution, respectively; iteratively, the probability generated by each point for each component of the model is used to update the mean vector:

[0036]

[0037] Where M is the total number of features, in GMM, Representing local features The posterior probability of being assigned to the j-th cluster.

[0038] Preferably, in step c2, the similarity calculation is performed using the cosine similarity method, which estimates the degree of similarity between two feature vectors by calculating the cosine value between them. The calculation method is as follows:

[0039]

[0040]

[0041] The more similar the directions of two vectors, the closer their values ​​are to 1, meaning the more similar the patterns are.

[0042] Preferably, in step S4, the SGD model is trained with an initial learning rate of 0.01, a weight decay coefficient of 0.0001, and a momentum of 0.9.

[0043] Preferably, a linear decay scheduler is used in step S4 to gradually decay the learning rate to 0 when the required number of steps is reached.

[0044] Preferably, the dimension d of the global feature in step S4 is set to 1024.

[0045] Compared with the prior art, the beneficial effects of the present invention are:

[0046] This invention presents a 3D pattern retrieval method based on style-consistent mesh flattening. The method includes the following steps: S1, obtaining a pattern dataset from 2D cultural relic images; S2, obtaining a 3D model through 3D reconstruction technology, followed by preprocessing of the 3D model; S3, preprocessing the 3D textures using style-consistent flattening technology to obtain a fully flattened 2D image of the 3D textures; S4, extracting features from the preprocessed flattened 3D image using Transformer and visual labeling methods; and S5, calculating the feature distance between 3D textures based on the extracted features, using the feature distance as the similarity between 3D textures, and selecting the top N 3D models as the results. This invention achieves the expected accuracy in retrieving 3D patterns from cultural relics with different configurations, demonstrating good versatility and enabling the flattening and retrieval of 3D patterns from cultural relics, regardless of the format, lighting, material, or resolution of the 3D model. Attached Figure Description

[0047] Figure 1 This is a detailed flowchart of the present invention;

[0048] Figure 2 This is a schematic diagram of a three-dimensional model of the present invention;

[0049] Figure 3 This is a schematic diagram of the flattened three-dimensional model of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0052] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "setting" should be interpreted broadly. For example, they can refer to a fixed connection or setting, a detachable connection or setting, or an integrated connection or setting. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0053] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a number" means two or more, unless otherwise explicitly specified.

[0054] Example

[0055] Please see Figure 1-3 As shown, the present invention provides a technical solution for a three-dimensional pattern retrieval method based on style consistency grid flattening: This method includes the following steps:

[0056] S1. Obtain pattern datasets from two-dimensional cultural relic images. Two-dimensional images can be any images with Chinese cultural patterns, and the images can be obtained from museums or the Internet.

[0057] S2. Obtain a 3D model through 3D reconstruction technology, and then perform preprocessing operations on the 3D model. 3D modeling can obtain a 3D model through 3D reconstruction technology such as photogrammetry and laser scanning, and perform preprocessing operations such as mesh denoising, hole repair and mesh simplification on the 3D model.

[0058] S3. The 3D texture is preprocessed using style consistency flattening technology to obtain a 2D image of the 3D texture that is completely flattened.

[0059] a1. Use the boundary-first flattening algorithm to flatten the 3D mesh model and extract the flattened 3D model;

[0060] a2. Import the original 3D model and the 3D model obtained in step a1 using the Blender tool. Use the baking plugin to bake the original 3D model into the 3D model obtained in step a1. Obtain the flattened 3D texture through UV mapping.

[0061] a3. Use annotation software such as LabelMe to annotate the patterns in the flattened texture for feature calculation;

[0062] S4. Use Transformer and visual marker methods to extract features from the preprocessed 3D flattened image;

[0063] b1. Use spatial attention to extract local information from the image, for local feature sets. L attention maps were generated. They are implemented by L 1×1 convolutional layers; the parameters representing the convolutional layers are... The attention module can adaptively discover special regions in an image, represented as pattern regions in 3D pattern retrieval. The attention graph is calculated as follows:

[0064]

[0065] in This is the i-th attention map, where w and h are the width and height of the local image, respectively;

[0066] b2. Visual tagging technology constructs visual tags by aggregating local features. The local visual tags are calculated as follows:

[0067]

[0068] In the formula , Furthermore, visual tagging techniques help capture more distinctive visual patterns, thereby generating more general and robust image representations, which are equivalent to abstract expressions of patterns in specific regions within an image. In form:

[0069]

[0070] Where N is the Gaussian mixture number, Let z be the local features of an image of dimension c, and z be the latent clustering assignment variable. and Let be the mean vector and variance of the j-th Gaussian distribution, respectively; iteratively, the probability generated by each point for each component of the model is used to update the mean vector:

[0071]

[0072] Where M is the total number of features, in GMM, Representing local features Assigned

[0073] The posterior probability to the j-th cluster, and in this method, taking into account...

[0074]

[0075] Rephrased as

[0076]

[0077] in,

[0078]

[0079] in Set to 1, for This can be interpreted as local features. The soft clustering assignment to the i-th visual pattern is consistent with... The meaning is the same. It is the average vector corresponding to the i-th visual pattern, and its L2 norm is proportional to the probability of occurrence of the visual pattern. The model is trained using SGD with an initial learning rate of 0.01, a weight decay coefficient of 0.0001, and a momentum of 0.9. A linear decay scheduler is used to gradually decay the learning rate to 0 when the required number of steps is reached. The dimension d of the global feature is set to 1024.

[0080] b3. Visual features T are used as input, and T is mapped to a query with three C×C dimensional learnable matrices. ,key Sum Attention is calculated as

[0081]

[0082] Among them, normalized similarity Different visual markers and Model the correlation between them;

[0083] b4. Calculate the similarity S using MHA. In MHA, different projection matrices for the query, key, and value are used for different headers. These matrices map the visual tokens to different subspaces. MHA then uses... Layer weights Connecting, fusing, and reducing the dimensionality of the outputs from different heads, in form,

[0084]

[0085] Where N is the number of heads. For the output of the i-th head

[0086] S5. Calculate the feature distance between 3D textures by extracting features, use the feature distance as the similarity between 3D textures, sort them and select the top N 3D models as the results;

[0087] c1. Calculate the features of all patterns in the library and save them to an npz file;

[0088] c2. Input the 3D model and its texture for query. Extract the query features according to steps S3 and S4, and calculate the similarity with all patterns in the library. Sort the calculation results and select the top 10 as the results for display. The similarity calculation is the cosine similarity method, which estimates the degree of similarity between two feature vectors by calculating the cosine value between them. The calculation method is as follows:

[0089]

[0090]

[0091] The more similar the directions of two vectors, the closer their values ​​are to 1, meaning the more similar the patterns are.

[0092] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional pattern retrieval method based on style-consistent grid flattening, characterized in that: This method includes the following steps: S1. Obtain pattern datasets from two-dimensional cultural relic images; S2. Obtain a 3D model through 3D reconstruction technology, and then perform preprocessing operations on the 3D model; S3. The 3D texture is preprocessed using style consistency flattening technology to obtain a 2D image of the 3D texture that is completely flattened. S4. Extract features from the preprocessed 3D flattened image using Transformer and visual labeling methods; S5. Calculate the feature distance between 3D textures by extracting features, use the feature distance as the similarity between 3D textures, sort them and select the top N 3D models as the results for display. Step S3 includes the following steps: a1. Use the boundary-first flattening algorithm to flatten the 3D mesh model and extract the flattened 3D model; a2. Import the original 3D model and the 3D model obtained in step a1 using the Blender tool. Use the baking plugin to bake the original 3D model into the 3D model obtained in step a1. Obtain the flattened 3D texture through UV mapping. a3. Using labeling software, labelme is used to annotate the patterns in the flattened texture for feature calculation; Step S4 includes the following steps: b1. Use spatial attention to extract local information from the image, for local feature sets. L attention maps were generated. They are implemented by L 1×1 convolutional layers; the parameters representing the convolutional layers are... The attention module can adaptively discover special regions in an image, represented as pattern regions in 3D pattern retrieval. The attention graph is calculated as follows: in This is the i-th attention map, where w and h are the width and height of the local image, respectively; Explanation of Opinion Statement The meaning, Meaning b2. Visual tagging technology constructs visual tags by aggregating local features; local visual tags are calculated as follows: In the formula , ; b3. Visual features T are used as input, and T is mapped to a query with three C×C dimensional learnable matrices. ,key Sum Attention is calculated as Among them, normalized similarity Different visual markers and Model the correlation between them; b4. Calculate the similarity S using MHA. In MHA, different projection matrices for the query, key, and value are used for different headers. These matrices map the visual tokens to different subspaces. MHA then uses... Layer weights Connecting, fusing, and reducing the dimensionality of the outputs from different heads, in form, Where N is the number of heads. This is the output of the i-th head.

2. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 1, characterized in that: The two-dimensional cultural relic image mentioned in step S1 can be any image with Chinese cultural patterns.

3. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 1, characterized in that: In step S5: c1. Calculate the features of all patterns in the library and save them to an npz file; c2. Input the 3D model and its texture for query. Extract the query features according to steps S3 and S4, calculate the similarity with all patterns in the library, sort the calculation results, and select the top 10 as the results to be displayed.

4. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 1, characterized in that: Step b2 is equivalent to an abstract expression of the pattern of a specific region within the image. In form, Where N is the Gaussian mixture number, Let z be the local features of an image of dimension c, and z be the latent clustering assignment variable. and Let be the mean vector and variance of the j-th Gaussian distribution, respectively; iteratively, the probability generated by each point for each component of the model is used to update the mean vector: Where M is the total number of features, in GMM, Representing local features The posterior probability of being assigned to the j-th cluster.

5. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 3, characterized in that: In step c2, the similarity calculation uses the cosine similarity method, which estimates the degree of similarity between two feature vectors by calculating the cosine value between them. The calculation method is as follows: The more similar the directions of two vectors, the closer their values ​​are to 1, meaning the more similar the patterns are.

6. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 1, characterized in that: In step S4, the model is trained using SGD with an initial learning rate of 0.01, a weight decay coefficient of 0.0001, and a momentum of 0.

9.

7. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 1, characterized in that: In step S4, a linear decay scheduler is used to gradually decay the learning rate to 0 when the required number of steps is reached.

8. The three-dimensional pattern retrieval method based on style consistency grid flattening according to claim 1, characterized in that: In step S4, the dimension d of the global feature is set to 1024.

Images