Three-dimensional modeling and image redrawing method, system, terminal and medium for classical architecture

Abstract

The application relates to the field of cultural heritage digitization. The application discloses a classical building three-dimensional modeling and image redrawing method, system, terminal and medium, which can model and redraw images of fictitious classical buildings or classical buildings that existed in history but have disappeared today, and improve the accuracy of classical building modeling and the cultural expression of classical buildings. The method comprises obtaining text description information and building feature images corresponding to a classical building, and extracting building feature information from the text description information; inputting the text description information and the building feature information into a Dall-E 3 model to generate an intention map; constructing a three-dimensional model of the classical building based on the building feature information and the building feature images and referring to the intention map; selecting a two-dimensional image of the three-dimensional model at a preset angle, redrawing the classical building based on the building feature information and the two-dimensional image to generate a redrawing image.

Description

Technical Field

[0001] This application relates to the field of cultural heritage digitization. More specifically, this application relates to a method, system, terminal, and medium for three-dimensional modeling and image redrawing of classical architecture. Background Technology

[0002] Existing methods for digitally recreating classical architecture typically employ digital twin technology based on 3D point cloud data. However, this technology requires expensive equipment and specialized personnel for on-site data collection and processing, including field measurements, photography, and hand-drawn drawings. Consequently, this technology is limited to modeling existing classical buildings and cannot model fictional classical buildings or historically significant but now defunct structures. Furthermore, this digital twin technology focuses on reproducing the physical attributes of classical architecture (such as size, shape, and structure), neglecting the cultural connotations of the architecture itself, resulting in inaccurate modeling. Summary of the Invention

[0003] The purpose of this application is to provide a method, system, terminal, and medium for 3D modeling and image redrawing of classical architecture. This method can not only model fictional classical buildings or historically existing but now vanished classical buildings, but also improve the accuracy of classical building modeling. This application is mainly achieved through the following technical solutions:

[0004] A first aspect of this application provides a method for three-dimensional modeling and image redrawing of classical architecture, including:

[0005] Obtain textual description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the textual description information;

[0006] Input the textual description information and the architectural feature information into the Dall-E 3 model to generate an intentional image;

[0007] Based on the architectural feature information and the architectural feature image, and with reference to the conceptual drawing, a three-dimensional model of the classical building is constructed.

[0008] Select a two-dimensional image of the three-dimensional model at a preset angle, and redraw the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image.

[0009] According to one embodiment of this application, the step of extracting architectural feature information from the textual description information includes:

[0010] Get building hints;

[0011] A large-scale language model is used to extract the building feature information from the text description information based on the building prompt words.

[0012] According to one embodiment of this application, the step of inputting the textual description information and the architectural feature information into the Dall-E 3 model to generate an intentional image includes:

[0013] The Dall-E 3 model encodes the text description information and the building feature information to obtain a text vector;

[0014] The text vector is input into the generator network of the Dall-E 3 model, and the intention graph is output.

[0015] According to one embodiment of this application, the step of constructing a three-dimensional model of the classical building based on the architectural feature information and the architectural feature image, and with reference to the conceptual drawing, includes:

[0016] Based on the architectural feature information and the architectural feature image, and with reference to the wall data of the conceptual drawing, the wall structure of the classical building is constructed.

[0017] Based on the building feature information and the building feature image, and with reference to the wall perimeter data of the conceptual drawing, the auxiliary structure and the peripheral non-load-bearing structure of the wall structure are constructed.

[0018] Based on the building feature information and the building feature image, and with reference to the roof data of the conceptual map, the roof structure corresponding to the wall structure is constructed;

[0019] By integrating the wall structure, the auxiliary structure, the external non-load-bearing structure, and the roof structure, an initial three-dimensional model is constructed.

[0020] The initial 3D model is rendered to construct the 3D model.

[0021] According to one embodiment of this application, the step of redrawing the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image includes:

[0022] A reverse algorithm is used to identify the two-dimensional image and obtain multiple positive keywords;

[0023] The multiple positive keywords are filtered to obtain at least one target keyword;

[0024] Obtain reverse keywords and scenario keywords;

[0025] Generate combined keywords based on the target keywords, the reverse keywords, and the scenario keywords;

[0026] The redrawn image is generated based on the architectural feature information, the combined keywords, and the two-dimensional image.

[0027] According to one embodiment of this application, the step of filtering the plurality of positive keywords to obtain at least one target keyword includes:

[0028] Obtain the confidence value corresponding to each of the positive keywords, and evaluate the confidence of each of the positive keywords;

[0029] Set preset values;

[0030] Each confidence value is compared with the preset value. If the confidence value is greater than the preset value, the positive keyword corresponding to the confidence value is marked as the target keyword.

[0031] According to one embodiment of this application, the step of generating the redrawn image based on the building feature information, the combined keywords, and the two-dimensional image includes:

[0032] Based on the architectural feature information and the combined keywords, the parameters of the Stable Diffusion model are configured to obtain a graph-generated graph model.

[0033] The two-dimensional image is reconstructed based on the graph-to-graph model to generate the reconstructed image.

[0034] A second aspect of this application provides a 3D modeling and image redrawing system for classical architecture, comprising:

[0035] The extraction module is used to acquire textual description information and architectural feature images corresponding to classical architecture, and to extract architectural feature information from the textual description information.

[0036] The intention diagram generation module is used to input the text description information and the architectural feature information into the Dall-E 3 model to generate an intention diagram;

[0037] A 3D model building module is used to build a 3D model of the classical building based on the architectural feature information and the architectural feature image, and with reference to the concept drawing;

[0038] The redrawing image generation module is used to select a two-dimensional image of the three-dimensional model at a preset angle, and redraw the classical building based on the architectural feature information and the two-dimensional image to generate a redrawing image.

[0039] A third aspect of this application provides a terminal device, including a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the steps of the three-dimensional modeling and image redrawing method for classical architecture described in the first aspect of this application.

[0040] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform the steps of the three-dimensional modeling and image redrawing method for classical architecture described in the first aspect of this application.

[0041] The beneficial effects of the embodiments of this application include:

[0042] This application embodiment employs a text-to-3D model and 3D model-to-2D image (i.e., redrawn image) conversion approach to achieve 3D modeling and image redrawing effects for classical architecture. Specifically, this application embodiment obtains textual description information and architectural feature images corresponding to the classical architecture, and extracts architectural feature information from the textual description information; inputs the textual description information and architectural feature information into a Dall-E 3 model to generate an intentional image; constructs a 3D model of the classical architecture based on the architectural feature information and the architectural feature image, and with reference to the intentional image; selects a 2D image of the 3D model at a preset angle, and redraws the classical architecture based on the architectural feature information and the 2D image to generate a redrawn image. The 3D modeling and image redrawing method for classical architecture provided by this application embodiment can achieve 3D modeling and image redrawing of classical architecture without the need for on-site measurement and photography by staff. Instead, it combines textual description information and architectural feature images corresponding to the classical architecture. Therefore, compared with the prior art, this application embodiment can model fictional classical architecture with written records or classical architecture that existed in history but has now disappeared. Furthermore, the various written records enhance the authenticity of classical architecture, thereby improving the reliability and accuracy of classical architectural modeling. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 The flowcharts for some embodiments of the method for 3D modeling and image redrawing of classical architecture according to this application are shown below.

[0045] Figure 2 Flowcharts for the three-dimensional modeling and image redrawing method for classical architecture according to this application in some other embodiments;

[0046] Figure 3 This is a rendering of the 3D model of this application;

[0047] Figure 4 This is a rendering of the 3D model of this application;

[0048] Figure 5 A rendering of the redrawn image for this application;

[0049] Figure 6 A rendering of the redrawn image for this application;

[0050] Figure 7 A rendering of the redrawn image for this application;

[0051] Figure 8 The diagram below shows the principle block diagram of the three-dimensional modeling and image redrawing system for classical architecture in some embodiments of this application.

[0052] Figure 9 This is a schematic block diagram of the terminal device of this application in some embodiments. Detailed Implementation

[0053] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0054] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0055] The terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0056] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or apparatus.

[0057] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items.

[0058] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0059] <Methods for 3D Modeling and Image Redrawing of Classical Architecture>

[0060] like Figure 1 The diagram shown is a flowchart of a method for three-dimensional modeling and image redrawing of classical architecture provided in the first aspect of an embodiment of this application. Figure 1 The method for 3D modeling and image redrawing of classical architecture includes:

[0061] S1. Obtain textual description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the textual description information. (Reference) Figure 2 As shown, in Figure 2 In this context, the textual description information is described as "text," and the architectural feature image is described as "image information."

[0062] The term "classical architecture" specifically refers to fictional classical architecture, historically existing classical architecture that has since disappeared, or currently existing classical architecture. For example, fictional classical architecture could be a fictional building from works such as *Dream of the Red Chamber*, *Water Margin*, or *Romance of the Three Kingdoms*. Historically existing classical architecture that has since disappeared could be a building that actually existed at a certain historical period but has not survived to the present day. Currently existing classical architecture could be famous classical buildings such as Yueyang Tower or the Drum Tower. It should be understood that historically existing classical architecture that has since disappeared must have relevant written records, such as clear descriptions of its structure, environment, internal spatial layout, and architectural features in history books or other media.

[0063] The textual description information is a type of documentary record information, which may originate from media such as books, journals, online articles, or newspapers.

[0064] The architectural feature image can be a related image from the documented information.

[0065] In this embodiment of the application, the classical architecture is taken as an example from the architecture within the Grand View Garden in "Dream of the Red Chamber". Therefore, the textual description information can be all or part of the textual information selected from each book in "Illustrated History of Chinese Architecture", "A Visit to the Red Chamber", and "Dream of the Red Chamber", and the architectural feature images can be all or part of the images selected from each book in "Illustrated History of Chinese Architecture", "A Visit to the Red Chamber", and "Dream of the Red Chamber".

[0066] The textual description information must include information on the architectural style of the classical buildings, water flow direction, orientation and layout, garden landscape, and characteristics of the architectural complex. Specifically, the architectural style information refers to the wall structure, roof structure, external non-load-bearing structures, and auxiliary structures of the classical buildings. The wall structure includes the framework of the walls themselves, as well as the structural layout of doors and windows installed on the walls. The roof structure includes the roof framework and roof material. The auxiliary structures refer to corridors, staircases, and bridges around the buildings. The water flow direction information refers to the direction of water flow around the classical buildings. The orientation and layout information refers to the location of garden landscape elements around the classical buildings. The garden landscape information refers to information on ponds, artificial hills, plants, sculptures, and stone carvings around the classical buildings. The architectural complex characteristics information refers to the architectural style, water flow direction, orientation and layout, and garden landscape information presented by the combination of multiple classical buildings.

[0067] The architectural feature information includes, but is not limited to, the garden landscape information, the orientation layout information, and the architectural style information. In other embodiments, the architectural feature information also includes keywords and cultural characteristics. The specific content of the architectural feature information can be determined by those skilled in the art based on actual needs.

[0068] In some embodiments, the textual description information also includes artistic keywords. These artistic keywords include, but are not limited to, landscape paintings, ancient poems, ponds, artificial mountains, flowers and trees, stone carvings, or wood carvings commonly found in classical architecture. In other embodiments, the artistic keywords can also be extracted from photographs, hand-drawn sketches, or online images of classical architecture.

[0069] Furthermore, the step of extracting architectural feature information from the textual description information includes:

[0070] S11, Obtain building hints.

[0071] This application employs cue word technology to generate architectural cue words for extracting descriptive features of classical architecture. For example, the architectural cue words may include painted beams and columns, symmetry, carvings, tiled roofs, and slender windows. In other embodiments, the architectural cue words can be determined by those skilled in the art based on actual needs.

[0072] S12. Using a large-scale language model, extract the building feature information from the text description information based on the building prompt words.

[0073] The large language model can be ChatGPT (Chat Generative Pre-trained Transformer, a chatbot model released by OpenAI), ChatGLM (a generative language model built on the OpenAI GPT model framework), or KiMi.

[0074] For example, when the building prompt is "painted beams and columns," the building feature information includes the building style information, specifically the text information "artistic patterns painted on beams and columns"; when the building prompt is "symmetry," the building feature information includes the building style information, specifically the text information "symmetrical doors and windows on both sides"; when the building prompt is "carving," the building feature information includes the garden landscape information, specifically the text information "exquisite stone and wood carving decorations"; when the building prompt is "tile roof," the building feature information includes the building style information, specifically the text information "roof made of traditional tiles"; when the building prompt is "slender windows," the building feature information includes the building style information, specifically the text information "narrow and long window design."

[0075] It should be understood that the architectural feature information is a portion of the textual description information.

[0076] When the 3D modeling and image redrawing method for classical architecture is applied to an architectural complex, in order to summarize the style of each classical building, the large-scale language model will also divide the architectural complex into multiple architectural regions based on the textual description information. For example, taking the Grand View Garden in "Dream of the Red Chamber" as an example, the large-scale language model is divided into four regions: Pastoral Poetry Garden, Secluded Green Path Garden, Moonlit Landscape Garden, and Zen Blessing Garden, and the corresponding architectural feature information is extracted according to the region.

[0077] In some embodiments, where the textual description information includes descriptions of multiple classical buildings, the step of extracting architectural feature information from the textual description information after step S12 further includes:

[0078] The word vector model of a large language model and the K-means clustering algorithm are used to perform cluster analysis on the architectural feature information. Individual classical buildings are classified into groups according to their corresponding architectural feature information (i.e. architectural characteristics), thereby forming architectural group style features.

[0079] S2. Input the textual description information and the architectural feature information into the Dall-E 3 model to generate a conceptual diagram. (Reference) Figure 2 As shown, in Figure 2 In this context, the textual description information is described as "original text," and the architectural feature information is described as "architectural characteristics."

[0080] Further, step S2 includes:

[0081] S21. The Dall-E 3 model encodes the text description information and the building feature information to obtain a text vector.

[0082] The Dall-E 3 model is an OpenAI image generation model.

[0083] The encoding process is to convert the textual description information and the architectural feature information into a digital form that a computer can understand, namely a text vector.

[0084] Step S21 can be implemented using the Transformer model in the Dall-E 3 model to ensure that the Dall-E 3 model can capture semantic information in the text.

[0085] S22. Input the text vector into the generator network of the Dall-E3 model and output the intention graph.

[0086] The generator network can combine a variational autoencoder (VAE) and a generative adversarial network (GAN) to process the text vector. Specifically, the variational autoencoder can map the text vector to a latent space and generate the intention graph through a decoder; while the generative adversarial network can improve the quality and realism of the intention graph through adversarial training between the generator and the discriminator.

[0087] In other embodiments, the concept image can be understood as a creative fusion image or conceptual architectural design drawing, which can provide rich design inspiration and creative space for the formation of the 3D model. However, the concept image may contain errors in details such as proportions and roof forms. Therefore, the concept image is considered as reference material for 3D modeling and rendering.

[0088] S3. Based on the architectural feature information and the architectural feature image, and with reference to the conceptual drawing, construct a three-dimensional model of the classical building.

[0089] refer to Figure 2 This application embodiment can utilize the Grasshopper plugin of Rhino software to perform systematic parametric modeling of buildings (or building complexes). More specifically, this application embodiment uses the miniLibs 3D model library within the Grasshopper plugin to generate the 3D model. The miniLibs 3D model library has a rich set of components for detailed modeling; specific components can include walls, doors, windows, railings, and furniture. Using the miniLibs 3D model library can effectively reduce modeling time.

[0090] Furthermore, the Grasshopper plugin can also analyze the lighting and ventilation of auxiliary spaces (i.e., auxiliary structures) to ensure their comfort and practicality.

[0091] Further, step S3 includes:

[0092] S31. Based on the architectural feature information and the architectural feature image, and with reference to the wall data of the conceptual drawing, construct the wall structure of the classical building.

[0093] Step S31 can be understood as analyzing the basic structure of the classical building in the intended image by combining the architectural feature information and the architectural feature image, and clarifying the function and relationship of each component in the basic structure. The basic structure is the wall structure. Of course, in other embodiments, the basic structure may also include the framework of the classical building and other components.

[0094] S32. Based on the building feature information and the building feature image, and referring to the wall perimeter data of the intention drawing, draw the auxiliary structure and the perimeter non-load-bearing structure of the wall structure.

[0095] The auxiliary structure can be used to optimize the streamlined design inside classical buildings and enhance their ecological environment to ensure their harmony with the main structure of the classical building.

[0096] The external non-load-bearing structure includes the external walls and decorative structures.

[0097] S33. Based on the building feature information and the building feature image, and referring to the roof data of the intention diagram, draw the roof structure corresponding to the wall structure.

[0098] Specifically, in this embodiment, based on the shape of the classical building in the schematic diagram, the baseline (i.e., baseline outline) of the roof structure is drawn using the line segment component, and the basic geometry of the roof is determined by selecting different curves or polygons. Next, components such as "Loft" and "Surface from Curve" are used to convert the baseline into a three-dimensional surface, forming the preliminary shape of the roof. Then, according to design requirements, the tilt angle of the roof surface is adjusted; this step may require the use of the "Rotate" or "Scale" component.

[0099] In other implementations, the roof truss structure (such as timber beams, steel beams, etc.) can be designed by meshing using components such as "Divide Surface" according to the roof's load requirements. Regarding the connections between components in the roof structure, the connection relationships between components can be set to ensure the stability and safety of the roof structure. For example, "Line" or "Curve" components can be used to represent connection lines.

[0100] It should be understood that the implementation of step S33 should take into account both the aesthetics and practicality of the roof structure.

[0101] S34. Integrate the wall structure, the auxiliary structure, the peripheral non-load-bearing structure, and the roof structure to construct an initial three-dimensional model.

[0102] S35. Render the initial 3D model to construct the 3D model. The rendered image of the 3D model can be referenced. Figure 3 and Figure 4 As shown.

[0103] In the embodiments of this application, reference is made to Figure 2 The initial 3D model is simulated and rendered with high precision using Lumion software to construct the 3D model. The 3D model is a rendered image. In other embodiments, the Lumion software can also output a panoramic image.

[0104] In other embodiments, the construction of the three-dimensional model can be based on the Grand View Garden in "Dream of the Red Chamber" as an example, and the model can be constructed from four aspects: wall structure, auxiliary structure, roof structure and external non-load-bearing structure. Specifically, regarding the wall structure, after the wall model is completed, the spatial layout of the rooms inside the walls can be set up according to the "small five-room annex" in the original text of "Dream of the Red Chamber" and the analysis of the internal furniture. The configuration of four horizontal and six vertical rooms is adopted, and the size of each room is determined according to the Qing Dynasty construction ruler (0.32m), and one zhang (3.2m) is selected as the standard width of the bay. In terms of auxiliary structures, Song Dynasty architecture is referenced, and the decorative nature of the brackets in the Ming and Qing Dynasties is considered. The scale of the auxiliary structures is appropriately reduced to highlight their decorative function. In terms of the roof structure, the "raised and folded" proportion is followed. By reasonably calculating this proportion, the model is made both stable and beautiful, thus conforming to the Qing Dynasty building specifications. In terms of the external non-load-bearing structure, the elements such as the platform, doors and windows, walls, and roof of the classical building are systematically parametrically designed and modeled. At the same time, the connecting passages between individual classical buildings are added, such as the covered corridor and the hanging flower gate, to complete the overall three-dimensional model of the Yihong Courtyard building (or building complex).

[0105] S4. Select a two-dimensional image of the three-dimensional model at a preset angle, and redraw the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image.

[0106] The two-dimensional image is also Figure 2 The three-dimensional model image in the image.

[0107] The preset angle can be any angle of the three-dimensional model, and the specific angle can be determined by those skilled in the art based on actual needs.

[0108] The redrawn image is a two-dimensional image.

[0109] In some embodiments, the step of redrawing the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image includes:

[0110] S5. The two-dimensional image is identified using a reverse algorithm to obtain multiple positive keywords.

[0111] This application uses the back-inference algorithm in the Stable Diffusion model to identify the two-dimensional image, thereby obtaining multiple positive keywords and comprehensively extracting the characteristics of classical architecture.

[0112] The reverse algorithm is the wd14-vit-v2-git algorithm used by the Stable Diffusion model in the WD1.4 tagger toolbox. In other embodiments, the reverse algorithm can be determined by those skilled in the art based on actual needs.

[0113] Stable Diffusion, as an advanced generative model, can enhance the detail, texture, and clarity of 2D images through image redrawing. Since 2D images generated from 3D models may suffer from under-rendering or loss of detail, Stable Diffusion can compensate for this deficiency. Images generated by Stable Diffusion have more complex and natural texture effects, making them look more realistic and artistically expressive.

[0114] Examples of positive keywords include "scenery" or "Chinese architecture".

[0115] S6. Filter the multiple positive keywords to obtain at least one target keyword.

[0116] Further, step S6 includes:

[0117] S61. Obtain the confidence value corresponding to each positive keyword, and evaluate the confidence of each positive keyword.

[0118] S62, Set preset values.

[0119] The preset value can be 0.4, or 40%. In other embodiments, the preset value can be determined by those skilled in the art based on actual needs.

[0120] S63. Compare each confidence value with the preset value. If the confidence value is greater than the preset value, mark the positive keyword corresponding to the confidence value as the target keyword.

[0121] The implementation of step S6 ensures the accuracy of target keyword extraction.

[0122] S7. Obtain reverse keywords and scenario keywords.

[0123] The reverse keywords can be "grayscale" and "low resolution", etc.

[0124] The scene keywords are distinctive keywords related to the scene, such as "Mid-Autumn Festival" or "full moon".

[0125] S8. Generate combined keywords based on the target keywords, the reverse keywords, and the scenario keywords.

[0126] Step S8 mainly involves merging the target keyword, the reverse keyword, and the scenario keyword to form the combined keyword. Figure 2 In the text, step S7 is described as "keyword integration".

[0127] S9. Generate the redrawn image based on the building feature information, the combined keywords, and the two-dimensional image.

[0128] Further, step S9 includes:

[0129] S91. Based on the building feature information and the combined keywords, configure the parameters of the Stable Diffusion model to obtain the graph-generated graph model.

[0130] For example, the values ​​of the configured parameters can be found in Table 1:

[0131] Table 1

[0132] Configuration parameter Large base model anything-v5-PrtRE VAE model vae-ft-mse-840000-ema-pruned CLIP Termination Layer Number 2 Sampling methods DPM++2M Karras Number of iterations 40 Input dimensions 1920x1080 Output size 1920x1080 CFG Scale 7.5 Redraw Amplitude 0.75 random number seed -1

[0133] In Table 1, the base model is used to generate images with different content and styles; the VAE model is used to improve the quality and detail of the generated images; the CLIP termination layer number is used to ensure effective handling of the relationship between text and images; the sampling method is used to achieve fast and high-quality image generation through denoising and optimized scheduling; the iteration step number is used to optimize the detail and quality in the image generation process; the input / output size is used to provide sufficient resolution to show more details; the CFG Scale is used to guide the high relevance of the generated image to the input prompt; the redraw magnitude is used to determine the degree of preservation of original information; and the random number seed is used to ensure that different images are generated each time.

[0134] S92. The two-dimensional image is reconstructed based on the graph-generated image model to generate the reconstructed image. The process of generating the reconstructed image can be understood as the process of image reconstruction.

[0135] For details, please refer to Figure 2 The LoRA model is introduced into the original drawing model to enhance the attention to details unique to classical architecture, including eaves, window shapes, and carved patterns, so as to achieve fine-tuning of classical architectural drawings.

[0136] The LoRA (Low-Rank Adaptation of Large Language Models) model is a low-rank adaptation technique for fine-tuning large language models. It reduces parameters by introducing a low-rank matrix, lowering fine-tuning costs while maintaining model performance. LoRA models can be found on AI graphics model websites such as LiblibAI, civitai, and tusi.

[0137] In some implementations, a ControlNet plugin (i.e., ControlNet technology) is introduced into the image-generating model. The ControlNet plugin inputs depth and semantic information to control the generation process of the redrawn image, improving the accuracy of the design and restoration of classical architectural images. This step is... Figure 2 The ControlNet plugin incorporates "depth control" and "semantic control" steps. In other implementations, the plugin also inputs line art information to control the generation process of the redrawn image.

[0138] The ControlNet plugin inputs these three types of information to ensure the accurate expression of classical architecture in terms of space, semantics, and shape structure.

[0139] The ControlNet plugin includes at least a depth control unit and a semantic segmentation control unit. The parameter information for the depth control unit and the semantic segmentation control unit can be found in Table 2.

[0140] Table 2

[0141]

[0142]

[0143] The resulting image of the redrawn image can be seen in Figures 5-7. Specifically, Figure 5 This is a rendering of the Concave Crystal Stream Pavilion in the Grand View Garden from "Dream of the Red Chamber". Figure 6 This is a rendering of the Purple Lotus Isle in the Grand View Garden from "Dream of the Red Chamber". Figure 7 This is a rendering of the Grand View Garden villa for visiting relatives in "Dream of the Red Chamber".

[0144] The 3D modeling and image redrawing method for classical architecture provided in this application does not require on-site measurement and photography to obtain data. Instead, it combines textual descriptions and architectural feature images corresponding to the classical architecture to achieve 3D modeling and image redrawing. Therefore, compared with existing technologies, this application can model fictional classical buildings with written records or historically existing but now vanished classical buildings. Furthermore, the various written records increase the authenticity of the classical architecture, thereby improving the reliability and accuracy of classical architecture modeling.

[0145] This application's embodiments can be applied to historical and cultural heritage spaces. By deeply analyzing historical documents and manuscripts, and combining advanced 3D modeling and image redrawing technologies, this scarce information can be transformed into intuitive visual representations. This modeling process can fill gaps in historical records and provide a more vivid cultural heritage experience, making it easier for people to understand and appreciate the unique style and cultural background of ancient architecture.

[0146] Furthermore, since no additional on-site archaeology, reconstruction projects, or high-precision physical model creation is required, the embodiments of this application can significantly reduce the overall modeling cost.

[0147] Modeling systems for classical architecture

[0148] like Figure 8 The diagram shown is a principle block diagram of a three-dimensional modeling and image redrawing system for classical architecture provided in the second aspect of an embodiment of this application. Figure 8 The 3D modeling and image redrawing system 100 for classical architecture includes:

[0149] Extraction module 101 is used to acquire text description information and architectural feature images corresponding to classical buildings, and extract architectural feature information from the text description information;

[0150] The intention map generation module 102 is used to input the text description information and the architectural feature information into the Dall-E3 model to generate an intention map;

[0151] The 3D model construction module 103 is used to construct a 3D model of the classical building based on the building feature information and the building feature image, and with reference to the concept drawing;

[0152] The redrawing image generation module 104 is used to select a two-dimensional image of the three-dimensional model at a preset angle, and redraw the classical building based on the architectural feature information and the two-dimensional image to generate a redrawing image.

[0153] Terminal Equipment

[0154] A third aspect of this application provides a terminal device, the schematic diagram of which is as follows: Figure 9As shown. The terminal device includes a processor, memory, network interface, display screen, and temperature sensor connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the terminal device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for 3D modeling and image redrawing of classical architecture. The display screen can be a liquid crystal display (LCD) or an e-ink display. The temperature sensor is pre-installed inside the terminal device to detect the operating temperature of the internal components.

[0155] Those skilled in the art will understand that Figure 9 The schematic diagram shown is only a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0156] In some embodiments, the terminal device includes a processor and a memory for storing computer programs. The processor is used to call and run the computer programs stored in the memory to perform the steps of the three-dimensional modeling and image redrawing method for classical architecture provided in the first aspect of the embodiments of this application.

[0157] Computer-readable storage media

[0158] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform the steps of the three-dimensional modeling and image redrawing method for classical architecture described in the first aspect of this application.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0160] The technical features of the above embodiments can be combined without changing the basic principles of this application. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the patent protection scope of this application should be determined by the appended claims.

Claims

1. A method for three-dimensional modeling and image redrawing of classical architecture, characterized in that, include: Obtain textual description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the textual description information; Input the textual description information and the architectural feature information into the Dall-E 3 model to generate an intentional image; Based on the architectural feature information and the architectural feature image, and with reference to the conceptual drawing, a three-dimensional model of the classical building is constructed. Select a two-dimensional image of the three-dimensional model at a preset angle, and redraw the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image; The step of redrawing the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image includes: using a back-inference algorithm to identify the two-dimensional image and obtain multiple positive keywords; filtering the multiple positive keywords to obtain at least one target keyword; obtaining reverse keywords and scene keywords; generating a combined keyword based on the target keyword, the reverse keyword, and the scene keyword; and generating the redrawn image based on the architectural feature information, the combined keyword, and the two-dimensional image.

2. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that, The step of extracting architectural feature information from the textual description information includes: Get building hints; A large-scale language model is used to extract the building feature information from the text description information based on the building prompt words.

3. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that, The step of inputting the textual description information and the architectural feature information into the Dall-E 3 model to generate the concept map includes: The Dall-E 3 model encodes the text description information and the building feature information to obtain a text vector; The text vector is input into the generator network of the Dall-E 3 model, and the intention graph is output.

4. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that, The step of constructing a three-dimensional model of the classical building based on the architectural feature information and the architectural feature image, and with reference to the conceptual drawing, includes: Based on the architectural feature information and the architectural feature image, and with reference to the wall data of the conceptual drawing, the wall structure of the classical building is constructed. Based on the building feature information and the building feature image, and with reference to the wall perimeter data of the conceptual drawing, the auxiliary structure and the peripheral non-load-bearing structure of the wall structure are constructed. Based on the building feature information and the building feature image, and with reference to the roof data of the conceptual map, the roof structure corresponding to the wall structure is constructed; By integrating the wall structure, the auxiliary structure, the external non-load-bearing structure, and the roof structure, an initial three-dimensional model is constructed. The initial 3D model is rendered to construct the 3D model.

5. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that, The step of filtering the plurality of positive keywords to obtain at least one target keyword includes: Obtain the confidence value corresponding to each of the positive keywords, and evaluate the confidence of each of the positive keywords; Set preset values; Each confidence value is compared with the preset value. If the confidence value is greater than the preset value, the positive keyword corresponding to the confidence value is marked as the target keyword.

6. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that, The step of generating the redrawn image based on the architectural feature information, the combined keywords, and the two-dimensional image includes: Based on the architectural feature information and the combined keywords, the parameters of the Stable Diffusion model are configured to obtain a graph-generated graph model. The two-dimensional image is reconstructed based on the graph-to-graph model to generate the reconstructed image.

7. A three-dimensional modeling and image redrawing system for classical architecture, characterized in that, include: The extraction module is used to acquire textual description information and architectural feature images corresponding to classical architecture, and to extract architectural feature information from the textual description information. The intention diagram generation module is used to input the text description information and the architectural feature information into the Dall-E 3 model to generate an intention diagram; A 3D model building module is used to build a 3D model of the classical building based on the architectural feature information and the architectural feature image, and with reference to the concept drawing; The redrawing image generation module is used to select a two-dimensional image of the three-dimensional model at a preset angle, and redraw the classical building based on the architectural feature information and the two-dimensional image to generate a redrawing image; The image redrawing generation module is further configured to use a reverse algorithm to identify the two-dimensional image and obtain multiple positive keywords; to filter the multiple positive keywords to obtain at least one target keyword; to obtain reverse keywords and scene keywords; and to generate combined keywords based on the target keyword, the reverse keywords, and the scene keywords. The redrawn image is generated based on the architectural feature information, the combined keywords, and the two-dimensional image.

8. A terminal device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to perform the steps of the three-dimensional modeling and image redrawing method for classical architecture as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the steps of the three-dimensional modeling and image redrawing method for classical architecture as described in any one of claims 1 to 6.

Images