Classical building three-dimensional modeling and image redrawing method and system, terminal and medium

Abstract

The invention relates to the field of cultural genetic digitization. The invention discloses a three-dimensional modeling and image redrawing method and system of a classical building, a terminal and a medium, which can perform modeling processing and image redrawing on an imaginary classical building or a classical building which exists in history and is already disappeared nowadays, and improve the modeling accuracy of the classical building and the cultural expression of the classical building. The method comprises the following steps: acquiring character description information and a building feature image corresponding to a classical building, and extracting building feature information from the character description information; inputting the text description information and the building feature information into a Dall-E3 model to generate an intention graph; based on the building feature information and the building feature image, constructing a three-dimensional model of the classical building by referring to the intention map; and selecting a two-dimensional image of the three-dimensional model at a preset angle, and redrawing the classical building based on the building feature information and the two-dimensional image to generate a redrawn image.

Description

Technical Field

[0001] The present application relates to the field of cultural heritage digitization. More specifically, the present application relates to a method, system, terminal and medium for three-dimensional modeling and image redrawing of classical architecture. Background Art

[0002] Existing classical buildings usually use digital twin technology to digitally reproduce classical buildings based on three-dimensional point cloud data. However, this digital twin technology requires expensive equipment and professional staff to collect and process field data, such as field measurements, photography, and hand-drawn drawings. As a result, this technology is limited to modeling classical buildings that actually exist in real life, and cannot model fictional classical buildings or classical buildings that existed in history but have disappeared today. In addition, this digital twin technology focuses on the reproduction of the physical properties of classical buildings (such as size, shape, and structure), ignoring the cultural connotations of the classical buildings themselves, resulting in inaccurate modeling effects. Summary of the invention

[0003] The purpose of the embodiment of the present application is to provide a method, system, terminal and medium for three-dimensional modeling and image redrawing of classical architecture, which can not only model fictional classical architecture or classical architecture that existed in history but has disappeared today, but also improve the accuracy of classical architecture modeling. The embodiment of the present application is mainly achieved through the following technical solutions:

[0004] A first aspect of an embodiment of the present application provides a method for three-dimensional modeling and image redrawing of a classical building, comprising:

[0005] Acquire text description information and architectural feature images corresponding to the classical architecture, and extract architectural feature information from the text description information;

[0006] Input the text description information and the building feature information into the Dall-E 3 model to generate an intention diagram;

[0007] Based on the architectural feature information and the architectural feature image, and with reference to the intention map, a three-dimensional model of the classical building is constructed;

[0008] A two-dimensional image of the three-dimensional model at a preset angle is selected, and the classical building is redrawn based on the architectural feature information and the two-dimensional image to generate a redrawn image.

[0009] According to an embodiment of the present application, the step of extracting building feature information from the text description information includes:

[0010] Get architectural hint words;

[0011] A large language model is used to extract the building feature information from the text description information based on the building prompt words.

[0012] According to an embodiment of the present application, the step of inputting the text description information and the building feature information into the Dall-E 3 model to generate an intention diagram includes:

[0013] The Dall-E 3 model encodes the text description information and the building feature information to obtain a text vector;

[0014] The text vector is input into the generator network of the Dall-E 3 model, and the intention graph is output.

[0015] According to an embodiment of the present application, the step of constructing the three-dimensional model of the classical building based on the building feature information and the building feature image and referring to the intention map includes:

[0016] Based on the building feature information and the building feature image, and with reference to the wall data of the intentional image, a wall structure of the classical building is constructed;

[0017] Based on the building feature information and the building feature image, and with reference to the peripheral wall data of the intentional image, construct the auxiliary structure and the peripheral non-load-bearing structure of the wall structure;

[0018] Based on the building feature information and the building feature image, and with reference to the roof data of the intention map, construct a roof structure corresponding to the wall structure;

[0019] The wall structure, the auxiliary structure, the peripheral non-load-bearing structure and the roof structure are integrated to construct an initial three-dimensional model;

[0020] The initial three-dimensional model is rendered to construct the three-dimensional model.

[0021] According to an embodiment of the present application, the step of redrawing the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image includes:

[0022] Using a reverse algorithm to identify the two-dimensional image to obtain multiple positive keywords;

[0023] Screening the multiple positive keywords to obtain at least one target keyword;

[0024] Get reverse keywords and scenario keywords;

[0025] Generate a combined keyword based on the target keyword, the reverse keyword and the scenario keyword;

[0026] The redrawn image is generated based on the architectural feature information, the combined keywords, and the two-dimensional image.

[0027] According to an embodiment of the present application, the step of screening the multiple positive keywords to obtain at least one target keyword includes:

[0028] Obtaining a confidence value corresponding to each of the positive keywords, and evaluating the confidence of each of the positive keywords;

[0029] Set preset values;

[0030] Each confidence value is compared with the preset value, and when the confidence value is greater than the preset value, the forward keyword corresponding to the confidence value is marked as the target keyword.

[0031] According to an embodiment of the present application, the step of generating the redrawn image based on the building feature information, the combined keywords and the two-dimensional image includes:

[0032] Configuring parameters of a Stable Diffusion model based on the building feature information and the combined keywords to obtain a graph-generated graph model;

[0033] The two-dimensional image is reconstructed based on the image-generated model to generate the redrawn image.

[0034] A second aspect of the embodiment of the present application provides a three-dimensional modeling and image redrawing system for classical architecture, comprising:

[0035] An extraction module, used to obtain text description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the text description information;

[0036] An intention diagram generation module, used for inputting the text description information and the building feature information into the Dall-E 3 model to generate an intention diagram;

[0037] A three-dimensional model building module, used to build a three-dimensional model of the classical building based on the building feature information and the building feature image and with reference to the intention map;

[0038] The redrawing image generation module is used to select a two-dimensional image of the three-dimensional model at a preset angle, redraw the classical building based on the building feature information and the two-dimensional image, and generate a redrawing image.

[0039] A third aspect of an embodiment of the present application provides a terminal device, including: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the three-dimensional modeling and image redrawing method of classical architecture described in the first aspect of the embodiment of the present application.

[0040] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the method for three-dimensional modeling and image redrawing of classical architecture described in the first aspect of the embodiment of the present application.

[0041] The beneficial effects of the embodiments of the present application include:

[0042] The embodiment of the present application adopts the idea of ​​transforming text to three-dimensional model and three-dimensional model to two-dimensional image (i.e., redrawing image) to achieve the three-dimensional modeling and image redrawing effect of classical architecture. Specifically, the embodiment of the present application obtains the text description information and architectural feature image corresponding to the classical architecture, and extracts the architectural feature information from the text description information; inputs the text description information and the architectural feature information into the Dall-E 3 model to generate an intention map; constructs the three-dimensional model of the classical architecture based on the architectural feature information and the architectural feature image, and refers to the intention map; selects the two-dimensional image of the three-dimensional model at a preset angle, redraws the classical architecture based on the architectural feature information and the two-dimensional image, and generates a redrawing image. The three-dimensional modeling and image redrawing method of classical architecture provided in the embodiment of the present application does not require the staff to obtain data by means of field measurement and field photography, but combines the text description information and architectural feature image corresponding to the classical architecture to achieve the three-dimensional modeling and image redrawing of the classical architecture. Therefore, compared with the prior art, the embodiment of the present application can model and process fictional classical architecture with text records or classical architecture that existed in history but has disappeared today. Moreover, various written records increase the authenticity of classical architecture, thereby improving the reliability and accuracy of classical architecture modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flowchart of the three-dimensional modeling and image redrawing method of classical architecture in some embodiments of the present application;

[0045] Figure 2 A flowchart of the three-dimensional modeling and image redrawing method of classical architecture in other embodiments of the present application;

[0046] Figure 3 The rendering of the three-dimensional model of this application;

[0047] Figure 4 The rendering of the three-dimensional model of this application;

[0048] Figure 5 The rendering of the redrawn image for this application;

[0049] Figure 6 The rendering of the redrawn image for this application;

[0050] Figure 7 The rendering of the redrawn image for this application;

[0051] Figure 8 A principle block diagram of the three-dimensional modeling and image redrawing system of classical architecture of the present application in some embodiments;

[0052] Fig. 9 This is a functional block diagram of the terminal device of the present application in some embodiments. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0054] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0055] The terms "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0056] The terms "comprises," "includes," or any other variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements explicitly listed but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0057] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more related listed items.

[0058] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0059] <3D Modeling and Image Redrawing Methods of Classical Architecture>

[0060] like Figure 1 FIG. 1 is a flowchart of a method for three-dimensional modeling and image redrawing of a classical building provided in the first aspect of the embodiment of the present application. Figure 1 In the method, the three-dimensional modeling and image redrawing method of the classical architecture includes:

[0061] S1. Obtain text description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the text description information. Figure 2 As shown, in Figure 2 In the example, the text description information is described as "text" and the building feature image is described as "image information".

[0062] The classical buildings are specifically fictional classical buildings, classical buildings that existed in history but have disappeared today, or classical buildings that exist now. Exemplarily, fictional classical buildings can be virtual buildings in works such as "Dream of Red Mansions", "Water Margin" or "Romance of the Three Kingdoms". Classical buildings that existed in history but have disappeared today can be buildings that actually existed in a certain period in history, but the building has not survived to this day. The classical buildings that exist now can be famous classical buildings such as Yueyang Tower or Drum Tower. It should be understood that classical buildings that existed in history but have disappeared today must have relevant written records, for example, there must be clear written records of the building structure, environmental description, spatial layout within the building, and architectural features in history books or other carriers.

[0063] The text description information is a kind of documentary record information and can come from carriers such as books, journals, online articles or newspapers.

[0064] The architectural feature image may be implemented as a related picture in the document record information.

[0065] In the embodiment of the present application, the classical architecture is taken as an example of the architecture in the Grand View Garden in the Dream of Red Mansions. Therefore, the text description information can be all or part of the text information selected from each of the books "Image History of Chinese Architecture", "Moving to the Red Mansion" and "Dream of Red Mansions", and the architectural feature image can be all or part of the images selected from each of the books "Image History of Chinese Architecture", "Moving to the Red Mansion" and "Dream of Red Mansions".

[0066] The text information in the text description information shall contain the architectural style information, water flow direction information, orientation layout information, garden landscape information and building complex characteristics information of the classical building. Among them, the architectural style information refers to the wall structure, roof structure, peripheral non-load-bearing structure and auxiliary structure of the classical building. The wall structure includes the structure of the wall itself, and also includes the structural layout of the doors and windows set on the wall. The roof structure includes the roof structure and the roof material. The auxiliary structure refers to the corridors, stairs and bridges outside the building. The water flow direction information refers to the flow direction of water outside the classical building. The orientation layout information refers to the layout position of the garden landscape objects outside the classical building. The garden landscape information refers to the information of ponds, rockery, plants, sculptures and stone carvings outside the classical building. The building complex characteristics information refers to the architectural style information, water flow direction information, orientation layout information and garden landscape information presented by a combination of multiple classical buildings.

[0067] The architectural feature information includes but is not limited to the garden landscape information, the orientation layout information and the architectural style information. In other embodiments, the architectural feature information also includes keywords and cultural characteristics. The specific content of the architectural feature information can be determined by those skilled in the art according to actual needs.

[0068] In some embodiments, the text description information also includes art keywords. The art keywords include, but are not limited to, landscape paintings, ancient poetry, ponds, rockery, flowers and trees, stone carvings or wood carvings commonly seen in classical architecture. In other embodiments, the art keywords can also be extracted in combination with photos, hand-drawn drafts or online pictures of classical architecture.

[0069] Furthermore, the step of extracting building feature information from the text description information includes:

[0070] S11. Obtain building prompt words.

[0071] The embodiment of the present application uses the prompt word technology to generate architectural prompt words for extracting the descriptive features of classical architecture. For example, the architectural prompt words can be painted beams and columns, symmetry, carvings, tiled roofs, and slender windows. In other embodiments, the architectural prompt words can be determined by those skilled in the art according to actual needs.

[0072] S12: Using a large language model to extract the building feature information from the text description information based on the building prompt words.

[0073] The large language model can be ChatGPT (Chat Generative Pre-trained Transformer, a chatbot model released by OpenAI), ChatGLM (a generative language model built based on the OpenAI GPT model framework) or Kimi.

[0074] Exemplarily, when the building prompt word is "painted beams and columns", the building feature information contains the building style information, and the specific text information is "artistic patterns painted on beams and columns"; when the building prompt word is "symmetry", the building feature information contains the building style information, and the specific text information is "bilaterally symmetrical doors and windows"; when the building prompt word is "carving", the building feature information contains the garden landscape information, and the specific text information is "fine stone carvings and wood carvings"; when the building prompt word is "tile roof", the building feature information contains the building style information, and the specific text information is "roof paved with traditional tiles"; when the building prompt word is "slender windows", the building feature information contains the building style information, and the specific text information is "narrow and long window design".

[0075] It should be understood that the building feature information is a part of the text information in the text description information.

[0076] When the 3D modeling and image redrawing method of classical architecture is applied to a building complex, in order to summarize the style of each classical building, the large language model will also divide the building complex into multiple building areas based on the text description information. For example, taking the Grand View Garden in A Dream of Red Mansions as an example, the large language model divides it into four areas: Pastoral Poetry Area, Green Path Area, Moon Shadow Landscape Area, and Zen Blessing Area, and extracts the corresponding architectural feature information according to the area.

[0077] In some implementations, when the text description information includes descriptions of multiple classical buildings, after step S12, the step of extracting architectural feature information from the text description information further includes:

[0078] The word vector model of a large language model and the K-means clustering algorithm are used to perform cluster analysis on the architectural feature information, and the individual classical buildings are classified into groups according to their corresponding architectural feature information (ie, architectural features), thereby forming the style characteristics of the architectural complex.

[0079] S2. Input the text description information and the building feature information into the Dall-E 3 model to generate an intention diagram. Figure 2 As shown, in Figure 2 In the example, the text description information is described as "original text" and the building feature information is described as "building features".

[0080] Furthermore, the S2 step includes:

[0081] S21. The Dall-E 3 model encodes the text description information and the building feature information to obtain a text vector.

[0082] The Dall-E 3 model is an OpenAI image generation model.

[0083] The encoding process is to convert the text description information and the building feature information into a digital form that can be understood by a computer, namely, a text vector.

[0084] The step S21 can be implemented by the Transformer model in the Dall-E 3 model to ensure that the Dall-E 3 model can capture the semantic information in the text.

[0085] S22. Input the text vector into the generator network of the Dall-E 3 model, and output the intention graph.

[0086] The generator network can process the text vector by combining a variational autoencoder (VAE) and a generative adversarial network (GAN). Specifically, the variational autoencoder maps the text vector to a latent space and generates the intention graph through a decoder, while the generative adversarial network improves the quality and fidelity of the intention graph through adversarial training of the generator and the discriminator.

[0087] In other embodiments, the intentional drawing can be understood as a creative fusion image or a conceptual architectural design drawing, which can provide rich design inspiration and creative space for the formation of a three-dimensional model. However, the intentional drawing may have errors in details such as proportions and roof forms. Therefore, the intentional drawing is regarded as a reference material for three-dimensional modeling and rendering.

[0088] S3. Constructing a three-dimensional model of the classical building based on the architectural feature information and the architectural feature image and with reference to the intention map.

[0089] refer to Figure 2 , the embodiment of the present application can call the Grasshopper plug-in of the Rhino software to perform systematic parametric modeling of the building (or building complex). More specifically, the embodiment of the present application uses the miniLibs three-dimensional model library in the Grasshopper plug-in to generate the three-dimensional model. The miniLibs three-dimensional model library has a wealth of components for detailed modeling. Specific components can be walls, doors and windows, railings, and furniture. The use of the miniLibs three-dimensional model library can effectively reduce modeling time.

[0090] Furthermore, the Grasshopper plug-in can also analyze the lighting, ventilation, etc. of the auxiliary space (ie, the auxiliary structure) to ensure its comfort and practicality.

[0091] Furthermore, the S3 step includes:

[0092] S31, constructing the wall structure of the classical building based on the building feature information and the building feature image and referring to the wall data of the intention map.

[0093] Step S31 can be understood as analyzing the basic structure of the classical building in the intentional image in combination with the architectural feature information and the architectural feature image, and clarifying the functions and relationships of the components in the basic structure. The basic structure is the wall structure. Of course, in other embodiments, the basic structure may also include the architecture and other components of the classical building.

[0094] S32: based on the building feature information and the building feature image, and with reference to the peripheral wall data of the intentional image, draw the auxiliary structure and the peripheral non-load-bearing structure of the wall structure.

[0095] The auxiliary structure can be used to optimize the streamline design inside the classical building and enhance the ecological environment of the classical building to ensure its coordination with the main structure of the classical building.

[0096] The peripheral non-load-bearing structure includes peripheral walls and decorative structures.

[0097] S33: based on the building feature information and the building feature image, and with reference to the roof data of the intention diagram, draw a roof structure corresponding to the wall structure.

[0098] Specifically, the embodiment of the present application uses the line segment component to draw the reference line (i.e., the reference outline) of the roof structure according to the appearance of the classical building in the intentional drawing, and determines the basic geometric shape of the roof by selecting different curves or polygons. Then, using components such as "Loft" and "Surface from Curve", the reference line is converted into a three-dimensional surface to form the preliminary shape of the roof. Then, according to the design requirements, the inclination angle of the roof surface is adjusted. This step may require the use of "Rotate" or "Scale" components to achieve.

[0099] In other embodiments, according to the load requirements of the roof, the roof truss structure (such as wooden beams, steel beams, etc.) can be designed by meshing through components such as "Divide Surface". Regarding the connection of components in the roof structure, the connection relationship between the components can be set to ensure the stability and safety of the roof structure. For example, the "Line" or "Curve" component can be used to represent the connection line.

[0100] It should be understood that the implementation of step S33 should take into consideration the aesthetics and practicality of the roof structure.

[0101] S34, integrating the wall structure, the auxiliary structure, the peripheral non-load-bearing structure and the roof structure to construct an initial three-dimensional model.

[0102] S35, rendering the initial three-dimensional model to construct the three-dimensional model. The effect diagram of the three-dimensional model can be referred to Figure 3 and Figure 4 shown.

[0103] In the embodiments of the present application, reference is made to Figure 2 , calling Lumion software to perform high-precision simulation rendering on the initial three-dimensional model to construct the three-dimensional model. The three-dimensional model is a rendering. In other embodiments, the Lumion software can also output a panoramic view.

[0104] In other implementations, regarding the construction of the three-dimensional model, the Yihong Courtyard of the Grand View Garden in "Dream of Red Mansions" can be taken as an example, and modeling can be performed from four aspects: wall structure, auxiliary structure, roof structure and peripheral non-load-bearing structure. Specifically, in terms of wall structure, after the wall modeling is completed, the spatial layout of the rooms inside the wall can be set according to the analysis of the "small five-room porch" and internal furniture in the original text of "Dream of Red Mansions", and the configuration of four horizontal and six vertical rooms is adopted. The size of each room is determined according to the Qing Dynasty construction scale (0.32m), and one zhang (3.2m) is selected as the width of the standard bay; in terms of auxiliary structure, reference is made to Song Dynasty architecture, and considering the decorativeness of brackets in Ming and Qing Dynasties, the scale of auxiliary structure is appropriately reduced to highlight its decorative function; in terms of roof structure, the "juzhe" ratio calculation is followed, and by reasonably calculating the ratio, the model is made stable and beautiful, thus meeting the Qing Dynasty architectural specifications; in terms of peripheral non-load-bearing structures, the base, doors and windows, walls, roofs and other elements of classical architecture are systematically parametrically designed and modeled, and at the same time, the connection channels between individual classical buildings are supplemented, such as corridors and hanging flower gates, so as to complete the overall three-dimensional model of Yihongyuan building (or building complex).

[0105] S4. Selecting a two-dimensional image of the three-dimensional model at a preset angle, redrawing the classical building based on the architectural feature information and the two-dimensional image, and generating a redrawn image.

[0106] The two-dimensional image is Figure 2 3D model image in .

[0107] The preset angle may be any angle of the three-dimensional model, and may be specifically determined by those skilled in the art according to actual needs.

[0108] The redrawn image is a two-dimensional image.

[0109] In some implementations, the step of redrawing the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image includes:

[0110] S5. Using an inverse algorithm to identify the two-dimensional image, and obtaining a plurality of positive keywords.

[0111] The embodiment of the present application uses the inverse algorithm in the Stable Diffusion model to identify the two-dimensional image to obtain multiple positive keywords, thereby comprehensively refining the characteristics of the classical architecture.

[0112] The reverse inference algorithm is the wd14-vit-v2-git algorithm used by the Stable Diffusion model in the toolbox WD1.4 tagger. In other embodiments, the reverse inference algorithm can be determined by those skilled in the art according to actual needs.

[0113] As an advanced generative model, Stable Diffusion can improve the details, texture and clarity of two-dimensional images through image redrawing. Since two-dimensional images generated by three-dimensional models may have problems such as insufficient rendering or loss of details, Stable Diffusion can make up for this defect. The images generated by Stable Diffusion have more complex and natural texture effects, making the images look more realistic and more artistic.

[0114] The positive keywords may be exemplarily “scenery” or “Chinese-style architecture”.

[0115] S6. Screening the multiple positive keywords to obtain at least one target keyword.

[0116] Furthermore, the step S6 comprises:

[0117] S61: Obtain a confidence value corresponding to each of the positive keywords, and evaluate the confidence of each of the positive keywords.

[0118] S62. Set a preset value.

[0119] The preset value may be 0.4, that is, 40%. In other implementations, the preset value may be determined by those skilled in the art according to actual needs.

[0120] S63: Compare each confidence value with the preset value, and when the confidence value is greater than the preset value, mark the forward keyword corresponding to the confidence value as the target keyword.

[0121] The implementation of step S6 can ensure the accuracy of target keyword extraction.

[0122] S7. Obtain reverse keywords and scene keywords.

[0123] The reverse keywords may be “grayscale”, “low resolution”, etc.

[0124] The scene keywords are characteristic keywords related to the scene, and exemplary keywords may be “Mid-Autumn Festival”, “full moon”, etc.

[0125] S8. Generate a combined keyword based on the target keyword, the reverse keyword and the scenario keyword.

[0126] The step S8 mainly combines the target keyword, the reverse keyword and the scene keyword to form the combined keyword. Figure 2 In the example, step S7 is described as “keyword integration”.

[0127] S9. Generate the redrawn image based on the building feature information, the combined keywords and the two-dimensional image.

[0128] Furthermore, the step S9 includes:

[0129] S91. Configure parameters of a Stable Diffusion model based on the building feature information and the combined keywords to obtain a graph-generated graph model.

[0130] For example, the values ​​of the parameters may be configured as shown in Table 1:

[0131] Table 1

[0132] Configuration parameter Large model of base anything-v5-PrtRE VAE Model vae-ft-mse-840000-ema-pruned CLIP termination layer 2 Sampling method DPM++2M Karras Iteration steps 40 Input size 1920x1080 Output size 1920x1080 CFG Scale 7.5 Redraw range 0.75 Random number seed -1

[0133] In Table 1, the base model is used to generate images with different contents and styles; the VAE model is used to improve the quality and details of the generated images; the CLIP termination layer number is used to ensure the effective processing of the relationship between text and image; the sampling method is used to achieve fast and high-quality image generation through denoising and optimization scheduling; the iteration step number is used to optimize the details and quality in the image generation process; the input and output sizes are used to provide sufficient resolution to show more details; the CFG Scale is used to guide the high correlation between the generated image and the input prompt; the redrawing amplitude is used to determine the degree of retention of the original information; the random number seed is used to ensure that a different image is generated each time.

[0134] S92: reconstruct the two-dimensional image based on the image-generated model to generate the redrawn image. The process of generating the redrawn image can be understood as the process of image reconstruction.

[0135] Specifically, refer to Figure 2 ,The LoRA model is introduced into the described graph model to enhance the attention to the unique elements in classical architecture, including eaves, window shapes, carved patterns and other details, so as to achieve fine-tuning of the classical architecture drawing.

[0136] The LoRA model (Low-Rank Adaptation of Large Language Models) is a low-rank adaptation technology for fine-tuning large language models. It reduces parameters by introducing low-rank matrices, reduces fine-tuning costs, and maintains model performance. The LoRA model can be searched through AI drawing model websites such as LiblibAI, civitai, and tusi.

[0137] In some implementations, a ControlNet plug-in (i.e., ControlNet technology) is introduced into the image generation model, and the depth information and semantic information are input through the ControlNet plug-in to control the generation process of the redrawn image, thereby improving the precise control of the design and restoration of classical architectural images. This step is Figure 2 In other embodiments, the ControlNet plug-in further inputs line drawing information to control the generation process of the redrawn image.

[0138] The ControlNet plug-in inputs these three types of information to ensure the accurate expression of classical architecture in terms of space, semantics and shape structure.

[0139] The ControlNet plug-in includes at least a depth control unit and a semantic segmentation control unit. For parameter information of the depth control unit and the semantic segmentation control unit, reference may be made to Table 2.

[0140] Table 2

[0141]

[0142]

[0143] The effect diagram of redrawing the image can be referred to as shown in Figures 5-7. Specifically, Figure 5 This is the rendering of the Aojingxi Pavilion in the Grand View Garden in A Dream of Red Mansions; Figure 6 This is the rendering of Zi Lingzhou in the Grand View Garden in A Dream of Red Mansions; Figure 7 This is the rendering of the villa for visiting relatives in the Grand View Garden in "Dream of Red Mansions".

[0144] The method for three-dimensional modeling and image redrawing of classical architecture provided in the embodiment of the present application does not require the staff to obtain data by means of field measurement and field photography, but can realize three-dimensional modeling and image redrawing of classical architecture by combining text description information and architectural feature images corresponding to the classical architecture. Therefore, compared with the prior art, the embodiment of the present application can model and process fictional classical architecture with text records or classical architecture that existed in history but has disappeared today. In addition, various text records increase the authenticity of classical architecture, thereby improving the reliability and accuracy of classical architecture modeling.

[0145] The embodiments of the present application can be applied to historical and cultural heritage spaces. By deeply analyzing historical documents and manuscripts, combined with advanced 3D modeling technology and image redrawing technology, these scarce information can be transformed into intuitive visual expressions. This modeling process can fill in the gaps in historical records and provide a more vivid cultural heritage experience, making it easier for people to understand and feel the unique style and cultural background of ancient architecture.

[0146] Furthermore, since no additional field archaeology, reconstruction engineering or high-precision physical model production is required, the embodiments of the present application can significantly reduce the overall modeling cost.

[0147] <Modeling system for classical architecture>

[0148] like Figure 8 As shown, it is a principle block diagram of a 3D modeling and image redrawing system of classical architecture provided by the second aspect of the embodiment of the present application. Figure 8 In the embodiment, the three-dimensional modeling and image redrawing system 100 of classical architecture includes:

[0149] Extraction module 101, used to obtain text description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the text description information;

[0150] The intention diagram generation module 102 is used to input the text description information and the building feature information into the Dall-E3 model to generate an intention diagram;

[0151] A three-dimensional model building module 103, for building a three-dimensional model of the classical building based on the building feature information and the building feature image and with reference to the intention map;

[0152] The redrawn image generation module 104 is used to select a two-dimensional image of the three-dimensional model at a preset angle, redraw the classical building based on the building feature information and the two-dimensional image, and generate a redrawn image.

[0153] <Terminal Equipment>

[0154] In a third aspect of the embodiments of the present application, a terminal device is provided, wherein the principle block diagram of the terminal device can be as follows: Fig. 9As shown. The terminal device includes a processor, a memory, a network interface, a display screen and a temperature sensor connected via a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a three-dimensional modeling and image redrawing method of classical architecture is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0155] Those skilled in the art will understand that Fig. 9 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In some embodiments, the terminal device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the three-dimensional modeling and image redrawing method of classical architecture provided in the first aspect of the above-mentioned embodiment of the present application.

[0157] <Computer Readable Storage Medium>

[0158] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the method for three-dimensional modeling and image redrawing of classical architecture described in the first aspect of the embodiment of the present application.

[0159] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0160] The technical features of the above embodiments can be combined without changing the basic principles of the present application. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of patent protection of the present application shall be subject to the attached claims.

Claims

1. A method for three-dimensional modeling and image redrawing of classical architecture, characterized in that: include: Acquire text description information and architectural feature images corresponding to the classical architecture, and extract architectural feature information from the text description information; Input the text description information and the building feature information into the Dal lE 3 model to generate an intention map; Based on the architectural feature information and the architectural feature image, and with reference to the intention map, a three-dimensional model of the classical building is constructed; A two-dimensional image of the three-dimensional model at a preset angle is selected, and the classical building is redrawn based on the architectural feature information and the two-dimensional image to generate a redrawn image.

2. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that: The step of extracting building feature information from the text description information comprises: Get architectural hint words; A large language model is used to extract the building feature information from the text description information based on the building prompt words.

3. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that: The step of inputting the text description information and the building feature information into the Dal lE 3 model to generate the intention map includes: The Dal lE 3 model encodes the text description information and the building feature information to obtain a text vector; The text vector is input into the generator network of the Dal lE 3 model and the intention graph is output.

4. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that: The step of constructing the three-dimensional model of the classical building based on the building feature information and the building feature image and referring to the intention map comprises: Based on the building feature information and the building feature image, and with reference to the wall data of the intentional image, a wall structure of the classical building is constructed; Based on the building feature information and the building feature image, and with reference to the peripheral wall data of the intentional image, construct the auxiliary structure and the peripheral non-load-bearing structure of the wall structure; Based on the building feature information and the building feature image, and with reference to the roof data of the intention map, construct a roof structure corresponding to the wall structure; The wall structure, the auxiliary structure, the peripheral non-load-bearing structure and the roof structure are integrated to construct an initial three-dimensional model; The initial three-dimensional model is rendered to construct the three-dimensional model.

5. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 1, characterized in that: The step of redrawing the classical building based on the architectural feature information and the two-dimensional image to generate a redrawn image comprises: using an inverse algorithm to identify the two-dimensional image to obtain a plurality of positive keywords; Screening the multiple positive keywords to obtain at least one target keyword; Get reverse keywords and scenario keywords; Generate a combined keyword based on the target keyword, the reverse keyword and the scenario keyword; The redrawn image is generated based on the architectural feature information, the combined keywords, and the two-dimensional image.

6. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 5, characterized in that: The step of screening the plurality of positive keywords to obtain at least one target keyword comprises: Obtaining a confidence value corresponding to each of the positive keywords, and evaluating the confidence of each of the positive keywords; Set preset values; Each confidence value is compared with the preset value, and when the confidence value is greater than the preset value, the forward keyword corresponding to the confidence value is marked as the target keyword.

7. The method for three-dimensional modeling and image redrawing of classical architecture according to claim 5, characterized in that: The step of generating the redrawn image based on the building feature information, the combined keywords and the two-dimensional image comprises: Configuring parameters of a Stable Diffusion model based on the building feature information and the combined keywords to obtain a graph-generated graph model; The two-dimensional image is reconstructed based on the image-generated model to generate the redrawn image.

8. A three-dimensional modeling and image redrawing system for classical architecture, characterized in that: include: An extraction module, used to obtain text description information and architectural feature images corresponding to classical architecture, and extract architectural feature information from the text description information; An intention diagram generation module, used for inputting the text description information and the building feature information into the Dal lE 3 model to generate an intention diagram; A three-dimensional model building module, used to build a three-dimensional model of the classical building based on the building feature information and the building feature image and with reference to the intention map; The redrawing image generation module is used to select a two-dimensional image of the three-dimensional model at a preset angle, redraw the classical building based on the building feature information and the two-dimensional image, and generate a redrawing image.

9. A terminal device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the method for three-dimensional modeling and image redrawing of classical architecture as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the steps of the method for three-dimensional modeling and image redrawing of classical architecture as described in any one of claims 1 to 7.

Images