Stable diffusion large model-based sluice engineering construction scheme design method

Abstract

The invention discloses a sluice engineering construction scheme design method based on a table diffusion large model, and the method comprises the following steps: S1, lora model training; selecting a pre-training model suitable for the sluice engineering building class task, and performing training and performance evaluation on the pre-training model by using a data set constructed by the specific task effect picture of the sluice engineering building class until a lora model which meets the requirement and is used for local fine tuning is trained; and S2, building scheme generation: according to specific sluice engineering building scheme requirements, selecting a building type large model, and generating a building effect picture in combination with the trained lora model and the Controlnet unit with the set Controlnet parameters. The method has the advantages that traditional design thinking limitation can be broken through, brand new thinking and possibility are brought to sluice engineering building scheme design, unique modeling and structural design can be generated, and the aesthetic value and the landscape effect of sluice engineering are improved while functional requirements are met.

Description

Technical Field

[0001] The invention relates to the technical field related to image generation and water conservancy engineering, and in particular to a sluice engineering architectural scheme design method based on a stable diffusion large model. Background Art

[0002] With the rapid development of deep neural network technology, artificial intelligence has made revolutionary breakthroughs in the field of image synthesis. As a leader among them, the stable diffusion model can intelligently generate matching images based on the text description provided by the user. This technology has extremely high practical value and innovative significance in literary creation, visual design, and CG production in the game field, which brings new means to improve the quality and efficiency of renderings in the engineering industry.

[0003] However, when generating images, the stable diffusion large model not only has specific requirements for the basic large model type, but also with the increase of output image resolution and the strength of detail control, the required fine-tuning small model will also be more subdivided. The large and small models currently available on the market are mostly building models of buildings, schools, and shopping malls, which leads to the direct generation of images basically composed of these three types of building elements, which in turn causes the failure of the drawing process of the sluice building. Therefore, it is necessary to specially train the detail control type of lora small model to correct the drawing content, and finally fine-tune the details to obtain the expected image results. Summary of the invention

[0004] The object of the present invention is to provide a sluice engineering architectural design method based on a stable diffusion large model, thereby solving the above-mentioned problems existing in the prior art.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for designing a sluice engineering construction scheme based on a stable diffusion large model comprises the following steps:

[0007] S1, lora model training;

[0008] Select a pre-trained model suitable for sluice engineering construction tasks, and use a dataset constructed from specific task renderings of sluice engineering construction to train and evaluate the performance of the pre-trained model until a LoRa model that meets the requirements for local fine-tuning is trained;

[0009] S2. Building plan generation:

[0010] According to the specific requirements of the sluice project construction plan, a large building model is selected, and the building renderings are generated by combining the trained LoRa model and the ControlNet unit with set ControlNet parameters.

[0011] Preferably, for the pre-trained model, a fine-tuning layer is defined, and a low-rank matrix is ​​introduced and initialized for the fine-tuning layer; during the training process, the layers in the pre-trained model that are not selected for fine-tuning are frozen, and only the parameters related to the low-rank matrix will be updated, and the weights of the layers that are not selected for fine-tuning remain unchanged.

[0012] Preferably, set an appropriate loss function and optimizer to compile the model for the pre-trained model;

[0013] During the training process of the lora model, you need to set the positive and negative prompt words, number of iterations, sampling method, size, prompt word guidance coefficient, and random number seed.

[0014] Preferably, the number of the controlnet units is two;

[0015] The controlnet parameters include original solution model picture, control type, control weight, control mode, scaling mode, preprocessor, and script.

[0016] Preferably, the original scheme model picture is a line drawing of a human view and a top view produced by building a rough model of the sluice project construction scheme using modeling software; the perspective of the human view fits the real human visual height, and the building is upright within the perspective; the top view perspective is a 45-degree angle overlooking the entire construction project range;

[0017] Preferably, the script includes an X / Y / Z icon that can batch produce different lora models and Vincent diagrams of different sampling methods.

[0018] Preferably, the effect diagram in the data set includes an image and corresponding label text;

[0019] The image perspectives include bird's-eye view and human-eye view; the image size is uniform, distortion-free and clear, and the content can fully reflect the facade features of the sluice building; the image is labeled with text, which is a complete sentence described in natural language and can clearly describe the overall and partial content and style of the sluice building;

[0020] The background of the image is filled with pure white, and when adding label text, the "white background" label is added; by entering "white background" in the reverse prompt word, the pure white background domain is prevented from appearing in the generated image, so as to train the Lora model to recognize the association between pure color blocks and text labels.

[0021] Preferably, step S2 specifically includes adding the text image to the graphic image for local redrawing to eliminate local errors and deviations; then adding the graphic image to post-processing to optimize the overall image, set the overall style, and enlarge the size to a preset size to output the building rendering.

[0022] Preferably, the text-generated image is added to the graphic-generated image for partial redrawing to eliminate the generated local errors and deviations. Specifically, the specific content is eliminated by first describing it in the reverse prompt word, and then supplementing the content to be generated in the forward prompt word;

[0023] For the upper building columns and the background sky and waterside slopes, the junction needs to be further refined, and the three types of content need to be partially redrawn separately; the door, window and roof styles can be quickly changed to the required style through redrawing, and the structural appearance materials can be quickly changed to the required materials through redrawing.

[0024] Preferably, if the generated architectural renderings do not meet the requirements, Adobe Photoshop is used to make fine adjustments to the details;

[0025] When fine-tuning details with Adobe Photoshop, use the AI ​​generation function to quickly generate fills for image content modification, image repair, or creative modification; use the box selection or lasso tool to select the local content that needs to be modified, activate the Generative Fill text generation fill panel, enter concise prompts or leave blanks to freely produce content to generate the final architectural rendering.

[0026] The beneficial effects of the present invention are: 1. The traditional architectural design of sluice projects often relies on the experience of designers and manual drawings, which is a cumbersome and time-consuming process. The present invention is based on a large stable diffusion model and can quickly generate a variety of preliminary design schemes, greatly shortening the design cycle. 2. In the traditional design process, due to the need for multiple modifications and experiments, high manpower, material and time costs will be incurred. The present invention can provide a more complete solution at the beginning of the design, reducing unnecessary modifications and rework, thereby reducing the overall design cost. 3. The large stable diffusion model used in the present invention has a strong innovation ability, which can break through the limitations of traditional design thinking, bring new ideas and possibilities to the architectural design of sluice projects, and can generate unique shapes and structural designs. While meeting functional requirements, it enhances the aesthetic value and landscape effect of sluice projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the design method in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] like Figure 1 As shown, in this embodiment, a sluice engineering architectural design method based on a stable diffusion large model is provided, comprising the following steps:

[0030] 1. Dataset Construction

[0031] Collect and prepare specific task renderings of sluice engineering buildings to build a data set. The renderings include images and label texts; the image perspectives include bird's-eye view and human-eye view. The image size is unified into 512*512 pixels, the image is distortion-free and clear, and the content can fully reflect the facade features of the sluice building. Label texts are added to the images. The label texts are complete sentences in human natural language, which can clearly describe the overall and local content and style of the sluice building.

[0032] During the image processing, the background is filled with pure white. When adding labels, add the "whitebackground" label to allow the Lora model to recognize and learn this specific content in the training image. By entering "white background" in the reverse prompt word, the pure white background domain can be prevented from appearing in the generated image, and the Lora model can be trained to recognize the association between pure color blocks and labels.

[0033] 2. Lora model training

[0034] Select a pre-trained model suitable for sluice engineering construction tasks, and use a dataset constructed from specific task renderings of sluice engineering construction to train and evaluate the performance of the pre-trained model until a lora model that meets the requirements for local fine-tuning is trained.

[0035] You need to choose a pre-trained model that is suitable for the task. For example, in image recognition tasks, choose an architectural model that has been pre-trained on a large-scale dataset. Choose a small building bottom floor type with a realistic overall style and natural details.

[0036] In this embodiment, for the pre-trained model, a fine-tuning layer is defined, and a low-rank matrix is ​​introduced and initialized for the fine-tuning layer; during the training process, the layers in the pre-trained model that are not selected for fine-tuning are frozen, and only the parameters related to the low-rank matrix will be updated, and the weights of the layers that are not selected for fine-tuning remain unchanged.

[0037] Set appropriate loss functions and optimizers for the pre-trained model and compile the model, including setting training parameters such as learning rate and batch size. The number of learning steps is kept between 3000 and 6000, the number of images is controlled within 300, and each image is trained 2 to 8 times.

[0038] The dataset is divided into a training set and a validation set. The pre-trained model is trained using the training set, and the performance of the trained model is evaluated on the validation set to ensure that the fine-tuning process is effective and the model is not overfitted. The Lora model that meets the requirements is trained iteratively.

[0039] In this embodiment, during the training process of the lora model, it is necessary to set the positive and negative prompt words, the number of iteration steps, the sampling method, the size, the prompt word guidance coefficient, the random number seed, etc.

[0040] Among them, the positive and negative prompt words should be a whole sentence paragraph, and the keywords should include sluice gates, buildings, rivers, fields, etc.

[0041] Specifically, the positive and negative prompt word contents contain complete sentences of the training data set; the number of iteration steps is controlled between 20 and 30 steps, too low resolution is too low, and too high resolution leads to excessive content generation; the size remains the original size, the random number seed remains empty, and the sampling method and prompt word guidance coefficient are set reasonably.

[0042] During the model training process, the initial screening stage aims to select the model training rounds and Lora weight values ​​with excellent performance. Under the premise of keeping the random seed unchanged, different training cycles and weight values ​​are tested, and each combination produces an image. In order to deepen the test of the control effect of the two variables of training rounds and weight values, we shorten the test interval of these variables, test the models of different rounds in turn, and generate images under different weight values. The number of generated images for each group of variables is increased to 2 or more, keeping other parameters unchanged. In order to confirm the accuracy of the screening results and evaluate the consistency of the Lora model in generating surface images in Stable Diffusion, it is necessary to generate 20-30 images at the same time and calculate the pass rate of these images for verification.

[0043] 3. Building Scheme Generation

[0044] According to the specific requirements of the sluice project construction plan, a large building model is selected, and combined with the trained LoRa model and the ControlNet unit with set ControlNet parameters, the three main key factors that control the generated image are determined, and then the building renderings are generated.

[0045] In this embodiment, the number of the controlnet units is two, and more units will destroy the original painting style.

[0046] The controlnet parameters include original solution model picture, control type, control weight, control mode, scaling mode, preprocessor, and script.

[0047] Among them, the original plan model picture is a line drawing of a human view and a bird's-eye view produced by building a rough model of the sluice project construction plan using modeling software such as revit / microstation / sketchup / rhino; the perspective of the human view fits the real human visual height, and the building is upright within the perspective; the bird's-eye view perspective is a 45-degree angle overlooking the entire construction project.

[0048] The script contains X / Y / Z icons, which can batch produce Wensheng diagrams of different sizes of models and different sampling methods. The number of models and sampling methods should be kept within 10 respectively to avoid overloading the computer and causing a black screen.

[0049] In this embodiment, the original scheme model image should be free of shadows and have clear outlines, the control type should select line draft to control architectural details, and the control weight, control mode, scaling mode, preprocessor, script, etc. should be set within a reasonable range.

[0050] In this embodiment, the text-generated image is added to the graphic-generated image for local redrawing to eliminate the local errors and deviations; then the graphic-generated image is added to the post-processing to optimize the overall image, set the overall style, and enlarge the size to a preset size to output the architectural rendering.

[0051] In this embodiment, the text map is added to the graphic map for partial redrawing to eliminate the generated local errors and deviations. Specifically, the specific content is eliminated by first describing it in the reverse prompt word, and then supplementing the content to be generated in the forward prompt word; especially the upper building columns and the background sky and the waterside slopes need to be further refined at the junction, and the three types of content are partially redrawn separately; the door and window and roof styles are quickly changed to the required style by redrawing, and the structural appearance material is quickly changed to the required material by redrawing, and white and gray materials are mostly used.

[0052] In this embodiment, if the generated architectural rendering does not meet the requirements, Adobe Photoshop is used to perform fine-tuning of the details;

[0053] When fine-tuning details with Adobe Photoshop, use the AI ​​generation function to quickly generate fills for image content modification, image repair, or creative modification; use the box selection or lasso tool to select the local content that needs to be modified, activate the Generative Fill text generation fill panel, enter concise prompts or leave blanks to freely produce content to generate the final architectural rendering.

[0054] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0055] The present invention provides a method for designing a water gate engineering architectural scheme based on a stable diffusion large model. The traditional water gate engineering architectural scheme design often relies on the designer's experience and manual drawing, and the process is cumbersome and time-consuming. The present invention is based on a stable diffusion large model and can quickly generate a variety of preliminary design schemes, greatly shortening the scheme design cycle. In the traditional design process, due to the need for multiple modifications and experiments, high manpower, material and time costs will be incurred. The present invention can provide a more complete scheme at the beginning of the design, reducing unnecessary modifications and rework, thereby reducing the overall design cost. The stable diffusion large model used in the present invention has a strong innovation ability, can break through the limitations of traditional design thinking, and bring new ideas and possibilities to the design of water gate engineering architectural schemes. It can generate unique shapes and structural designs, while meeting functional requirements, and enhancing the aesthetic value and landscape effect of water gate engineering.

[0056] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A sluice engineering architectural design method based on a stable diffusion large model, characterized by: The following steps are included: S1, lora model training; Select a pre-trained model suitable for sluice engineering construction tasks, and use a dataset constructed from specific task renderings of sluice engineering construction to train and evaluate the performance of the pre-trained model until a LoRa model that meets the requirements for local fine-tuning is trained; S2. Building plan generation: According to the specific requirements of the sluice project construction plan, a large building model is selected, and the building renderings are generated by combining the trained LoRa model and the ControlNet unit with set ControlNet parameters.

2. The method for designing a sluice engineering construction scheme based on a stable diffusion large model according to claim 1 is characterized in that: For the pre-trained model, define a fine-tuning layer, introduce a low-rank matrix into the fine-tuning layer and initialize it; During the training process, the layers in the pre-trained model that are not selected for fine-tuning are frozen. Only the parameters related to the low-rank matrix will be updated, and the weights of the layers that are not selected for fine-tuning remain unchanged.

3. The sluice engineering architectural design method based on the stable diffusion large model according to claim 2 is characterized by: Compile the model by setting appropriate loss function and optimizer for the pre-trained model; During the training process of the lora model, you need to set the positive and negative prompt words, number of iterations, sampling method, size, prompt word guidance coefficient, and random number seed.

4. The method for designing a sluice engineering construction scheme based on a stable diffusion large model according to claim 1 is characterized in that: The number of the controlnet units is two; The controlnet parameters include original solution model picture, control type, control weight, control mode, scaling mode, preprocessor, and script.

5. The method for designing a water gate construction scheme based on a stable diffusion large model according to claim 4 is characterized in that: The original scheme model pictures are line drawings of human view and top view produced by building a rough model of the sluice engineering construction scheme using modeling software; the perspective of the human view fits the real human visual height, and the building is upright in the perspective; The top view is a 45-degree angle overlooking the entire construction project.

6. The method for designing a sluice engineering construction scheme based on a stable diffusion large model according to claim 4 is characterized in that: The script includes X / Y / Z icons that can batch produce different lora models and Wensheng diagrams of different sampling methods.

7. The method for designing a water gate construction scheme based on a stable diffusion large model according to claim 1 is characterized in that: The effect diagram in the data set includes an image and corresponding label text; The image perspectives include bird's-eye view and human-eye view; the image size is uniform, distortion-free and clear, and the content can fully reflect the facade features of the sluice building; Add label text to the image. The label text is a complete sentence described in natural language, which can clearly describe the overall and local content and style of the sluice building; The background of the image is filled with pure white, and the "white background" label is added when adding label text; by entering "white background" in the reverse prompt word, the pure white background domain is prevented from appearing in the generated image, so as to train the Lora model to recognize the association between pure color blocks and text labels.

8. The method for designing a water gate construction scheme based on a stable diffusion large model according to claim 1 is characterized in that: Step S2 specifically includes adding the text image to the graphic image for partial redrawing to eliminate the local errors and deviations; then adding the graphic image to the post-processing to optimize the overall image, set the overall style, and enlarge the size to a preset size to output the architectural rendering.

9. The method for designing a sluice engineering construction scheme based on a stable diffusion large model according to claim 8 is characterized in that: Add the text-generated graph to the graphic-generated graph for partial redrawing to eliminate the generated local errors and deviations. Specifically, eliminate the specific content, first describe it in the reverse prompt word, and then add the content to be generated in the forward prompt word; For the upper building columns and the background sky and waterside slopes, the junction needs to be further refined, and the three types of content need to be partially redrawn separately; the door, window and roof styles can be quickly changed to the required style through redrawing, and the structural appearance materials can be quickly changed to the required materials through redrawing.

10. The method for designing a water gate construction scheme based on a stable diffusion large model according to claim 8 is characterized in that: If the generated architectural renderings do not meet the requirements, use Adobe Photoshop to fine-tune the details; When fine-tuning details with Adobe Photoshop, use the AI ​​generation function to quickly generate fills for image content modification, image repair, or creative modification; use the box selection or lasso tool to select the local content that needs to be modified, activate the Generative Fill text generation fill panel, enter concise prompts or leave blanks to freely produce content to generate the final architectural rendering.

Images