A method and system for generating images for rural landscape design based on knowledge graphs and large models

Abstract

This invention relates to the field of image generation technology, and discloses a method and system for generating rural landscape design images based on knowledge graphs and large-scale models. The method includes constructing a knowledge graph covering core elements of rural landscape design; building a multi-level labeling system based on the knowledge graph; acquiring pre-processed rural landscape sample images and labeling them using the multi-level labeling system to obtain a training set specific to the rural landscape design domain; constructing a basic diffusion large-scale model architecture and training it using the training set specific to the rural landscape design domain to obtain a trained rural landscape design image generation model; and using the trained model to generate rural landscape design images. This invention solves the problems of insufficient domain knowledge support and lack of professional data labeling in general large-scale models for rural landscape design image generation, thereby improving the efficiency and quality of rural landscape design image generation.

Description

Technical Field

[0001] This invention relates to the field of image generation technology, and more specifically, to a method and system for generating images of rural landscape design based on knowledge graphs and large models. Background Technology

[0002] Rural landscape design, as a key means to improve the quality of the rural environment and activate the endogenous driving force of rural areas, is in increasing demand. Traditional design methods relying on manual modeling and drawing are inefficient and have limited accuracy, making them unable to meet the actual needs of current rural landscape design. Therefore, the introduction of intelligent tools has become an inevitable trend. Large-scale image generation models based on artificial intelligence, with their powerful learning capabilities and efficient drawing output, offer a new solution for rural landscape design.

[0003] Current mainstream image generation models (such as StableDiffusion and Midjourney) mainly rely on general corpora or material libraries for training, lacking a precise understanding of professional terminology and regional cultural symbols in the field of rural landscape design. This results in problems such as unstable image quality, incompatibility with rural regional characteristics, and difficulty in accurately matching rural scenes. At the same time, existing professional large-scale models in the field of urban and rural planning mostly focus on urban planning and design, urban landscape design, or general architectural design, lacking a systematic construction of a full-element knowledge graph in the field of rural landscape design and high-quality annotation and training of image data. This makes the existing technology have defects in low quality and efficiency in the generation of rural landscape design images. Summary of the Invention

[0004] To overcome the shortcomings of low image generation quality and efficiency in existing rural landscape design image generation technologies, this invention proposes the following technical solution:

[0005] Firstly, this invention proposes a method for generating images of rural landscape design based on knowledge graphs and large models, comprising:

[0006] Construct a knowledge graph covering the core elements and hierarchical classification of rural landscape design;

[0007] A multi-level tagging system is constructed based on the knowledge graph;

[0008] Obtain preprocessed rural landscape sample images, and annotate the rural landscape sample images based on the multi-level labeling system to obtain a training set specifically for the field of rural landscape design.

[0009] A basic diffusion model architecture is constructed, and the basic diffusion model architecture is trained using the training set specifically for the rural landscape design field to obtain a trained rural landscape design image generation model.

[0010] A trained rural landscape design image generation model is used to generate rural landscape design images.

[0011] As a preferred technical solution, a knowledge graph covering the core elements and hierarchical classification of rural landscape design is constructed, including:

[0012] The pre-screened core entities of rural landscape were hierarchically structured to obtain hierarchical classification results.

[0013] Based on the hierarchical classification results, construct structured triples containing head entities, relations, and tail entities;

[0014] The head entity, relation, and tail entity in the structured triple are mapped to the vector space, and the vector representation of the head entity vector and relation vector is optimized so that the spatial distance between the sum of the head entity vector and relation vector and the tail entity vector meets the preset distance threshold condition.

[0015] A knowledge graph for rural landscape design is constructed based on vectorized triples that satisfy the preset distance threshold condition.

[0016] As a preferred technical solution, the basic diffusion model architecture includes a variational autoencoder module, a noise prediction module, a text encoding module, and a multi-channel control module; the variational autoencoder module includes an encoding unit and a decoding unit;

[0017] The encoding unit is used to map the input sample image to a low-dimensional latent space to obtain latent variables;

[0018] The text encoding module is used to encode the input design requirement text to obtain a semantic embedding vector;

[0019] The multi-channel control module is used to extract multimodal features from the input sample image and design requirement text, and to perform feature fusion processing on the multimodal features to generate multimodal conditional fusion features; the multimodal features include edge and morphological control features, spatial awareness and semantic control features, style and content editing features, and special function class features;

[0020] The noise prediction module is used to fuse the latent variables, the semantic embedding vector, and the multimodal conditional fusion features, and to perform noise prediction and denoising operations on the fusion result to obtain denoised latent variables.

[0021] The decoding unit is used to reconstruct the denoised latent variables into a rural landscape design image.

[0022] As a preferred technical solution, the noise prediction module fuses the latent variables, the semantic embedding vector, and the multimodal conditional fusion features, and performs noise prediction and denoising operations on the fusion result to obtain denoised latent variables, including:

[0023] Cross-attention fusion is performed on the latent variables, the semantic embedding vectors, and the multimodal conditional fusion features to generate attention fusion features. Its expression is as follows:

[0024]

[0025] In the formula, For the current time step t Potential variables, For semantic embedding vectors, For multimodal conditional fusion features, For attention query matrix, Key matrix K transpose, The dimension of a single attention head in the attention mechanism. For the attention mechanism's value vector, This is a learnable parameter matrix used to map latent variables and semantic embedding vectors to the attention query matrix Q. This is used to map latent variables and semantic embedding vectors to a key matrix. K The learnable parameter matrix;

[0026] Based on attention fusion features Noise prediction is performed by freezing the main branch and conditional branch in the multi-channel control module, and its expression is as follows:

[0027]

[0028] In the formula, This represents the total prediction noise. This indicates the prediction noise for freezing the main branch. To control the scaling factor of the condition intensity, This represents the prediction noise of the conditional branch;

[0029] Based on the total prediction noise, latent variables are constructed. The ordinary differential equation for the inverse evolution is expressed as follows:

[0030]

[0031] In the formula, The rate of change function for inverse denoising. For the current time step t Noise scheduling parameters, For the current time step t diffusion parameters;

[0032] According to latent variables The inverse evolution of the ordinary differential equation, from the forward diffusion of the final noisy latent variable Initially, T represents the maximum time step. The differential relationship of the ordinary differential equation is solved iteratively using a sampler. When the time step returns to the initial stage, denoising latent variables are generated. .

[0033] As a preferred technical solution, the edge and shape control features include Canny edge features, probabilistic edge features, straight line shape features, and line drawing shape features;

[0034] The multi-channel control module extracts edge and morphological control features from the input sample image, including:

[0035] The Canny edge features are generated by sequentially applying Gaussian filtering, Sobel gradient calculation, non-maximum suppression, and double thresholding to the input sample image using the Canny operator. ;in, and These are the height and width of the feature, respectively;

[0036] Probabilistic edge features are extracted from the input sample image using PIDI-Net, and probabilistic edge features are generated based on these probabilistic edge features. ;

[0037] Line structure detection is performed on the input sample image, the endpoint coordinates of the line are extracted parametrically, and the line shape features are generated. ;in, The number of lines detected;

[0038] Multi-scale line art features are extracted from the input sample image using the HED network, and line art morphology features are generated based on these multi-scale line art features. .

[0039] As a preferred technical solution, the spatial perception and semantic control features include depth information features and semantic segmentation features;

[0040] The multi-channel control module extracts spatial perception and semantic control features from the input sample image, including:

[0041] The MiDaS network is used to predict the monocular depth of the input sample image, and depth information features are generated based on the monocular depth. ;in, and These are the height and width of the feature, respectively;

[0042] Based on the Deeplabv3+ model, semantic segmentation is performed on the input sample images according to the ADE20K protocol to generate semantic segmentation features. ;in, The number of object categories is represented by the segmentation marker for each channel, which corresponds to a category of rural landscape elements.

[0043] As a preferred technical solution, the style and content editing features include style vector features, low-rank adaptation matrix features, and editing mask features;

[0044] The multi-channel control module extracts style and content editing features from the input sample image and design requirement text, including:

[0045] The input sample image is encoded using an IP-Adapter image cue adapter to obtain style vector features. ;in, d The dimension of the feature;

[0046] Based on the design requirements text, a low-rank adaptation matrix feature is constructed using the T2I-Adapter text-to-image adapter. ;

[0047] Based on the design requirements text, generate editing mask features. The elements in the edit mask feature are identified by binary values, representing the areas in the input sample image that need to be redrawn.

[0048] Based on the rural landscape restoration requirements in the design document, a local redraw mask feature was constructed. ;Including local redrawing of mask features This is used to constrain the diffusion model to generate rural landscape details in a specified area.

[0049] As a preferred technical solution, the basic diffusion model architecture is trained using a training set specifically designed for the rural landscape design field, including:

[0050] In all the cross-attention layers of the noise prediction module of the basic diffusion model, a low-rank adapter based on LoRA is injected. The first stage of training is carried out by constructing a first objective function that includes CLIP semantic alignment loss and LPIPS perceptual loss, and the preliminary training model is obtained.

[0051] Obtain the binary evaluation results and number of edits for the generated rural landscape design images;

[0052] A reward function is constructed based on the binary evaluation results and the number of edits.

[0053] Based on the reward function, a second objective function is constructed using a policy optimization algorithm, which includes the importance sampling ratio of the generated action and the advantage function.

[0054] The second training stage is performed on the initial training model based on the second objective function to obtain a trained rural landscape design image generation model.

[0055] As a preferred technical solution, the first objective function The expression is as follows:

[0056]

[0057] In the formula, and These are the weight parameters, Design images of rural landscapes generated by the model. For the corresponding design text, Sample images from a training set specifically designed for the field of rural landscape design. Used to measure the semantic matching degree between the generated graph and the text. Image features extracted by the VGG network;

[0058] reward function The expression is as follows:

[0059]

[0060] In the formula, To evaluate the results of the generated rural landscape design images, =1 indicates that it passes. =0 indicates rejection. This indicates the number of times the rural landscape design image required manual editing;

[0061] Second objective function The expression is as follows:

[0062]

[0063] In the formula, To calculate the expected value of the data at time step t during training, For importance sampling ratio, Indicates the current strategy In rural areas Take action below The probability, Indicates the old strategy before the update. In rural areas Take action below The probability, The dominance function measures the actions taken to generate rural landscape design images. a Compared to the value gain of average action, Indicates will Crop to range [ ].

[0064] Secondly, this invention also proposes a rural landscape design image generation system based on knowledge graphs and large models, applicable to the rural landscape design image generation method based on knowledge graphs and large models as described in any of the solutions in the first aspect, comprising:

[0065] The first construction module is used to build a knowledge graph covering the core elements and hierarchical classification of rural landscape design.

[0066] The second construction module is used to construct a multi-level tag system based on the knowledge graph;

[0067] The annotation module is used to acquire preprocessed rural landscape sample images, and to annotate the rural landscape sample images based on the multi-level labeling system to obtain a training set specifically for the field of rural landscape design.

[0068] The third construction module is used to construct the basic diffusion model architecture. The basic diffusion model architecture is trained using the training set specifically for the rural landscape design field to obtain a trained rural landscape design image generation model.

[0069] The generation module uses a trained rural landscape design image generation model to generate rural landscape design images.

[0070] The beneficial effects of the present invention include at least the following:

[0071] This invention addresses the shortcomings of general-purpose large-scale models in generating images for rural landscape design, such as insufficient domain knowledge support and a lack of professional data annotation. It strengthens the model's semantic understanding of rural domain terms and features by constructing a knowledge graph covering core elements and hierarchical classifications of rural landscape design. Based on this knowledge graph, a multi-level labeling system is built to accurately annotate pre-processed rural landscape sample images, forming a dedicated training set. This training set is then used to train a basic diffusion model architecture, enabling the model to deeply learn professional features of rural landscapes. Finally, the trained model is used to generate images, improving both the efficiency and quality of rural landscape design image generation. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating the method for generating images of rural landscape design based on knowledge graphs and large models provided in an embodiment of the present invention.

[0073] Figure 2 This is a schematic diagram of the technical architecture of the rural landscape design image generation method provided in an embodiment of the present invention.

[0074] Figure 3 This is an example of a knowledge graph for rural landscape design provided in an embodiment of the present invention.

[0075] Figure 4(a) is the original sample image of rural landscape provided in the embodiment of the present invention, (b) is the Canny edge feature map provided in the embodiment of the present invention, (c) is the straight line morphology feature map provided in the embodiment of the present invention, (d) is the line drawing morphology feature map provided in the embodiment of the present invention, (e) is the probabilistic edge feature map provided in the embodiment of the present invention, (f) is the depth information feature map provided in the embodiment of the present invention, (g) is the semantic segmentation feature map provided in the embodiment of the present invention, and (h) is the graffiti constraint feature map provided in the embodiment of the present invention.

[0076] Figure 5 The training loss curve is provided for an embodiment of the present invention.

[0077] Figure 6 This is a schematic diagram of an X / Y visualization chart provided in an embodiment of the present invention.

[0078] Figure 7 This invention provides a rural landscape design image generation system based on knowledge graphs and large models, which is an embodiment of the present invention.

[0079] Figure 8 This is a flowchart illustrating the multidimensional control optimization strategy provided in an embodiment of the present invention.

[0080] Figure 9 (a) is the original sample image II of rural landscape provided in the embodiment of the present invention, (b) is the generated image of the StableDiffusion native basic model provided in the embodiment of the present invention, (c) is the generated image of the rural landscape design image generation model trained by the present invention provided in the embodiment of the present invention, and (d) is the generated image of the general large model provided in the embodiment of the present invention. Detailed Implementation

[0081] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred technical solutions. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred technical solutions are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0082] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0083] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0084] Example 1

[0085] This embodiment proposes a method for generating images of rural landscape design based on knowledge graphs and large models, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for generating images of rural landscape design based on knowledge graphs and large models, as provided in this embodiment. The method includes the following steps:

[0086] S1: Construct a knowledge graph covering the core elements and hierarchical classification of rural landscape design;

[0087] S2: Construct a multi-level tagging system based on the knowledge graph;

[0088] S3: Obtain preprocessed rural landscape sample images, and label the rural landscape sample images based on the multi-level labeling system to obtain a training set specifically for the field of rural landscape design.

[0089] In one example, renderings and real-life photos of rural landscapes with regional characteristics and local culture were collected through multiple channels, including website design, on-site photography, and rural planning and production projects. The collected images were then processed using the OpenCV library to perform noise reduction, contrast enhancement, and size standardization, uniformly adjusting the image resolution to no less than 512×512 pixels and converting the format to PNG. Images with unclear subjects, disorganized expressions, or ambiguities were discarded, resulting in a high-quality dataset of rural landscape sample images. Subsequently, based on a rural landscape knowledge graph, a self-developed annotation tool embedded in the rural landscape knowledge graph was used to perform professional manual annotation of the sample images: the labeling system was set at four levels. The first-level labels divided the rural landscape into six major scenes: natural environment, architecture, landscape, facilities, plants, and general scenes. The second and third-level labels refined the rural landscape categories and specific elements, such as topography, building type, waterfront landscape, road facilities, trees, and seasonal weather. The fourth-level labels further refined the characteristic attributes, style, and material of elements such as stone arch bridges, sloping roofs, and pebble tree pits. Compared to the WD1.4 tagger, this self-developed annotation tool improves the annotation professionalism by more than 75%; compared to BooruDatasetTagManager, the annotation efficiency is improved by more than 50%. Finally, based on the annotation results of this multi-level tagging system, a training set specifically for the field of rural landscape design is constructed.

[0090] S4: Construct a basic diffusion model architecture, and train the basic diffusion model architecture using the training set specifically for the rural landscape design field to obtain a trained rural landscape design image generation model.

[0091] S5: Use the trained rural landscape design image generation model to generate rural landscape design images.

[0092] In the specific implementation process, the core elements of rural landscape design, such as rural architecture, landscape, and natural environment, are first sorted out, and a multi-level classification structure of "scene-category-component" is divided to establish the relationship between elements and construct a knowledge graph of rural landscape design. Then, based on the hierarchical classification of the knowledge graph, multi-level labeling rules covering scene types and element attributes are designed. Subsequently, images are collected from channels such as design renderings and real rural photos, and after preprocessing such as noise reduction and size normalization, high-quality samples are selected and labeled according to multi-level labeling rules to form a special training set for the field of rural landscape design. Then, the StableDiffusion class basic diffusion model is selected, imported into the special training set, and training parameters are set to carry out model training to obtain a rural landscape design image generation model. Finally, the text requirements for rural landscape design are input, and the corresponding rural landscape design images are generated through the trained model.

[0093] Understandably, to address the issues of insufficient domain knowledge support and lack of professional data annotation in the generation of rural landscape design images using general-purpose large-scale models, this paper proposes a solution. This is achieved by constructing a knowledge graph covering the core elements and hierarchical classifications of rural landscape design, thereby strengthening the model's semantic understanding of rural domain terms and features. Based on this knowledge graph, a multi-level labeling system is built to accurately annotate pre-processed rural landscape sample images to form a dedicated training set. This training set is then used to train a basic diffusion model architecture, enabling the model to deeply learn the professional features of rural landscapes. Finally, the trained model is used to generate images, thus improving the efficiency and quality of rural landscape design image generation.

[0094] Example 2

[0095] This embodiment improves upon the rural landscape design image generation method based on knowledge graphs and large models proposed in Embodiment 1, such as... Figure 2 As shown, Figure 2This is a schematic diagram of the technical architecture of the rural landscape design image generation method provided in this embodiment of the invention. It should be noted that in this architecture, the basic model and its core modules form the platform's technical support layer. This module provides the fundamental support for image generation. This layer includes key components such as the VAE (Variational Autoencoder) module, the U-Net module, and the CLIP module. The VAE module extracts image features from the latent space through efficient encoding and decoding processes to ensure that the generated images have high quality and detail representation. The U-Net module, utilizing its symmetrical structure and skip connections, can effectively preserve details during image generation while improving image resolution and accuracy. The CLIP module, through comparative learning between text and images, ensures that the platform can accurately generate design images related to rural landscapes based on natural language descriptions, enhancing the system's semantic understanding. The basic model is built on StableDiffusionXL, possessing powerful image generation capabilities and producing high-quality visual content.

[0096] The professional model for generating rural landscape design images serves as the platform's core engine. Building upon the basic model, this embodiment employs LoRA (Low-Rank Adaptation) technology. Through targeted training of the professional model for generating rural landscape design images, a module is formed and deployed on the AIGC Creative Design Platform, enhancing the model's adaptability to specific rural landscape features. This module enables the platform to adjust the generated images according to the culture, style, and needs of specific rural areas, ensuring optimal results in terms of regional adaptability and stylistic consistency.

[0097] The application platform serves as the user interface for the entire platform, providing a series of functional modules to support the entire process of rural landscape design. The platform includes four functional modules: the basic creation module, which includes four main functions: "text-to-image," "image-to-image," "partial redrawing," and "high-definition restoration." The design scene module includes rural design scenes such as agricultural landscapes, portal nodes, waterfront landscapes, and road landscapes, allowing users to select suitable scenes based on their actual needs. The inspiration marketplace and my gallery modules provide designers with a space for creative exchange and result storage, facilitating the sharing of inspiration and the management of personal work.

[0098] Optionally, a knowledge graph covering the core elements and hierarchical classifications of rural landscape design can be constructed, including:

[0099] The pre-screened core entities of rural landscape were hierarchically structured to obtain hierarchical classification results.

[0100] Based on the hierarchical classification results, construct structured triples containing head entities, relations, and tail entities;

[0101] The head entity, relation, and tail entity in the structured triple are mapped to the vector space, and the vector representation of the head entity vector and relation vector is optimized so that the spatial distance between the sum of the head entity vector and relation vector and the tail entity vector meets the preset distance threshold condition.

[0102] A knowledge graph for rural landscape design is constructed based on vectorized triples that satisfy the preset distance threshold condition.

[0103] ;

[0104]

[0105] In one example, as shown in Table 1 and Figure 3 As shown in Table 1, a four-level classification system defines the core entity system of rural landscape design, covering six core areas: "natural environment, architecture, landscape, facilities, plants, and general scenes." First, the core entities in the architecture and landscape categories are pre-screened and hierarchically structured: "Rural houses" under the architecture category are subdivided into four-level entities, "sloping-roof farmhouses," based on "roof form"; and "public space landscapes" under the landscape category are subdivided into four-level entities, "Lingnan-style public spaces," based on "regional style." Second, a structured tripartite group is constructed, with "sloping-roof farmhouses" as the head entity. h "Landscape type" as the relationship r The "Lingnan-style public space" is the final entity. t Forming a triple ( h , r , t Next, the TransE algorithm is used to map the head entity, relation, and tail entity to a low-dimensional vector space, and the vector representation is optimized so that the spatial distance (such as Euclidean distance) between the sum of the head entity vector and the relation vector and the tail entity vector meets the preset distance threshold. Finally, the iteration is extended to the other four major categories of core entities, such as natural environment, facilities, plants, and general scenes (e.g., the "mountain environment" category of "natural environment" is subdivided into "slope mountain" and the "stone wall" category of "facilities" is subdivided into "Lingnan stone wall", constructing triples (({Lingnan stone wall}, {spatial relation}, {Lingnan banyan tree})) etc.). Based on the vectorized triples that meet the distance threshold, a knowledge graph of rural landscape design is systematically constructed.

[0106] Optionally, the basic diffusion model architecture includes a variational autoencoder module, a noise prediction module, a text encoding module, and a multi-channel control module; the variational autoencoder module includes an encoding unit and a decoding unit;

[0107] The encoding unit is used to map the input sample image to a low-dimensional latent space to obtain latent variables.

[0108] In this embodiment, the variational autoencoder module uses a variational autoencoder (VAE).

[0109] It should be noted that variational autoencoders (VAEs) are probabilistic generative models whose core objective is to achieve efficient data generation and reconstruction by learning the latent distribution of the data. VAEs assume the input data... From latent variables Generated by the decoder, i.e. , where follows a prior distribution It is usually a standard Gaussian distribution. encoder The posterior distribution is parameterized using a neural network, and the mean is output. and variance This maps the input data to the latent space. Decoder Data is reconstructed from latent variables, typically using a Gaussian or Bernoulli distribution. Since the true posterior distribution is difficult to solve directly, VAEs introduce an approximate posterior distribution through variational inference. And maximize the lower bound of evidence (ELBO) on log-likelihood:

[0110]

[0111] The loss function consists of two parts: (1) reconstruction loss, which measures the difference between the input data and the decoded output, commonly using mean squared error (MSE) or cross-entropy; (2) KL divergence, which constrains the similarity between the latent distribution and the prior distribution to avoid overfitting.

[0112] Input sample image via VAE encoder Mapping to a low-dimensional latent space:

[0113]

[0114] In this embodiment, the latent distribution is constrained through variational inference. This reduces the computational complexity of the diffusion model. Positive noise injection affects the latent variables. Perform Markov chain noise superposition and define the forward diffusion process:

[0115]

[0116] in Controlled by a cosine scheduling strategy, ensuring that the noise level from to Monotonically increasing, eventually .

[0117] The text encoding module is used to encode the input design requirement text to obtain a semantic embedding vector.

[0118] In this embodiment, the text encoding module uses CLIP (Contrastive Language-Image Pre-training) TextEncoder.

[0119] It's important to note that CLIP is a multimodal model. Its text encoder is based on a Transformer architecture, achieving semantic alignment between text and images through contrastive learning. Its structure employs a decoder-only Transformer, including multi-head self-attention (MHA) and feedforward network (FFN) layers. After word embedding and positional encoding, the input text captures contextual relationships through a self-attention mechanism, ultimately extracting a labeled vector as the overall text representation. CLIP is trained on a large-scale image-text pair dataset, with the objective function being the infoNCE loss.

[0120]

[0121] in and Normalized embeddings for images and text, respectively. Using temperature as a parameter, this loss maximizes the similarity of positive sample pairs and suppresses negative sample pairs. CLIP maps text and images to the same semantic space, allowing text descriptions to be directly used as guiding conditions for generative models.

[0122] During the iterative optimization of the large model, the input design requirement text y is used to generate semantic embedding vectors through a pre-trained CLIP text encoder. Where L is the sequence length and d=768 is the embedding dimension, the visualization is as follows:

[0123]

[0124] The multi-channel control module is used to extract multimodal features from the input sample image and design requirement text, and to perform feature fusion processing on the multimodal features to generate multimodal conditional fusion features; the multimodal features include edge and morphological control features, spatial awareness and semantic control features, style and content editing features, and special function class features;

[0125] The noise prediction module is used to fuse the latent variables, the semantic embedding vector, and the multimodal conditional fusion features, and to perform noise prediction and denoising operations on the fusion result to obtain denoised latent variables.

[0126] In this implementation, the noise prediction module adopts a U-Net+Scheduler structure.

[0127] It's important to note that U-Net's symmetric encoder-decoder structure and skip connections make it a core architecture for denoising tasks in diffusion models. Its design consists of an encoder (shrinking path), a decoder (expanding path), and skip connections. Specifically, it progressively downsamples through convolution and pooling to extract multi-scale features without retaining global context information. Then, it upsamples through deconvolution or interpolation to gradually restore spatial resolution. Combined with skip connections, it fuses the local detail features corresponding to the encoder, concatenating the feature maps of each encoder layer with those of the corresponding decoder layer. This achieves the fusion of low-level details (such as edges) and high-level semantics (such as object shape), enhancing segmentation and generation accuracy. Between the encoder and decoder, UNetModel's skip connection mechanism connects features from corresponding layers in the encoder with those in the decoder to help retain more spatial information and detail features. During image generation, UNetModel uses DownSample and UpSample to downsample and upsample samples, while also incorporating the interaction of multiple modules such as the ResBlock residual module, timestep_embedding, and SpatialTransformer.

[0128] The decoding unit is used to reconstruct the denoised latent variables into a rural landscape design image.

[0129] Optionally, this embodiment constructs a conditional reverse process. Using the U-Net noise prediction network This study estimates the noise residuals and uses ControlNet as a multi-channel control module to introduce multimodal conditional inputs to enhance the control capability of text-to-image diffusion. The core idea is to freeze and train the weights of the diffusion model and construct a trainable conditional encoding branch to achieve precise alignment between the generated content and the input constraints. ControlNet consists of two parallel networks: a frozen main branch retains all parameters of the trained diffusion model (StableDiffusionU-Net) to ensure that the original generation capability is not degraded; the trainable conditional branch replicates the encoding structure of the main branch and is connected to the main branch through zero-initialized convolutional layers (Zero-Conv), with initial weights close to zero to avoid interfering with the original model. The network fuses three parts of information: latent noise variables... Time step embedding (Injecting timing information via sinusoidal position encoding), text conditional embedding and multimodal conditional fusion features Perform cross-attention fusion to generate attention fusion features. Its expression is as follows:

[0130]

[0131] In the formula, For the current time step t Potential variables, For semantic embedding vectors, For multimodal conditional fusion features, For attention query matrix, Key matrix K transpose, The dimension of a single attention head in the attention mechanism. For the attention mechanism's value vector, This is a learnable parameter matrix used to map latent variables and semantic embedding vectors to the attention query matrix Q. This is used to map latent variables and semantic embedding vectors to a key matrix. K The learnable parameter matrix;

[0132] Based on attention fusion features Noise prediction is performed by freezing the main branch and conditional branch in the multi-channel control module, and its expression is as follows:

[0133]

[0134] In the formula, This represents the total prediction noise. This indicates the prediction noise for freezing the main branch. To control the scaling factor of the condition intensity, This represents the prediction noise of the conditional branch;

[0135] The mean prediction of the reverse process is made by a noise predictor. drive:

[0136]

[0137] Based on the total prediction noise, latent variables are constructed. The ordinary differential equation for the inverse evolution is expressed as follows:

[0138]

[0139] In the formula, The rate of change function for inverse denoising. For the current time step t Noise scheduling parameters, For the current time step tdiffusion parameters;

[0140] According to latent variables The inverse evolution of the ordinary differential equation, from the forward diffusion of the final noisy latent variable Initially, T represents the maximum time step. The differential relationship of the ordinary differential equation is solved iteratively using a sampler. When the time step returns to the initial stage, denoising latent variables are generated. .

[0141] Optionally, the edge and shape control features include Canny edge features, probabilistic edge features, straight line shape features, and line drawing shape features;

[0142] The multi-channel control module extracts edge and morphological control features from the input sample image, including:

[0143] The Canny edge features are generated by sequentially applying Gaussian filtering, Sobel gradient calculation, non-maximum suppression, and double thresholding to the input sample image using the Canny operator. ;in, and These represent the height and width of the features, respectively. The U-Net conditional branch learns the edge-texture mapping relationship through convolutional layers, forcing the generated image to match the texture. Structure alignment.

[0144] Probabilistic edge features are extracted from the input sample image using PIDI-Net, and probabilistic edge features are generated based on these probabilistic edge features. It preserves blurred boundary information and enhances texture continuity through channel weighting.

[0145] Line structure detection is performed on the input sample image, the endpoint coordinates of the line are extracted parametrically, and the line shape features are generated by spatial attention guidance. ;in, The number of lines detected;

[0146] Multi-scale line art features are extracted from the input sample image using the HED (Hierarchical Edge Detection) network, and line art morphology features are generated based on these multi-scale line art features. It preserves hair-level details and injects style information through AdalN.

[0147] Optionally, the spatial perception and semantic control features include depth information features and semantic segmentation features;

[0148] The multi-channel control module extracts spatial perception and semantic control features from the input sample image, including:

[0149] The MiDaS network is used to predict the monocular depth of the input sample image, and depth information features are generated based on the monocular depth. ;in, and These represent the height and width of the features, respectively. The numerical encoding represents the spatial hierarchy of the scene, and the depth map generates resolution by adjusting spatial attention.

[0150] Based on the Deeplabv3+ model, semantic segmentation is performed on the input sample images according to the ADE20K protocol to generate semantic segmentation features. ;in, The number of object categories is represented by a segmentation identifier for each category of rural landscape element. The segmentation map is generated locally by weighting the category channels.

[0151] Optionally, the style and content editing features include style vector features, low-rank adaptation matrix features, and editing mask features;

[0152] The multi-channel control module extracts style and content editing features from the input sample image and design requirement text, including:

[0153] The input sample image is encoded using an IP-Adapter image cue adapter to obtain style vector features. ;in, d As a feature dimension, the generation process is injected through a lightweight adapter, and broad-modal attention is used to fuse style and text.

[0154] Based on the design requirements text, a low-rank adaptation matrix feature is constructed using the T2I-Adapter text-to-image adapter. Construct a low-rank adaptation matrix Fine-tune text-image alignment and dynamically enhance text conditions.

[0155] Based on the design requirements text, generate editing mask features. The elements in the edit mask feature are identified by binary values, representing the areas in the input sample image that need to be redrawn.

[0156] Optionally, the special functional features include graffiti constraint features and local redraw mask features;

[0157] The multi-channel control module extracts special functional class features from the input sample image and design requirement text, including:

[0158] Extract user-generated hand-drawn rural landscape graffiti input from the design requirements text and generate graffiti features. As a loose constraint, allowing AI to freely add details, doodles enhance the creative space through sparse attention, in which... and These represent the feature's height and width, respectively, and 3 represents the number of channels.

[0159] Based on the rural landscape restoration requirements in the design document, a local redraw mask feature was constructed. ;Including local redrawing of mask features This is used to constrain the diffusion model to generate rural landscape details in a specified area. It is based on a mask. Specify the repair area, constrain the diffusion model to maintain the original image in a consistent region, and use latent space mixing and gradient guidance.

[0160] In one example, such as Figure 4 As shown, Figure 4 In this paper, (a) is the original sample image of rural landscape provided in an embodiment of the present invention, which presents the original image of rural landscape and serves as the basic reference for the generation task; (b) is the Canny edge feature map provided in an embodiment of the present invention, which extracts the outline of buildings and environment through Canny edge detection, providing a basis for model constraint of structural boundaries; (c) is the straight line morphology feature map provided in an embodiment of the present invention, which uses MLSD multi-level straight line detection to identify linear features such as building beams, columns, and roads, and strengthens geometric structure control; (d) is the line drawing morphology feature map provided in an embodiment of the present invention, which generates a fine Linear line drawing, refines the details of building doors, windows, decorations, etc., and improves the model's learning accuracy of the structure; (e) is the probability provided in an embodiment of the present invention. (a) The edge feature map outputs a SoftEdge result, simulating the gradual transition between buildings and landscapes, optimizing the natural transition of the generated image; (b) The depth information feature map provided in this embodiment of the invention constructs a depth perception map, distinguishing the spatial layers of buildings, vegetation, and distant views, assisting the model in shaping a three-dimensional sense of depth; (c) The semantic segmentation feature map provided in this embodiment of the invention completes semantic segmentation, using color to label functional areas such as buildings, vegetation, and roads, clarifying the distribution logic of elements; (d) The graffiti constraint feature map provided in this embodiment of the invention displays the Scribble graffiti control result, transforming user-drawn constraints into model-parseable guidance information, realizing customized creative details. The above sub-graphs together construct a multimodal control input system, supporting the ControlNet algorithm in accurately controlling the structure, layers, regions, and creativity in the process of generating rural landscapes.

[0161] Optionally, the basic diffusion model architecture is trained using a training set specifically designed for the rural landscape design domain, including:

[0162] In all the cross-attention layers of the noise prediction module of the basic diffusion model, a low-rank adapter based on LoRA is injected. The first stage of training is carried out by constructing a first objective function that includes CLIP semantic alignment loss and LPIPS perceptual loss, and the preliminary training model is obtained.

[0163] Obtain the binary evaluation results and number of edits for the generated rural landscape design images;

[0164] A reward function is constructed based on the binary evaluation results and the number of edits.

[0165] Based on the reward function, a second objective function is constructed using a policy optimization algorithm, which includes the importance sampling ratio of the generated action and the advantage function.

[0166] The second training stage is performed on the initial training model based on the second objective function to obtain a trained rural landscape design image generation model.

[0167] It should be noted that in all cross-attention layers of the noise prediction module of the basic diffusion model, a low-rank adapter based on LoRA is injected, and a first objective function is constructed that includes CLIP semantic alignment loss and LPIPS perceptual loss. The first stage of training is performed to obtain a preliminary trained model; the first objective function is... The expression is as follows:

[0168]

[0169] In the formula, and These are the weight parameters, Design images of rural landscapes generated by the model. For the corresponding design text, Sample images from a training set specifically designed for the field of rural landscape design. Used to measure the semantic matching degree between the generated graph and the text. Image features extracted by the VGG network;

[0170] Table 2 Training parameters for the first stage

[0171]

[0172] In this embodiment, during the model training phase, a gradient descent optimization framework is used to drive iterative parameter updates. Relying on the backpropagation algorithm, the gradient of the loss function (such as cross-entropy loss, mean squared error, etc.) with respect to the network weights is accurately derived, enabling dynamic parameter adjustment. As shown in Table 2, to balance training efficiency and convergence stability, 50 training epochs are set, and mini-batch data input accelerates gradient updates. Simultaneously, cosine annealing, dynamic decay, and learning rate scheduling mechanisms are introduced. In the early stages, a high learning rate is used to quickly explore the parameter space, while in the later stages, the learning rate is gradually reduced to achieve fine convergence. During training, the loss function value continuously decreases and stabilizes below 0.1, and the validation loss curve does not significantly increase, effectively avoiding overfitting risks and ensuring the model's ability to fit the training data.

[0173] like Figure 5 As shown, Figure 5 This is a training loss curve provided in an embodiment of the present invention, where the horizontal axis represents the training epoch and the vertical axis represents the loss value. Figure 5 As shown, in the initial stage (epoch 0-10), the loss value rapidly oscillates and decreases due to the randomness of mini-batch gradient updates, reflecting that the model quickly explores the parameter space and captures basic data patterns with a high learning rate in the early stage of learning rate scheduling. In the middle stage (epoch 10-35), the loss fluctuation increases but the mean continues to decrease, which is due to the local perturbation of gradient direction / amplitude caused by the difference in the distribution of mini-batch samples. At the same time, the dynamic adjustment of the learning rate prompts the model to jump out of local optima and continuously optimize the feature fitting ability. In the later stage (epoch 35-50), the fluctuation gradually narrows and the loss stably converges to below 0.1, indicating that the model parameter updates tend to be stable and the fitting with the training set data reaches a high level.

[0174] To further evaluate the model's generalization performance and to assess the system's generalization performance, this embodiment uses X / Y / visual charts to conduct multi-dimensional testing, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of the X / Y visualization chart provided in an embodiment of the present invention, where the X-axis corresponds to the model in different training epochs, and the Y-axis corresponds to different weight parameters of the model. Based on the test results, the study determines the optimal weight combination through grid search and selects the epoch of the model with the best performance on the validation set (such as the 35th epoch) as the final training result by combining an early stopping mechanism.

[0175] From an engineering deployment perspective, the feasibility of implementation is assessed by quantizing inference latency and memory footprint. Model pruning is introduced to compress redundant parameters, and quantization is used to reduce the computational accuracy requirements. While ensuring the accuracy of attitude estimation, computational efficiency and hardware compatibility are optimized, ultimately achieving a synergy between high accuracy and engineering practicality.

[0176] In the second stage, emphasis is placed on fine-tuning and alignment optimization. Based on the initial trained model, a reward function that integrates human scoring, semantic matching, and editing costs is constructed. And a second objective function was constructed by employing a proximal strategy optimization algorithm. The initial training model is then trained in the second stage to obtain a well-trained rural landscape design image generation model.

[0177] Among them, the reward function The expression is as follows:

[0178]

[0179] In the formula, To evaluate the results of the generated rural landscape design images, =1 indicates that it passes. =0 indicates rejection. The number of times a rural landscape design image requires manual editing follows an exponentially decaying distribution.

[0180] Second objective function The expression is as follows:

[0181]

[0182] In the formula, To calculate the expected value of the data at time step t during training, For importance sampling ratio, Indicates the current strategy In rural areas Take action below The probability, Indicates the old strategy before the update. In rural areas Take action below The probability of; As the dominant function, through GAE ( ) Calculations used to measure the actions involved in generating rural landscape design images. a Value gain compared to average action; Indicates will Crop to range [ ], cropping threshold .

[0183] The second phase significantly improved the design adoption rate and the accuracy of generating relevant professional terms such as "sloping roof" and "Guangfu style." After evaluation, unqualified models were merged or retrained, and by optimizing the accuracy of the labeling system and iterating hyperparameter combination strategies, the precise expression of professional design language was ultimately achieved.

[0184] Example 3

[0185] like Figure 7 As shown, this embodiment proposes a rural landscape design image generation system based on knowledge graphs and large models, which is applied to the rural landscape design image generation method based on knowledge graphs and large models as described in the above embodiment. It includes: a first construction module 100, a second construction module 200, an annotation module 300, a third construction module 400, and a generation module 500.

[0186] The system comprises the following modules: a first construction module 100 for constructing a knowledge graph covering the core elements and hierarchical classifications of rural landscape design; a second construction module 200 for constructing a multi-level labeling system based on the knowledge graph; a labeling module 300 for acquiring pre-processed rural landscape sample images and labeling them based on the multi-level labeling system to obtain a training set specifically for the rural landscape design field; a third construction module 400 for constructing a basic diffusion model architecture and training it using the training set specifically for the rural landscape design field to obtain a trained rural landscape design image generation model; and a generation module 500 for generating rural landscape design images using the trained rural landscape design image generation model.

[0187] It should be noted that the foregoing explanation of the rural landscape design image generation method based on knowledge graphs and large models also applies to the rural landscape design image generation system based on knowledge graphs and large models in this embodiment, and will not be repeated here.

[0188] This embodiment focuses on high-frequency and typical business needs in rural planning and design. Using the rural landscape design image generation system based on knowledge graphs and large models described in this embodiment as the core technology foundation, it independently developed the AIGC creative design platform and simultaneously carried out the construction of a professional application scenario system and the exploration of AI-assisted design workflows.

[0189] On the one hand, it innovatively constructs professional application scenario modules such as "entrance passages, building facade renovation, beautiful main streets, road environment, small parks and squares, and bird's-eye view landscape," forming a diversified design image generation system covering rural landscape from node details to overall layout. Among them, the rural landscape knowledge graph constructed by the first construction module provides domain knowledge support for the semantic refinement of each scenario; the multi-level labeling system established by the second construction module ensures the accuracy of structured modeling of key elements of scenarios such as "road environment and small parks and squares"; combined with the ControlNet multimodal control mechanism, and with the help of the rural landscape-specific training set jointly created by the annotation module and the third construction module, the model is injected with domain adaptation capabilities, realizing semantic alignment, morphological control, and landscape adaptation in the image generation process—such as the signage system of entrance passages and landscape collaborative design, style normalization and intelligent material replacement of building facades, and pedestrian environment optimization and commercial atmosphere creation of beautiful main streets, accurately matching the needs of rural design business.

[0190] On the other hand, the system explores an iterative human-machine collaborative AI-assisted design workflow. The core process relies on the system's functional modules: after designers submit design requirements via natural language descriptions or sketches, the system invokes a multi-task learning framework (based on knowledge graph semantic parsing capabilities) to simultaneously analyze functional requirements, style preferences, and technical constraints. Designers then use the platform's interactive annotation tools to perform geometric corrections, material replacements, and style adjustments on the initial output of the generated modules. After 3-5 rounds of human-machine iterative optimization, the final output is a solution that meets professional design standards and possesses regional characteristics. Real-world testing data verifies that this workflow, supported by knowledge graphs and a large-scale model system, improves the efficiency of rural design processes by over 30%, achieving a synergistic advancement in both the depth of professional scenario coverage and design efficiency.

[0191] To effectively evaluate the quality of the rural landscape image generation model, this embodiment conducts mathematical and statistical analysis on the generated images using objective evaluation indicators to quantify their technical quality characteristics. Simultaneously, it relies on the subjective perception of professional designers to complete an industry standard compatibility assessment, accurately measuring the degree to which the generated images match actual production needs. This collaborative evaluation method covers both technical performance verification and business requirement compatibility verification, forming a comprehensive and reliable quality verification closed loop.

[0192] In terms of objective evaluation, Fréchet Inception Distance (FID) is selected as the core quantitative indicator. FID is a classic indicator widely used in evaluating the performance of generative models. Its technical principle is as follows: the feature distributions of the generated image set and the real image set are mapped to a high-dimensional space through a pre-trained Inception network, and then fitted to Gaussian distributions respectively; the semantic feature similarity between the two image sets is quantified by calculating the Fréchet distance between the two Gaussian distributions. Specifically, the smaller the FID value, the closer the high-level semantic features of the generated image and the real image are, and the better the diversity and realism of the generated results; when the FID value is 0, it can be determined that the feature distributions of the two image sets are completely identical. Compared with traditional evaluation indicators, FID has two core advantages: first, the high-level semantic features extracted based on the Inception network are more relevant to human visual perception and better fit actual design needs; second, the difference is quantified by statistical distribution distance, and the calculation results are less affected by random noise and have better stability. The mathematical expression for calculating FID is:

[0193]

[0194] in, The mean of features extracted from a real image; The mean of the features extracted from the generated image; The covariance matrix for extracting features from a real image; The covariance matrix for extracting features from a real image; The trace of a matrix.

[0195] In this embodiment, a multi-dimensional control optimization strategy is proposed, such as... Figure 8 As shown, Figure 8 This is a flowchart illustrating the multidimensional control optimization strategy provided in an embodiment of the present invention. Within the rural image generation optimization framework based on the ControlNet module, it follows the following... Figure 8The process logic shown takes the "current state image" as the input starting point and relies on a multimodal conditional control mechanism to perform high-dimensional feature encoding and decoupled representation learning on the input image. A pre-trained StableDiffusion model is selected as the basic generation architecture, integrating four parallel ControlNet branches: Line Art, Depth Map, Line Contour (MLSD), and Segmentation. These branches process the structured control signals presented within the control module: the Line Art branch matches the "Line Composition" sub-image, extracting sketch-like features through a lightweight residual convolutional network to maintain the original scene composition logic; the Depth Map branch matches the "Depth Map" sub-image, using a monocular depth estimation network to construct 3D spatial constraints to ensure geometric consistency between the generated result and the original image; the Line Contour branch matches the "Line Contour" sub-image, using a Hough transform baseline segment detector to enhance the rigidity features of the building structure to anchor key linear elements; and the Segmentation branch matches the "Material Segmentation" sub-image, relying on a CLIP-driven semantic parsing module to accurately map material attributes such as vegetation, buildings, and sky.

[0196] In the parameter optimization phase, an adversarial training strategy is adopted in conjunction with a progressive learning rate decay mechanism to fine-tune the ControlNet weight matrix end-to-end. Dynamic gradient clipping and mixed precision training are introduced simultaneously to stabilize the training process. A multi-scale loss function is constructed using KL divergence and perceptual loss to balance the fidelity and creativity of the generated images.

[0197] In the inference phase, a parameterized control intensity adjustment mechanism is designed, and the condition injection weights are dynamically adjusted based on the cosine annealing strategy: the line control maintains high intensity (0.8-1.0) in the early stage of generation to ensure structural stability, and decays to 0.4-0.6 in the later stage of generation to release the potential diffusion model's detail creation space; for depth map and material segmentation, a layered attention mechanism is used to perform spatial adaptive control intensity allocation, applying strong constraints (≥0.6) to the building outline area, and relaxing constraints (≤0.4) to natural elements such as vegetation and sky to enhance generation diversity.

[0198] Experimental results show that the proposed solution significantly improves the FID (Focus on Identification and Desire) index compared to the baseline model. User studies have verified that the generated images have outstanding advantages in terms of structural rationality and aesthetic expressiveness. Through the end-to-end mapping of "current situation map → control module → generated drawings," the generated results are highly consistent with the current situation map in terms of form, scale, and elements, achieving a dialectical unity between controllable generation and creative divergence. This constructs an intelligent generation solution for rural landscape intelligent design that combines engineering robustness and artistic expression.

[0199] like Figure 9 As shown, Figure 9In the above, (a) is the original sample image II of the rural landscape provided in the embodiment of the present invention, which presents the real status of rural buildings and environment, clearly showing the original state of building materials, spatial layout and surrounding landscape, and serves as a reference for model training; (b) is the generated image of the StableDiffusion native basic model provided in the embodiment of the present invention, which is the generation result of the StableDiffusion native basic model. It is not much different from the current status image, but the image quality is poor; (c) is the generated image of the rural landscape design image generation model trained by the present invention provided in the embodiment of the present invention. It is the output of the model after being trained by the present invention. Relying on the rural landscape knowledge graph and the special training set, while preserving the rural texture of the buildings, it systematically optimizes the landscape layer, strengthens the style unity, fits the rural landscape, and modifies the building facades and roads to meet professional requirements, realizing the integration of rural characteristics and design aesthetics; (d) is the generated image of the general large model provided in the embodiment of the present invention. Although it enhances the atmosphere of the picture through rendering, it has problems such as distortion of rural building details and blurring of regional characteristics. Its style does not meet the needs of rural planning and design.

[0200] Based on this modular and structured application scenario system, the generated rural landscape design elements are more complete and comprehensive, encompassing rural natural environment, architecture, landscape, facilities, native plants, etc.; the matching degree of local rural characteristics reaches 90% (compared to 60% for the general model), the proportion error of landscape elements is ≤5%, and the FID (Frechet Inception Distance) score in the "style generation" task is 12.7, significantly better than the general model (FID=29.4). Simultaneously, in rural planning and design projects, it can assist in solving the image expression needs of early communication, mid-term concept generation, and late-stage project implementation, shortening the design time by more than 30% and reducing the per capita drafting and design workload by approximately 20%, effectively contributing to the advancement of rural living environment improvement work.

[0201] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0202] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0203] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0204] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0205] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0206] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for generating images of rural landscape design based on knowledge graphs and large models, characterized in that, include: Construct a knowledge graph covering the core elements and hierarchical classifications of rural landscape design, including: The pre-screened core entities of rural landscape were hierarchically structured to obtain hierarchical classification results. Based on the hierarchical classification results, construct structured triples containing head entities, relations, and tail entities; The head entity, relation, and tail entity in the structured triple are mapped to the vector space, and the vector representation of the head entity vector and relation vector is optimized so that the spatial distance between the sum of the head entity vector and relation vector and the tail entity vector meets the preset distance threshold condition. A knowledge graph for rural landscape design is constructed based on vectorized triples that satisfy the preset distance threshold condition. A multi-level tagging system is constructed based on the knowledge graph. The multi-level tagging system includes first-level tags for classifying rural landscape scene types, second-level tags for classifying scene constituent element categories, third-level tags for classifying element morphological features, and fourth-level tags for classifying element attribute details. Obtain preprocessed rural landscape sample images, and annotate the rural landscape sample images based on the multi-level labeling system to obtain a training set specifically for the field of rural landscape design. A basic diffusion model architecture is constructed, and the basic diffusion model architecture is trained using the training set specifically for the rural landscape design field to obtain a trained rural landscape design image generation model. A trained rural landscape design image generation model is used to generate rural landscape design images.

2. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 1, characterized in that, The basic diffusion model architecture includes a variational autoencoder module, a noise prediction module, a text encoding module, and a multi-channel control module; the variational autoencoder module includes an encoding unit and a decoding unit; The encoding unit is used to map the input sample image to a low-dimensional latent space to obtain latent variables; The text encoding module is used to encode the input design requirement text to obtain a semantic embedding vector; The multi-channel control module is used to extract multimodal features from the input sample image and design requirement text, and to perform feature fusion processing on the multimodal features to generate multimodal conditional fusion features; the multimodal features include edge and morphological control features, spatial awareness and semantic control features, and style and content editing features; The noise prediction module is used to fuse the latent variables, the semantic embedding vector, and the multimodal conditional fusion features, and to perform noise prediction and denoising operations on the fusion result to obtain denoised latent variables. The decoding unit is used to reconstruct the denoised latent variables into a rural landscape design image.

3. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 2, characterized in that, The noise prediction module fuses the latent variables, the semantic embedding vector, and the multimodal conditional fusion features, and performs noise prediction and denoising operations on the fusion result to obtain denoised latent variables, including: Cross-attention fusion is performed on the latent variables, the semantic embedding vectors, and the multimodal conditional fusion features to generate attention fusion features. Its expression is as follows: In the formula, For the current time step t Potential variables, For semantic embedding vectors, For multimodal conditional fusion features, For attention query matrix, Key matrix K transpose, The dimension of a single attention head in the attention mechanism. For the attention mechanism's value vector, This is a learnable parameter matrix used to map latent variables and semantic embedding vectors to the attention query matrix Q. This is used to map latent variables and semantic embedding vectors to a key matrix. K The learnable parameter matrix; Based on attention fusion features Noise prediction is performed by freezing the main branch and conditional branch in the multi-channel control module, and its expression is as follows: In the formula, This represents the total prediction noise. This indicates the prediction noise for freezing the main branch. To control the scaling factor of the condition intensity, This represents the prediction noise of the conditional branch; Based on the total prediction noise, latent variables are constructed. The ordinary differential equation for the inverse evolution is expressed as follows: In the formula, The rate of change function for inverse denoising. For the current time step t Noise scheduling parameters, For the current time step t diffusion parameters; According to latent variables The inverse evolution of the ordinary differential equation, from the forward diffusion of the final noisy latent variable Initially, T represents the maximum time step. The differential relationship of the ordinary differential equation is solved iteratively using a sampler. When the time step returns to the initial stage, denoising latent variables are generated. .

4. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 2 or 3, characterized in that, The edge and shape control features include Canny edge features, probabilistic edge features, straight line shape features, and line drawing shape features; The multi-channel control module extracts edge and morphological control features from the input sample image, including: The Canny edge features are generated by sequentially applying Gaussian filtering, Sobel gradient calculation, non-maximum suppression, and double thresholding to the input sample image using the Canny operator. ;in, and These are the height and width of the feature, respectively; Probabilistic edge features are extracted from the input sample image using PIDI-Net, and probabilistic edge features are generated based on these probabilistic edge features. ; Line structure detection is performed on the input sample image, the endpoint coordinates of the line are extracted parametrically, and the line shape features are generated. ;in, The number of lines detected; Multi-scale line art features are extracted from the input sample image using the HED network, and line art morphology features are generated based on these multi-scale line art features. .

5. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 2 or 3, characterized in that, The spatial perception and semantic control features include depth information features and semantic segmentation features; The multi-channel control module extracts spatial perception and semantic control features from the input sample image, including: The MiDaS network is used to predict the monocular depth of the input sample image, and depth information features are generated based on the monocular depth. ;in, and These are the height and width of the feature, respectively; Based on the Deeplabv3+ model, semantic segmentation is performed on the input sample images according to the ADE20K protocol to generate semantic segmentation features. ;in, The number of object categories represents the segmentation markers for each category of rural landscape elements.

6. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 2 or 3, characterized in that, The style and content editing features include style vector features, low-rank adaptation matrix features, and editing mask features; The multi-channel control module extracts style and content editing features from the input sample image and design requirement text, including: The input sample image is encoded using an IP-Adapter image cue adapter to obtain style vector features. ;in, d The dimension of the feature; Based on the design requirements text, a low-rank adaptation matrix feature is constructed using the T2I-Adapter text-to-image adapter. ; Based on the design requirements text, generate editing mask features. The elements in the edit mask feature are identified by binary values, representing the areas in the input sample image that need to be redrawn.

7. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 1, characterized in that, The basic diffusion model architecture is trained using a training set specifically designed for the rural landscape design field, including: In all the cross-attention layers of the noise prediction module of the basic diffusion model, a low-rank adapter based on LoRA is injected. The first stage of training is carried out by constructing a first objective function that includes CLIP semantic alignment loss and LPIPS perceptual loss, and the preliminary training model is obtained. Obtain the binary evaluation results and number of edits for the generated rural landscape design images; A reward function is constructed based on the binary evaluation results and the number of edits. Based on the reward function, a second objective function is constructed using a policy optimization algorithm, which includes the importance sampling ratio of the generated action and the advantage function. The second training stage is performed on the initial training model based on the second objective function to obtain a trained rural landscape design image generation model.

8. The method for generating images of rural landscape design based on knowledge graphs and large models according to claim 7, characterized in that, First objective function The expression is as follows: In the formula, and These are the weight parameters, Design images of rural landscapes generated by the model. For the corresponding design text, Sample images from a training set specifically designed for the field of rural landscape design. Used to measure the semantic matching degree between the generated graph and the text. Image features extracted by the VGG network; reward function The expression is as follows: In the formula, To evaluate the results of the generated rural landscape design images, =1 indicates that it passes. =0 indicates rejection. This indicates the number of times the rural landscape design image required manual editing; Second objective function The expression is as follows: in, In the formula, To calculate the expected value of the data at time step t during training, For importance sampling ratio, Indicates the current strategy In rural areas Take action below The probability, Indicates the old strategy before the update. In rural areas Take action below The probability, The dominance function measures the actions taken to generate rural landscape design images. a Compared to the value gain of average action, Indicates will Crop to range [ ].

9. A rural landscape design image generation system based on knowledge graphs and large models, applied to the rural landscape design image generation method based on knowledge graphs and large models as described in any one of claims 1 to 8, characterized in that, include: The first construction module is used to build a knowledge graph covering the core elements and hierarchical classification of rural landscape design. The second construction module is used to construct a multi-level tag system based on the knowledge graph; The annotation module is used to acquire preprocessed rural landscape sample images, and to annotate the rural landscape sample images based on the multi-level labeling system to obtain a training set specifically for the field of rural landscape design. The third construction module is used to construct the basic diffusion model architecture. The basic diffusion model architecture is trained using the training set specifically for the rural landscape design field to obtain a trained rural landscape design image generation model. The generation module uses a trained rural landscape design image generation model to generate rural landscape design images.

Images