Landscape design system based on artificial intelligence human-computer interaction

Abstract

The application discloses a landscape design system based on artificial intelligence human-computer interaction, which comprises a multi-modal analysis module for extracting keywords and semantic description and generating user intention data; a structure segmentation module for processing site images, generating semantic segmentation mask images, and combining user intention data to construct structure control data; a text prompt module for converting user intention data into text generation prompt information; a diffusion generation module for receiving text generation prompt information and structure control data to generate a landscape design graph; an interactive feedback module for displaying the landscape design graph, updating semantic segmentation mask images and text generation prompt information, and generating control condition data; a three-dimensional modeling module for three-dimensional reconstruction to generate a three-dimensional landscape representation; and a space feedback module for extracting space parameters, outputting ecological analysis results, and feeding back to an interactive interface. The application realizes intelligent optimization of multi-modal intention-driven landscape graph generation and three-dimensional structure modeling.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and human-computer interaction, and in particular to a landscape design system based on artificial intelligence human-computer interaction. Background Technology

[0002] In traditional landscape design processes, professional designers typically rely on CAD and SketchUp tools for 2D drafting and 3D modeling, with design decisions primarily based on written requirements and site maps. This process is cumbersome and involves long information conversion chains, especially in the initial conceptualization phase. Designers often have to switch between multiple rounds of communication, drawing revisions, and effect previews, resulting in low efficiency and a strong reliance on design experience, with limited user participation. Furthermore, traditional methods mainly rely on manual or parametric modeling, lacking automatic generation capabilities and failing to achieve intelligent mapping from user intent to design graphics.

[0003] In recent years, with the development of generative artificial intelligence and image generation models, some studies have begun to attempt to use text-to-image generation methods for style sketching or interior scene composition. However, most of these techniques focus on static image generation, lacking an understanding of site structure, semantic regions, and three-dimensional space, and especially lacking structural control capabilities, making it difficult to achieve landscape designs with strong controllability and conforming to real spatial relationships. At the same time, existing 3D modeling methods based on image segmentation are mostly used for urban modeling or individual building reconstruction, lacking interactive expression mechanisms for multi-semantic and multi-scale landscape spaces.

[0004] More importantly, most existing generation systems cannot support real-time modifications and feedback from users during the design process, and lack a complete closed-loop optimization path from 2D images to 3D spatial parameters and then to ecological performance. The lack of coordination mechanisms between multiple stages such as structure preservation, semantic consistency, and ecological assessment makes it difficult for design results to meet practical implementation requirements, and the ecological rationality is also difficult to verify.

[0005] Therefore, how to provide a landscape design system based on artificial intelligence and human-computer interaction is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a landscape design system based on artificial intelligence-assisted human-computer interaction. This invention fully integrates core technologies such as multimodal intent parsing, semantic segmentation and perception, text prompt generation, controllable image diffusion, 3D geometric reconstruction, and ecological performance feedback analysis. It details the entire process from starting with user intent to generating a structurally controllable landscape design drawing, then reconstructing the 3D spatial structure and extracting spatial parameters. This method possesses advantages such as intelligent design response, precise structural expression, real-time user interaction, and closed-loop ecological feedback, significantly improving landscape design efficiency and spatial rationality. It is suitable for next-generation landscape planning systems oriented towards intelligent assisted design.

[0007] According to an embodiment of the present invention, a landscape design system based on artificial intelligence human-computer interaction includes:

[0008] The multimodal parsing module is used to receive voice, text, image and hand-drawn sketch input, extract keywords and semantic descriptions, and generate user intent data;

[0009] The structure segmentation module is used to process site images, generate semantic segmentation mask maps, and construct structure control data by combining user intent data.

[0010] The text prompt module is used to convert user intent data into text to generate prompt information;

[0011] The diffusion generation module is used to receive text generation prompts and structural control data to generate landscape design drawings;

[0012] The interactive feedback module is used to display landscape design drawings, collect user operations and update semantic segmentation mask images and text generation prompts, and generate control condition data.

[0013] The 3D modeling module is used to receive landscape design drawings and control condition data, perform 3D reconstruction, and generate a 3D landscape representation.

[0014] The spatial feedback module is used to extract spatial parameters based on the 3D landscape representation, output ecological analysis results, and feed them back to the interactive interface.

[0015] Optionally, modules can be integrated using the following methods:

[0016] S1. Collect multimodal information from user input to construct user intent data;

[0017] S2. Construct an object segmentation model, collect site images uploaded by users, and process the site images based on user intent data and site images through the object segmentation model to generate semantic segmentation mask images and form structural control data.

[0018] S3. Construct text generation prompts based on user intent data, input the text generation prompts and structural control data into the conditional control diffusion network, execute the diffusion generation process, and output the preliminary landscape design drawing.

[0019] S4. Display the preliminary landscape design drawing in the visual human-computer interaction interface, receive user interaction operations, update the semantic segmentation mask image and text generation prompt information, and generate control condition data.

[0020] S5. Input the control condition data into the condition control diffusion network, execute the re-diffusion generation process, output the optimized landscape design drawing, analyze the landscape design drawing, generate a three-dimensional landscape representation, extract the spatial parameters of structural layout, functional distribution and light distribution, and form landscape spatial data.

[0021] S6. Based on landscape spatial data, conduct ecological performance analysis and output ecological assessment indicators including sunshine duration distribution, ventilation path, water body layout, green space ratio and spatial connectivity to form ecological performance data.

[0022] S7. Compare and analyze the ecological performance data with the user intent data, generate feedback information and provide it to the user, and the user performs the interactive operation in step S4 based on the feedback information.

[0023] Optionally, the multimodal information specifically includes voice input, image input, text input, and hand-drawn sketch input.

[0024] Optionally, S2 specifically includes:

[0025] S21. Collect the site images uploaded by the user, and convert the site images into a three-channel image matrix I∈R with size H×W. H×W×3 Where H is the image height, W is the image width, and R is the set of real numbers;

[0026] S22. Based on user intent data, extract the target keyword set K = {k1,k2,…,k}. n}, where each keyword k i This represents the type of landscape region that the user is interested in, i∈{1,2,…,n}, where n is the number of semantic targets input by the user;

[0027] S23. Construct multimodal input information for prompting, wherein the multimodal input information includes a spatial cue point set P = {p1, p2, ..., p...} m}, where each p j Let T = {t1, t2, ..., t} be the image coordinate points and the keyword semantic set T = {t1, t2, ..., t}. n}, where each t i Corresponding to keyword k iThe semantic description of j∈{1,2,…,m}, where m is the number of prompts input by the user;

[0028] S24. Construct an object segmentation model, which includes an image encoder, a cue encoder, and a mask decoder. Input the three-channel image matrix I into the image encoder to extract the image feature tensor F. Input the cue information space cue point set and the keyword semantic set into the cue encoder to generate the cue vector E.

[0029] S25. Input the image feature tensor F and the cue vector E into the mask decoder, and output the candidate mask set.

[0030] S26. Calculate each candidate mask. With keyword k i The semantic consistency score between them, the scoring function is:

[0031] S27. Based on the calculation result of the scoring function, select the mask with the highest score. As an effective mask, and perform boundary refinement processing;

[0032] S28. Assign a unique semantic label to each semantic region in the effective mask. i This forms a semantic mapping set;

[0033] S29. Encode the effective mask and semantic mapping set into structural control data C. struct , which serves as the input control condition for the image generation process.

[0034] Optionally, S3 specifically includes:

[0035] S31. Based on user intent data, extract the target keyword set K and the keyword semantic set T;

[0036] S32. Construct a text prompt string P by assembling a semantic description set according to a preset template. text Used to express the overall design intent, the preset template refers to a rule structure that splices together the contents of the target keyword set into a coherent text prompt according to spatial relationships and functional logic. The text prompt string is then encoded to generate an overall text prompt embedding vector v. text ;

[0037] S33, Transfer structural control data C struct The input condition controls the control branch path of the diffusion network, and the spatial mask feature map F is extracted through multi-layer convolution and downsampling operations. m ;

[0038] S34. Extract each semantic tag. i Corresponding keyword ki semantic embedding vector Spatial mask feature map F m Perform cross-modal attention fusion to generate a spatial-semantic fusion feature map F. fusion ;

[0039] S35. Based on keyword semantic matching degree, user intent salience, and historical interaction frequency, calculate the weighting coefficient w for each masked region. i The keyword semantic matching degree is calculated based on the cosine similarity of the embedded vectors, the user intent salience is scored according to the explicitness of the keywords in the user input, and the historical interaction frequency is obtained based on the statistics of the user's actual regional operation behavior during the design process.

[0040] S36. Adjust the weighting coefficient w i The image is mapped to the image space to form a pixel-level weight map W, which is then weighted pixel-wise with the spatial-semantic fusion feature map to obtain a weighted control feature map F. control ;

[0041] S37. Perform hierarchical processing on the structural control data, and extract control features F from the main structural region and the detail region respectively. (g) With F (f) And inject it into different levels of the conditional control diffusion network, the former into the shallow coding path and the latter into the deep decoding path;

[0042] S38. Construct diffusion-generated input pairs (P) text ,F control The initial state of the diffusion process is initialized to a Gaussian noise image. Image x represents Gaussian noise. T It is sampled from a Gaussian distribution with a mean of 0 and a covariance of identity matrix I;

[0043] S39. Perform a denoising process at each diffusion step t, and introduce a structure-semantic consistency feedback adjustment factor η. t The image state is updated at each diffusion time t by adjusting the structure-semantic consistency feedback factor.

[0044] S310. Complete the diffusion process and output the final image x0 as the preliminary landscape design drawing G=x0. The spatial structure is guided by the control feature map, and the visual semantics are guided by the text prompts to meet the user's intent requirements.

[0045] Optionally, S4 specifically includes:

[0046] S41. Display the generated preliminary landscape design drawing in the visual human-computer interaction interface. Users can view the various functional areas and layout content on the interface.

[0047] S42. Receive user interaction input in the interface, the interaction input including clicking, selecting, dragging, outlining, dragging, semantic annotation, deleting or renaming operations on areas in the image;

[0048] S43. Based on user interaction input, modify the original semantic segmentation mask image, including adding, merging, adjusting boundaries and reclassifying mask regions, and generate an updated semantic segmentation mask image.

[0049] S44. Simultaneously modify the initial text to generate prompt information. The modification includes reorganizing the text structure according to the new area layout or label order, and replacing and adding keyword description content.

[0050] S45. Combine the updated semantic segmentation mask with the updated text generation prompt information to generate control condition data for structural control information and semantic control information.

[0051] S46. Output the updated control condition data to guide the next round of conditional control diffusion generation process, thereby achieving interactive image optimization.

[0052] Optionally, S5 specifically includes:

[0053] S51. Using the generated control condition data as input, the conditional control diffusion network is invoked to execute the diffusion generation process, and the optimized landscape design map G is output. ′ ;

[0054] S52, Regarding the landscape design drawing G ′ Semantic region parsing is performed to extract the geometric boundaries and semantic label information of each semantic region, and a graph structure is constructed. Where V represents the set of region nodes and E represents the set of region adjacency relationships;

[0055] S53, Graph-based structure Construct a graph neural network encoder for each node v j Perform spatial structure encoding to generate region embedding vector h j This forms a set of region embeddings, H;

[0056] S54. Input the semantic segmentation mask image and the region embedding set H into the structure-preserving mapping function to generate a three-dimensional structural model. The three-dimensional structural model includes the region contour shape, spatial orientation information and semantic topology. The structure-preserving mapping function reconstructs the two-dimensional semantic region into a three-dimensional landscape expression that maintains the original structural relationship by fusing semantic boundaries and structural embedding features, combined with spatial topological constraints and geometric consistency rules. Based on the three-dimensional structural model, a three-dimensional landscape representation with quantifiable spatial scale, real proportional structure and semantic attribute annotation is constructed. The three-dimensional landscape representation includes landscape contour, vertical height, spatial combination method and material texture mapping information.

[0057] S55. For each region node v in the three-dimensional structural model j Perform spatial layout extraction operation to generate spatial parameter vector s j ;

[0058] S56, All spatial parameter vectors of all regions {s j The data is associated with the corresponding semantic labels to output the labeled landscape spatial dataset.

[0059] The beneficial effects of this invention are:

[0060] This invention, by introducing a conditionally controlled diffusion network and a structure-preserving 3D modeling mechanism, achieves a complete closed-loop design process, from multimodal user intent to controllable landscape image generation, 3D model construction, and ecological performance feedback. This overcomes the shortcomings of existing technologies, such as poor generation controllability, weak 3D structural expression, low interactivity, and lack of ecological feedback mechanisms. By combining semantic segmentation with graph structure modeling, the system can not only accurately analyze functional areas in site images but also guide image generation based on semantic masks and text prompts, ensuring a high degree of consistency between the graphic output and user intent. During the visual human-computer interaction process, users can directly edit the morphology and semantic content of regions. The system can then optimize the design drawing generation and 3D reconstruction in real time based on updated control conditions, significantly improving design efficiency and user engagement.

[0061] Furthermore, the 3D landscape representation constructed in this invention not only preserves the geometric and semantic features of the region but also establishes structural connections between regions through spatial topological relationships, providing precise support for subsequent extraction of spatial layout parameters. Through the proposed adaptive spatial layout extraction mechanism, the system can extract spatial indicators such as the directionality, positional relationships, and orientation consistency of each functional area, and further conduct ecological performance analysis, achieving coordinated optimization from perception to generation, modeling, and evaluation. This multi-module coupled intelligent design approach not only improves the design response speed of landscape schemes but also significantly enhances the scientific and ecological rationality of spatial configuration, possessing high practical application value and promising prospects for promotion. Attached Figure Description

[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0063] Figure 1 This is a flowchart of the landscape design system based on artificial intelligence human-computer interaction proposed in this invention;

[0064] Figure 2 This is a schematic diagram illustrating the visualized human-computer interaction operation and control condition update of the landscape design system based on artificial intelligence human-computer interaction proposed in this invention. Detailed Implementation

[0065] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0066] refer to Figure 1 and Figure 2 Landscape design systems based on artificial intelligence and human-computer interaction include:

[0067] The multimodal parsing module is used to receive voice, text, image and hand-drawn sketch input, extract keywords and semantic descriptions, and generate user intent data;

[0068] The structure segmentation module is used to process site images, generate semantic segmentation mask maps, and construct structure control data by combining user intent data.

[0069] The text prompt module is used to convert user intent data into text to generate prompt information;

[0070] The diffusion generation module is used to receive text generation prompts and structural control data to generate landscape design drawings;

[0071] The interactive feedback module is used to display landscape design drawings, collect user operations and update semantic segmentation mask images and text generation prompts, and generate control condition data.

[0072] The 3D modeling module is used to receive landscape design drawings and control condition data, perform 3D reconstruction, and generate a 3D landscape representation.

[0073] The spatial feedback module is used to extract spatial parameters based on the 3D landscape representation, output ecological analysis results, and feed them back to the interactive interface.

[0074] In this embodiment, the modules are interconnected using the following method:

[0075] S1. Collect multimodal information from user input to construct user intent data, which is constructed by fusing multimodal information from voice input, text description, image annotation, and hand-drawn sketches;

[0076] S2. Construct an object segmentation model, collect site images uploaded by users, and process the site images based on user intent data and site images through the object segmentation model to generate semantic segmentation mask images and form structural control data.

[0077] S3. Construct text generation prompts based on user intent data, input the text generation prompts and structural control data into the conditional control diffusion network, execute the diffusion generation process, and output the preliminary landscape design drawing.

[0078] S4. Display the preliminary landscape design drawing in the visual human-computer interaction interface, receive user interaction operations, update the semantic segmentation mask image and text generation prompt information, and generate control condition data.

[0079] S5. Input the control condition data into the condition control diffusion network, execute the re-diffusion generation process, output the optimized landscape design drawing, analyze the landscape design drawing, generate a three-dimensional landscape representation, extract the spatial parameters of structural layout, functional distribution and light distribution, and form landscape spatial data.

[0080] S6. Based on landscape spatial data, conduct ecological performance analysis and output ecological assessment indicators including sunshine duration distribution, ventilation path, water body layout, green space ratio and spatial connectivity to form ecological performance data.

[0081] S7. Compare and analyze the ecological performance data with the user intent data, generate feedback information and provide it to the user, and the user performs the interactive operation in step S4 based on the feedback information.

[0082] The landscape design method described in this invention constructs a closed-loop intelligent design process by integrating key steps such as multimodal input parsing, semantic structure segmentation, condition-controlled diffusion generation, 3D reconstruction, and ecological feedback analysis. Compared with traditional landscape design methods based on manual modeling or static generation, this invention has significant advantages such as high structural control accuracy, fast response speed, strong interactivity, and complete spatial expression. By introducing a multi-object segmentation model and text prompts to jointly control the diffusion process, accurate mapping of user intent to image generation can be achieved. Utilizing a visual interface and interactive feedback mechanism, users can adjust the design results in real time, and the system dynamically updates the generated results accordingly. Furthermore, through image parsing and structural reconstruction, a 3D landscape representation that can be used for analysis is formed, and spatial parameters are extracted for ecological performance assessment, achieving an effective connection from visual generation to scientific decision-making. This invention significantly improves the intelligence level and ecological rationality of the landscape design process and is suitable for the construction of a new generation of digital design platforms.

[0083] In this embodiment, the multimodal information specifically includes voice input, image input, text input, and hand-drawn sketch input.

[0084] In this embodiment, S2 specifically includes:

[0085] S21. Collect the site images uploaded by the user, and convert the site images into a three-channel image matrix I∈R with size H×W. H×W×3 Where H is the image height, W is the image width, and R is the set of real numbers;

[0086] S22. Based on user intent data, extract the target keyword set K = {k1,k2,…,k}. n}, where each keyword k i This represents the type of landscape region that the user is interested in, i∈{1,2,…,n}, where n is the number of semantic targets input by the user;

[0087] S23. Construct multimodal input information for prompting, wherein the multimodal input information includes a spatial cue point set P = {p1, p2, ..., p...} m}, where each p j Let T = {t1, t2, ..., t} be the image coordinate points and the keyword semantic set T = {t1, t2, ..., t}. n}, where each t i Corresponding to keyword k i The semantic description of j∈{1,2,…,m}, where m is the number of prompts input by the user;

[0088] S24. Construct an object segmentation model, which includes an image encoder, a cue encoder, and a mask decoder. Input the three-channel image matrix I into the image encoder to extract the image feature tensor F. Input the cue information space cue point set and the keyword semantic set into the cue encoder to generate the cue vector E.

[0089] S25. Input the image feature tensor F and the cue vector E into the mask decoder, and output the candidate mask set.

[0090] S26. Calculate each candidate mask. With keyword k i The semantic consistency score between them, the scoring function is:

[0091]

[0092] in, This represents the intersection-union ratio (IoU) between the mask and the reference region. Indicates the Hausdorff distance between boundary contours. These are the embedding vectors for the mask region and the semantic keywords, respectively, where α, β, and γ are weight coefficients, and d... max This represents the maximum acceptable boundary difference distance value in the current image segmentation task. Indicates candidate mask With keyword k i The cosine similarity between the corresponding semantic vectors Indicates candidate mask Visual semantic embedding vectors, Indicates keyword k i Text semantic embedding vectors, Indicates the relationship with keyword k i The corresponding reference mask area;

[0093] The scoring function aims to measure the degree of matching between candidate mask regions and user-input keywords at both the spatial structure and semantic context levels, thereby selecting the segmentation result that best matches the user's intent. This function integrates three key metrics: Intersection over Union (IoU), boundary distance, and semantic similarity. IoU reflects the degree of overlap between the candidate mask and the reference region in terms of area; a higher IoU indicates more accurate structural positioning. Boundary distance measures the maximum deviation between mask boundary contours, reflecting the fineness of the geometric boundaries; a lower BoU is better. Semantic similarity calculates the consistency between the visual semantics of the image and the semantics of the keywords through embedding vectors, and is a key indicator to ensure a true correspondence between semantic labels and image regions. By weighted and fused these three metrics, this function comprehensively evaluates the effectiveness of candidate masks from multiple dimensions, helping to enhance semantic interpretability while maintaining structural accuracy, thus providing a stable and reliable control data input foundation for subsequent image generation.

[0094] S27. Based on the calculation result of the scoring function, select the mask with the highest score. As an effective mask, and perform boundary refinement processing;

[0095] S28. Assign a unique semantic label to each semantic region in the effective mask. i This forms a semantic mapping set;

[0096] S29. Encode the effective mask and semantic mapping set into structural control data C. struct , which serves as the input control condition for the image generation process.

[0097] This invention significantly improves the quality and controllability of structural control data by introducing a structured object segmentation model to perform accurate semantic analysis of user-uploaded site images. By fusing image features with user multimodal prompts, the system can achieve high-precision mask generation for specific semantic regions. Compared to traditional image segmentation methods, this invention introduces a semantic consistency scoring function, comprehensively considering the spatial overlap, boundary similarity, and semantic embedding similarity between the mask and keywords, resulting in a high degree of structural and semantic matching in the generated results. Simultaneously, it employs boundary refinement and semantic label allocation mechanisms to ensure the semantic independence and spatial recognition accuracy of each region. The resulting structural control data not only possesses advantages such as strong interpretability and clear structure but can also serve as precise input for generating conditional control diffusion images, effectively ensuring the consistency between subsequent design drawings and user intent. This invention represents a significant technical improvement in image segmentation accuracy, multimodal information fusion capabilities, and subsequent generation guidance, making it suitable for critical scenarios in intelligent design systems where structural controllability is a key requirement.

[0098] In this embodiment, S3 specifically includes:

[0099] S31. Based on user intent data, extract the target keyword set K and the keyword semantic set T;

[0100] S32. Construct a text prompt string P by assembling a semantic description set according to a preset template. text Used to express the overall design intent, the preset template refers to a rule structure that splices together the contents of the target keyword set into a coherent text prompt according to spatial relationships and functional logic. The text prompt string is then encoded to generate an overall text prompt embedding vector v. text ;

[0101] S33, Transfer structural control data C struct The input condition controls the control branch path of the diffusion network, and the spatial mask feature map F is extracted through multi-layer convolution and downsampling operations. m ;

[0102] S34. Extract each semantic tag. i Corresponding keyword k i semantic embedding vector v ki , and spatial mask feature map F m Perform cross-modal attention fusion to generate a spatial-semantic fusion feature map F. fusion ;

[0103] S35. Based on keyword semantic matching degree, user intent salience, and historical interaction frequency, calculate the weighting coefficient w for each masked region. iThe keyword semantic matching degree is calculated based on the cosine similarity of the embedding vectors, the user intent salience is scored according to the explicitness of the keywords in the user input, and the historical interaction frequency is obtained based on the statistical analysis of the user's actual regional operation behavior during the design process.

[0104] w i =σ(λ1·cos(v) ki ,v text )+λ2·u i +λ3·f i );

[0105] Where σ() is the activation function, v text Embed the vector for the overall text prompt, u i For the significance score of intent, f i λ1, λ2, and λ3 are the interaction frequency values, λ1, λ2, and λ3 are the weighting coefficients, and cos is the cosine function;

[0106] Weighting coefficient w i The formula aims to adaptively adjust the guidance intensity of different semantic regions during image generation, thereby improving the accuracy of structural control and the expressiveness of user intent. This formula comprehensively considers three key factors: semantic matching degree, user intent salience, and historical interaction frequency. Semantic matching degree is measured by the cosine similarity between the semantic vector of the keyword and the semantic embedding of the overall text prompt, ensuring that the guidance region semantically aligns with user needs. The intent salience score reflects the user's priority for specific keywords. The interaction frequency value records the frequency of user operations on a certain type of region in previous designs, reflecting their behavioral preferences. These three factors are weighted and summed, then mapped to normalized adjustment coefficients via an activation function, enabling the system to differentiate the generation intensity of each region during spatial generation. This mechanism endows the model with stronger personalized expressive capabilities and structural control flexibility, avoiding the problems of "meaning" or "ignoring" semantic regions in the generated image, significantly enhancing the relevance, accuracy, and user satisfaction of the generation effect.

[0107] S36. Adjust the weighting coefficient w i The image is mapped to the image space to form a pixel-level weight map W, which is then weighted pixel-wise with the spatial-semantic fusion feature map to obtain a weighted control feature map F. control ;

[0108] S37. Perform hierarchical processing on the structural control data, and extract control features F from the main structural region and the detail region respectively. (g) With F (f)The structural control data is injected into different levels of the conditional control diffusion network. The former is injected into the shallow encoding path, and the latter into the deep decoding path. The hierarchical processing of the structural control data refers to dividing the structural control data into main structure regions and detail regions according to the spatial scale, semantic weight and design priority relationship of each region in the semantic segmentation mask. After extracting their control features, they are injected into the diffusion network in a hierarchical manner. The main structure is used to guide the overall layout configuration and is injected into the shallow encoding path, while the detail regions are used for fine generation and are injected into the deep decoding path, thereby realizing master-slave control and fine-grained structure preservation in structural generation.

[0109] S38. Construct diffusion-generated input pairs (P) text ,F control The initial state of the diffusion process is initialized to a Gaussian noise image. Image x represents Gaussian noise. T It is sampled from a Gaussian distribution with a mean of 0 and a covariance of identity matrix I;

[0110] S39. Perform a denoising process at each diffusion step t, and introduce a structure-semantic consistency feedback adjustment factor η. t :

[0111]

[0112] The image state is updated at each diffusion time t using a structure-semantic consistency feedback adjustment factor:

[0113] x t-1 =Denoise(x t |P text ,F control ,t,η t );

[0114] Where, x t The image state at the current diffusion step t, μ and δ are weighting coefficients, and IoU(x) t C struct ) represents the currently generated image x t With structural control data C struct Structural matching score between them Represents the current image x t Extracted visual semantic embedding vector, v text The semantic embedding vector represents the text-generated prompt information, where Denoise is the denoising function, and x t-1 The image state at diffusion step t-1;

[0115] The structure-semantic consistency feedback adjustment factor formula aims to enhance the dynamic adaptability of structure control and semantic guidance during the diffusion image generation process. This formula integrates information from both structure matching scores and semantic similarity, constructing an adjustment mechanism that dynamically changes with the number of diffusion steps. The structure matching score, based on the intersection-union ratio (IUU) between the currently generated image and the structure control data, reflects whether the generated image maintains the user-specified spatial layout. Semantic similarity is measured by the cosine similarity between the visual semantic embedding vector of the current image and the semantic vector of the text prompt, reflecting the semantic fit between the generated content and the user's intent. By weighting and combining these two indicators, a adjustment factor is formed to guide the image state update at each diffusion step, effectively suppressing structure drift and semantic deviation. This mechanism not only enhances the model's ability to understand complex intents but also significantly improves the consistency and stability of image generation, ensuring that the final output landscape image possesses both structural rationality and semantic accuracy, providing high-quality input for subsequent 3D modeling and ecological assessment.

[0116] S310. Complete the diffusion process and output the final image x0 as the preliminary landscape design drawing G=x0. The spatial structure is guided by the control feature map, and the visual semantics are guided by the text prompts to meet the user's intent requirements.

[0117] This invention achieves high controllability and semantic consistency in the image generation process by constructing a conditional control diffusion mechanism that integrates structural control and semantic cues. By introducing user intent data and keyword semantic embedding, and combining it with structural control data to construct a spatial-semantic fusion feature map, the generated image simultaneously meets user needs in terms of structural distribution and semantic expression. In particular, in the calculation of the weighting coefficients, semantic matching degree, intent salience, and interaction frequency are introduced as weighting factors, effectively achieving adaptive control of important regions and enhancing the system's responsiveness in the context of user interaction. Through a hierarchical structural injection strategy, main structural features and detail features are guided into different levels of the diffusion network, improving the structural fidelity and detail richness of the generated image. Simultaneously, a structural-semantic consistency feedback adjustment mechanism is introduced to dynamically adjust the image at each diffusion step, effectively reducing the risk of structural deviation and semantic drift. The final output preliminary landscape design map strictly corresponds to the structural control data in spatial layout and accurately reflects the textual intent in visual presentation, laying a high-quality image foundation for subsequent modeling and optimization, and significantly improving the intelligence, accuracy, and controllability of the landscape generation system.

[0118] In this embodiment, S4 specifically includes:

[0119] S41. Display the generated preliminary landscape design drawing in the visual human-computer interaction interface. Users can view the various functional areas and layout content on the interface.

[0120] S42. Receive user interaction input in the interface, the interaction input including clicking, selecting, dragging, outlining, dragging, semantic annotation, deleting or renaming operations on areas in the image;

[0121] S43. Based on user interaction input, modify the original semantic segmentation mask image, including adding, merging, adjusting boundaries and reclassifying mask regions, and generate an updated semantic segmentation mask image.

[0122] S44. Simultaneously modify the initial text to generate prompt information. The modification includes reorganizing the text structure according to the new area layout or label order, and replacing and adding keyword description content.

[0123] S45. Combine the updated semantic segmentation mask with the updated text generation prompt information to generate control condition data for structural control information and semantic control information.

[0124] S46. Output the updated control condition data to guide the next round of conditional control diffusion generation process, thereby achieving interactive image optimization.

[0125] This invention introduces a visual human-computer interaction mechanism, enabling bidirectional interaction and real-time optimization control between the user and the generated image, significantly improving the flexibility and customization level of the landscape design process. Users can directly view the initially generated landscape design image on the interface and perform interactive operations such as clicking, outlining, dragging, and renaming specific areas to achieve fine-grained editing of semantic masks and proactive adjustment of semantic intent. The system modifies the original semantic segmentation mask image in real time based on user actions, while simultaneously adjusting the corresponding text generation prompts to ensure consistency between image structure and semantic guidance. By combining the updated mask image and text generation prompts, a control condition data input diffusion model is formed, thereby completing interactive optimization generation. This mechanism constructs a closed-loop process of "perception-feedback-regeneration," greatly improving the response speed, personalization, and design quality of image generation. It solves the problems of inaccurate expression of user intent and high post-adjustment costs in existing generation models, demonstrating excellent interactive experience and practical engineering value.

[0126] In this embodiment, S5 specifically includes:

[0127] S51. Using the generated control condition data as input, the conditional control diffusion network is invoked to execute the diffusion generation process, and the optimized landscape design map G is output. ' ;

[0128] S52, Regarding the landscape design drawing G ′ Semantic region parsing is performed to extract the geometric boundaries and semantic label information of each semantic region, and a graph structure is constructed. Where V represents the set of regional nodes, E represents the set of regional adjacency relationships, and the landscape design map G... ′ Semantic region parsing refers to identifying and segmenting each functional region in the image based on the semantic segmentation mask map in the image generation result, extracting the geometric boundary, location center and corresponding semantic label of each region, and abstracting these regions as nodes in a graph structure; by analyzing the spatial contact relationship and adjacent boundaries between regions, establishing connection edges between nodes, and forming a graph structure containing region attributes and spatial topological relationships.

[0129] S53, Graph-based structure Construct a graph neural network encoder for each node v j Perform spatial structure encoding to generate region embedding vector h j This forms a set of region embeddings H, based on a graph structure. Constructing a graph neural network encoder refers to using the set of region nodes and their adjacency relationships formed after semantic region parsing as input to build a graph neural network model. The spatial location, geometric attributes, and semantic labels of each region node are used as initial features. The spatial relationships between nodes are aggregated and learned using the adjacency edge propagation mechanism, thereby encoding the contextual features of each region in the overall structure. The output is a set of region embedding vectors representing local semantics and global spatial relationships, which is used to guide 3D structure modeling and spatial parameter extraction.

[0130] S54. Input the semantic segmentation mask image and the region embedding set H into the structure-preserving mapping function to generate a three-dimensional structural model. The three-dimensional structural model includes the region contour shape, spatial orientation information and semantic topology. The structure-preserving mapping function reconstructs the two-dimensional semantic region into a three-dimensional landscape expression that maintains the original structural relationship by fusing semantic boundaries and structural embedding features, combined with spatial topological constraints and geometric consistency rules. Based on the three-dimensional structural model, a three-dimensional landscape representation with quantifiable spatial scale, real proportional structure and semantic attribute annotation is constructed. The three-dimensional landscape representation includes landscape contour, vertical height, spatial combination method and material texture mapping information.

[0131] S55. For each region node v in the three-dimensional structural model j Perform spatial layout extraction operation to generate spatial parameter vector s j :

[0132]

[0133] in, Let ω represent the set of adjacent regions of the j-th region. jk Δ is the weighting coefficient for adjacent regions. dir (v j ,v k) represents the direction difference, Δ dist (v j ,v k ) represents the difference in distance between the geometric centers, Δ norm (o j ,o k ) represents the angle difference between the normal vectors, h j Embedding for region map, The mean of the region embedding is given, and ρ1, ρ2, ρ3, and ρ4 are weighting coefficients. j Let o represent the unit normal vector of the j-th semantic region in three-dimensional space. k This represents the unit normal vector of the k-th semantic region in three-dimensional space;

[0134] Spatial parameter vector s j The formula serves as a crucial bridge connecting image generation results with spatial understanding capabilities. It comprehensively calculates the relative position, arrangement, and orientation consistency of each region within the layout from three aspects: the direction difference between adjacent regions, the distance difference between their geometric centers, and the angle between surface normal vectors. Simultaneously, it incorporates the difference between the embedded vector generated by the graph neural network and its global mean to measure the degree of deviation of that region from the overall structure. By weightedly combining these indicators, the system can generate rich and comprehensive spatial parameter vectors based on structural perception and spatial relationships, supporting subsequent ecological analysis, optimized layout, or functional layout adjustments. Compared to traditional spatial feature extraction methods based solely on geometric distance or simple statistical values, this formula integrates multi-dimensional elements such as topological structure, deep semantics, and spatial geometry, exhibiting stronger interpretability, robustness, and adaptability. It is one of the key mechanisms for achieving structure consistency-driven intelligent landscape modeling and evaluation.

[0135] S56, All spatial parameter vectors of all regions {s j The data is associated with the corresponding semantic labels to output the labeled landscape spatial dataset.

[0136] This invention constructs a complete modeling path from image generation to 3D structure reconstruction, achieving spatial structure restoration and quantitative expression of landscape design results. This effectively overcomes the problems of incomplete structural expression and unextractable spatial parameters in traditional image generation models. Through semantic region parsing and graph structure construction, the system can obtain the boundaries and adjacency relationships of each functional region and introduces a graph neural network encoder to model spatial topological relationships, generating region embedding features and improving the contextual structure perception capability of 3D reconstruction. Further integrating semantic masks and embedding information, a 3D structural model with spatial topological consistency and semantic preservation capabilities is constructed, achieving a comprehensive expression of multi-dimensional features such as landscape region contours, height, orientation, and materials. Through spatial layout extraction, the system can extract structural parameters such as directionality, positional relationships, and orientation consistency of each semantic region, forming a quantified spatial data set and ultimately constructing a labeled landscape spatial database. This method significantly improves the spatial realism, structural controllability, and subsequent evaluation operability of the design results, providing strong data support and a model foundation for ecological performance analysis and automatic layout optimization.

[0137] In this embodiment, S6 specifically includes:

[0138] Based on the extracted landscape spatial data, the location, scale, and combination relationship of each semantic region in three-dimensional space are evaluated from an ecological perspective. The distribution of sunshine duration is obtained by judging the impact of building or forest belt shading on sunlight access; ventilation paths are obtained by simulating air circulation based on regional layout and prevailing wind direction; the rationality of the layout is judged by evaluating its relative layout with green space and the coverage area of ​​water bodies; the green space ratio is obtained by calculating the vegetation coverage ratio; and spatial connectivity is judged by judging whether the path system efficiently connects various functional areas. Finally, a set of structured ecological assessment indicators is output to help optimize the sustainability and environmental adaptability of the design scheme.

[0139] In this embodiment, S7 specifically includes:

[0140] In the process of comparing and analyzing ecological performance data with user intent data, the system first performs semantic matching and quantitative comparison between the generated ecological assessment indicators and the preferences, functional needs, or key areas of concern expressed by the user in the initial input. For example, when a user wants "good lighting for the trails" or "a well-ventilated and open plaza," the system will determine whether the sunshine duration or ventilation path of the corresponding area in the current design drawing meets the standards. If not, feedback suggestions are calculated based on the deviation value. The system generates feedback information based on the degree of difference and weight priority, such as "the current green coverage rate of the children's area is too low, it is recommended to expand the lawn area" or "the density of the forest belt affects the main ventilation path, it is recommended to adjust the location," and returns these suggestions to the visual interactive interface in real time, guiding the user to modify the mask area or prompt content in step S4, thus achieving closed-loop optimization between ecological indicators and user intent.

[0141] Example 1:

[0142] To verify the feasibility of this invention in practice, it was applied to a landscape renovation and upgrade project of an urban ecological park. The design unit attempted to introduce the landscape design system based on artificial intelligence and human-computer interaction proposed in this invention to solve key problems such as low communication efficiency, mismatch between graphic output and intent, and lack of structural control and ecological feedback in previous projects. The site covers an area of ​​approximately 2.5 hectares. The original design drawings were 2D CAD files. The designers had to repeatedly communicate and optimize based on the client's vague descriptions such as "increasing interactivity," "increasing the proportion of green space," and "desiring a modern art plaza."

[0143] In this invention's system, designers can upload site images and express their intentions using voice, hand-drawn sketches, and keyword descriptions. The system automatically parses multimodal information to generate keyword sets and text prompts, and uses a multi-object segmentation model to structurally segment the site images, extracting semantic regions such as lawns, water bodies, plazas, and paths to construct structural control data. Subsequently, the system executes conditional control diffusion generation, with the initial generation of preliminary design drawings taking only about 18 seconds. Users can edit area boundaries and functional descriptions in real time through the interface, such as adjusting the location of water bodies or adding art installations. The system regenerates the image after each interaction, with an average of 5 optimization rounds.

[0144] The final image is automatically parsed into a 3D structure. The system extracts region embeddings based on graph structure and graph neural network encoding, constructs a 3D landscape representation model, extracts spatial layout parameters for each region, and completes ecological performance analysis, including sunlight accessibility, wind path, green space ratio, and spatial connectivity. A comparison with traditional design processes shows that: structural control accuracy improved from 0.55 to 0.86, semantic consistency score improved from approximately 0.60 to 0.91, the overall design cycle was reduced from 7-10 days to within 48 hours, and user satisfaction increased from an average of 6.3 to 9.7.

[0145] Table 1 Comparison of Traditional Design Process and Invention System

[0146]

[0147] As shown in Table 1, this invention significantly shortens the design cycle and overall design timeline in terms of design efficiency. In traditional processes, preliminary design generation typically takes 3 to 5 days, relying on manual drawing and repeated communication by designers. However, this invention's system, through semantic control and diffusion generation, can output preliminary design drawings within 15 to 20 seconds, and compresses the overall design cycle from 7 to 10 days to less than 48 hours, greatly improving work efficiency, especially suitable for scenarios with tight project deadlines.

[0148] Regarding user intent expression, this system supports multimodal input including voice, text, images, and sketches, greatly enhancing the flexibility and accuracy of user expression and avoiding the limitations of traditional methods that rely on "text + subjective understanding" and are prone to bias. Simultaneously, this system efficiently transforms user intent into structured image generation control conditions through keyword parsing and image semantic mapping.

[0149] In terms of design accuracy, the structural control accuracy of the design diagram in the traditional process is about 0.55, and the semantic consistency score is also low, at only 0.60. However, in this system, relying on algorithms such as object segmentation, weight adjustment mechanism and structure preservation reconstruction, these two indicators are improved to 0.86 and 0.91 respectively, which significantly improves the matching degree between the image and the user's needs.

[0150] Furthermore, traditional design tools almost never have real-time interactive feedback capabilities. This invention supports at least 5 rounds of interactive optimization, allowing users to adjust the local structure of an image and modify semantic prompts at any time. The system responds immediately and regenerates the design drawing, achieving a "what you see is what you get" design experience.

[0151] In terms of 3D modeling and spatial parameter representation, traditional processes require manual operation using specialized modeling software, taking 3 to 7 days. Furthermore, spatial parameter extraction relies on external GIS systems or manual annotation, resulting in low efficiency and poor consistency. This system automatically completes structural consistency modeling and outputs quantifiable spatial parameters. It also supports a built-in ecological performance analysis module, automatically generating evaluation indicators such as sunlight exposure, ventilation, and green space ratio, without requiring outsourcing or additional software support.

[0152] Overall, this invention outperforms traditional processes in terms of generation efficiency, structural control capabilities, semantic guidance accuracy, user interaction experience, and ecological decision support. Ultimately, user satisfaction increased from 6.3 to 9.7, demonstrating strong practicality and promotional value.

[0153] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A landscape design system based on artificial intelligence human-computer interaction, characterized in that, include: The multimodal parsing module is used to receive voice, text, image and hand-drawn sketch input, extract keywords and semantic descriptions, and generate user intent data; The structure segmentation module is used to process site images, generate semantic segmentation mask maps, and construct structure control data by combining user intent data. The text prompt module is used to convert user intent data into text to generate prompt information; The diffusion generation module is used to receive text generation prompts and structural control data to generate landscape design drawings; The interactive feedback module is used to display landscape design drawings, collect user operations and update semantic segmentation mask images and text generation prompts, and generate control condition data. The 3D modeling module is used to receive landscape design drawings and control condition data, perform 3D reconstruction, and generate a 3D landscape representation. The spatial feedback module is used to extract spatial parameters based on the three-dimensional landscape representation, output ecological analysis results, and feed them back to the interactive interface. The landscape design method based on artificial intelligence human-computer interaction, applied to the aforementioned landscape design system based on artificial intelligence human-computer interaction, includes the following steps: S1. Collect multimodal information from user input to construct user intent data; S2. Construct an object segmentation model, collect site images uploaded by users, and process the site images based on user intent data and site images through the object segmentation model to generate semantic segmentation mask images and form structural control data. S3. Construct text generation prompts based on user intent data, input the text generation prompts and structural control data into the conditional control diffusion network, execute the diffusion generation process, and output the preliminary landscape design drawing. S4. Display the preliminary landscape design drawing in the visual human-computer interaction interface, receive user interaction operations, update the semantic segmentation mask image and text generation prompt information, and generate control condition data. S5. Input the control condition data into the condition control diffusion network, execute the re-diffusion generation process, output the optimized landscape design drawing, analyze the landscape design drawing, generate a three-dimensional landscape representation, extract the spatial parameters of structural layout, functional distribution and light distribution, and form landscape spatial data. S6. Based on landscape spatial data, conduct ecological performance analysis and output ecological assessment indicators including sunshine duration distribution, ventilation path, water body layout, green space ratio and spatial connectivity to form ecological performance data. S7. Compare and analyze the ecological performance data with the user intent data, generate feedback information and provide it to the user, and the user performs the interactive operation in step S4 based on the feedback information. S2 specifically includes: S21. Collect the site image uploaded by the user, and convert the site image into a size of [size missing]. Three-channel image matrix ,in, Image height, Image width, It is the set of real numbers; S22. Extract the target keyword set based on user intent data. Each keyword This indicates the type of landscape area that the user is interested in. n is the number of semantic targets input by the user; S23. Construct multimodal input information for prompting, the multimodal input information including a set of spatial cue points. Each of them Image coordinates and keyword semantic set Each of them Corresponding to keywords semantic description, , The number of prompts entered by the user; S24. Construct an object segmentation model, which includes an image encoder, a cue encoder, and a mask decoder, and converts the three-channel image matrix... Input image encoder extracts image feature tensor The set of cue points in the cue information space and the set of keyword semantics are input into the cue encoder to generate cue vectors. ; S25. Convert the image feature tensor With hint vector Input a mask decoder, output a set of candidate masks ; S26. Calculate each candidate mask. With keywords The semantic consistency score between them is given by the following scoring function: ; S27. Based on the calculation result of the scoring function, select the mask with the highest score. As an effective mask, and perform boundary refinement processing; S28. Assign a unique semantic label to each semantic region in the effective mask. This forms a semantic mapping set; S29. Encode the effective mask and semantic mapping set into structural control data. , serving as the input control condition for the image generation process; S4 specifically includes: S41. Display the generated preliminary landscape design drawing in the visual human-computer interaction interface. Users can view the various functional areas and layout content on the interface. S42. Receive user interaction input in the interface, the interaction input including clicking, selecting, dragging, outlining, dragging, semantic annotation, deleting or renaming operations on areas in the image; S43. Based on user interaction input, modify the original semantic segmentation mask image, including adding, merging, adjusting boundaries and reclassifying mask regions, and generate an updated semantic segmentation mask image. S44. Simultaneously modify the initial text to generate prompt information. The modification includes reorganizing the text structure according to the new area layout or label order, and replacing and adding keyword description content. S45. Combine the updated semantic segmentation mask with the updated text generation prompt information to generate control condition data for structural control information and semantic control information. S46. Output the updated control condition data to guide the next round of conditional control diffusion generation process, thereby achieving interactive image optimization.

2. The landscape design system based on artificial intelligence human-computer interaction according to claim 1, characterized in that, The multimodal information specifically includes voice input, image input, text input, and hand-drawn sketch input.

3. The landscape design system based on artificial intelligence human-computer interaction according to claim 1, characterized in that, S3 specifically includes: S31. Extract the target keyword set based on user intent data. With keyword semantic set ; S32. Construct a text prompt string by assembling a semantic description set according to a preset template. Used to express the overall design intent, the preset template refers to a rule-based structure that concatenates the contents of the target keyword set into a coherent text prompt according to spatial relationships and functional logic. The text prompt string is then encoded to generate an overall text prompt embedding vector. ; S33, Transfer structural control data The input condition controls the control branch path of the diffusion network, and spatial mask feature maps are extracted through multi-layer convolution and downsampling operations. ; S34. Extract semantic tags Corresponding keywords semantic embedding vector , and spatial mask feature map Perform cross-modal attention fusion to generate spatial-semantic fusion feature maps. ; S35. Based on keyword semantic matching degree, user intent salience, and historical interaction frequency, calculate the weighting coefficient for each masked region. ; S36, Adjusting the weighting coefficients Mapped to image space to form pixel-level weight map Then, pixel-wise weighting is performed with the spatial-semantic fusion feature map to obtain a weighted control feature map. ; S37. Perform layered processing on the structural control data, extracting control features from the main structural region and the detail region respectively. and And inject it into different levels of the conditional control diffusion network, the former into the shallow coding path and the latter into the deep decoding path; S38. Construct diffusion-generated input pairs The initial state of the diffusion process is initialized as a Gaussian noise image. , representing a Gaussian noise image It is a matrix with a mean of 0 and a covariance of identity. Sampled from a Gaussian distribution; S39, in each diffusion step During the denoising process, a structure-semantic consistency feedback adjustment factor is introduced. The image state is updated at each diffusion time t by adjusting the structure-semantic consistency feedback factor. S310. Complete the diffusion process and output the final image. As a preliminary landscape design drawing The spatial structure is guided by the control feature map, and the visual semantics are guided by the text prompts, thus meeting the user's intent requirements.

4. The landscape design system based on artificial intelligence human-computer interaction according to claim 1, characterized in that, S5 specifically includes: S51. Using the generated control condition data as input, the conditional control diffusion network is invoked to execute the diffusion generation process, and the optimized landscape design drawing is output. ; S52, Landscape Design Drawings Semantic region parsing is performed to extract the geometric boundaries and semantic label information of each semantic region, and a graph structure is constructed. ,in, Represents a set of region nodes. Represents the set of adjacency relationships between regions; S53, Graph-based structure Construct a graph neural network encoder for each node. Perform spatial structure encoding to generate region embedding vectors Forming a set of region embeddings ; S54. Combine the semantic segmentation mask image with the region embedding set. The input structure-preserving mapping function generates a three-dimensional structural model. The three-dimensional structural model includes regional contour morphology, spatial orientation information, and semantic topology. The structure-preserving mapping function reconstructs a two-dimensional semantic region into a three-dimensional landscape expression that preserves the original structural relationships by fusing semantic boundaries and structural embedding features, combined with spatial topological constraints and geometric consistency rules. Based on the three-dimensional structural model, a three-dimensional landscape representation with quantifiable spatial scale, realistic proportional structure, and semantic attribute annotations is constructed. The three-dimensional landscape representation includes landscape contour, vertical height, spatial combination method, and material texture mapping information. S55. For each region node in the three-dimensional structural model Perform spatial layout extraction operations to generate spatial parameter vectors. ; S56, Spatial parameter vectors of all regions Associate the data with the corresponding semantic labels to output the labeled landscape spatial dataset.

Images