Garden design method and device based on image generation model and storage medium

Abstract

The invention relates to a garden design method and device based on an image generation model and a storage medium. The method comprises the following steps: in response to a scene image of a garden obtained from a user, carrying out region segmentation on the scene image to determine a range in which garden design needs to be carried out as a mask image; generating a personalized design recommendation list based on at least one of preference data and scene data of the user, and selecting a reference image of garden design from the design recommendation list by the user according to expectation; and analyzing the reference image to generate a character description for garden design, and generating a garden design effect picture conforming to expectation by taking the scene image, the mask image, the reference image and the character description as input conditions of an image generation model.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a garden design method, device and storage medium based on an image generation model. Background Art

[0002] Image inpainting technology aims to fill in missing parts in an image or remove unwanted objects to restore the integrity of the image. Traditional local repainting models generate the content of a specific area by using the information of neighboring pixels. These methods mainly include methods based on optimization algorithms, interpolation methods, and deep learning, such as convolutional neural networks. These methods still have limitations in terms of the transition effect between the edge of the area and the surrounding content.

[0003] In recent years, the Stable Diffusion model has become an emerging image generation technology that achieves higher quality image synthesis by gradually adding and removing noise. Its application in image restoration has shown significant advantages. The Stable Diffusion model can generate samples from the latent space to fill in the missing areas and generate richer and more natural image content. In addition, combined with contextual information, the diffusion model can better understand and reconstruct local patterns and details, thereby improving the restoration effect.

[0004] For different types of missing areas, researchers have gradually proposed a variety of deep learning-based strategies that combine the advantages of local redrawing models and diffusion methods to achieve more accurate and natural image restoration effects. The rapid development of these technologies has provided broad prospects for content creation, including home environment design. Summary of the invention

[0005] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided a garden design method based on an image generation model, comprising: in response to obtaining a scene image of a garden from a user, performing region segmentation on the scene image to determine a range where garden design is required as a mask image; generating a personalized design recommendation list based on at least one of the user's preference data and scene data, and the user selecting a reference image of the garden design from the design recommendation list according to expectations; and analyzing the reference image to generate a text description of the garden design, and using the scene image, mask image, reference image and text description as input conditions of the image generation model to generate a garden design rendering that meets expectations.

[0007] In some embodiments, region segmentation includes performing image processing using an image segmentation model to automatically identify a pixel range in the scene image as garden land.

[0008] In some embodiments, the image segmentation model is optimized to identify edges of garden land.

[0009] In some embodiments, the area segmentation further includes customizing the scope of the garden design based on user input.

[0010] In some embodiments, the scope of the garden design is visualized in the mask image so that the user can perform at least one of a confirmation operation and an adjustment operation.

[0011] In some embodiments, the user's preference data includes preference tags based on historical data analysis of the user, wherein the design recommendation list is sorted based on the preference tags.

[0012] In some embodiments, the scene data includes the user's geographic location information and environmental condition information, wherein the environmental condition information includes climate data and soil data corresponding to the geographic location information.

[0013] In some embodiments, the image generation model generates an expected garden design rendering including: adaptively processing a reference image to enhance image quality to obtain an adaptive reference image; locally redrawing a mask image based on the reference image and a text description to obtain a redrawn reference image; and generating a garden design rendering based on the adaptive reference image and the redrawn reference image.

[0014] In some embodiments, the image generation model generates an expected garden design rendering including: training a prompt context as a word element of the image generation model, thereby guiding the image generation model to perform context-aware image processing, wherein the concatenated text obtained by merging the prompt context with the text description is used as the text input of the image generation model.

[0015] In some embodiments, training the hint context includes: performing a local redrawing process using a randomly masked scene image to obtain a reconstructed unmasked image, wherein the hint context is trained so that the edge of the region of the unmasked image corresponding to the random mask is coordinated with the scene image.

[0016] In some embodiments, in response to a user wanting to design a specific area of ​​a garden, a prompt context is used to guide the image generation model to understand and generate a garden design rendering for the specific area.

[0017] In some embodiments, the image generation model generates an expected garden design rendering including: introducing an image prompt adapter into the image generation model to convert a reference image into an additional prompt input, wherein the additional prompt input and the text description are used together as word elements of the image generation model.

[0018] According to a second aspect of an embodiment of the present disclosure, there is provided a garden design device based on an image generation model, comprising: a processor; and a memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the aforementioned garden design method based on the image generation model.

[0019] According to a third aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer-executable instructions is provided. When the computer-executable instructions are executed by a processor, the processor implements the aforementioned garden design method based on an image generation model.

[0020] According to a fourth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer executable instructions, which, when executed by a processor, enable the processor to implement the aforementioned garden design method based on an image generation model.

[0021] The advantage of the embodiments of the present disclosure is that the optimized region segmentation technology is used to determine the part of the scene image uploaded by the user that needs garden design, and the user's preference notes are intelligently screened and combined with the machine learning algorithm to automatically recommend suitable design solutions to the user.

[0022] Another advantage of the embodiments disclosed herein is that the model's learnable word Pctxt is trained as an additional task prompt for text splicing with text descriptions, so as to guide the model to focus on contextual information to generate or repair images for user-specified areas, thereby making the transition between the area to be designed and other areas more natural, and the overall picture has a more reasonable filling effect.

[0023] It should be appreciated that the above advantages do not need to be all realized in one or some specific embodiments, but may be partially dispersed in different embodiments according to the present disclosure. The embodiments according to the present disclosure may have one or some of the above advantages, or may have other advantages alternatively or additionally. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The foregoing and other features and advantages of the present disclosure will become apparent from the following description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are incorporated herein and form a part of the specification, and are further used to explain the principles of the present disclosure and enable those skilled in the art to make and use the present disclosure. Among them:

[0025] Figure 1 A flow chart of a garden design method based on an image generation model according to an embodiment of the present disclosure is shown;

[0026] Figure 2 An exemplary block diagram of generating a garden design rendering by an image generation model according to some embodiments of the present disclosure is shown;

[0027] Figure 3 An example diagram showing input conditions of an image generation model when generating a garden design rendering according to some embodiments of the present disclosure;

[0028] Figure 4 An example diagram showing a partial redrawing of a mask image of a garden range by an image generation model according to some embodiments of the present disclosure;

[0029] Figure 5 An exemplary block diagram showing the use of prompt context to guide model output in an image generation model according to some embodiments of the present disclosure;

[0030] Figure 6 A schematic block diagram of a garden design device based on a diffusion model according to an embodiment of the present disclosure is shown;

[0031] Figure 7 An exemplary configuration of a computing device that can implement an embodiment according to the present disclosure is shown.

[0032] Note that in the embodiments described below, sometimes the same reference numerals are used in common between different drawings to represent the same parts or parts with the same functions, and their repeated descriptions are omitted. In some cases, similar numbers and letters are used to represent similar items, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0033] For ease of understanding, the positions, sizes, and ranges of various structures shown in the drawings and the like may not represent actual positions, sizes, and ranges, etc. Therefore, the present disclosure is not limited to the positions, sizes, and ranges disclosed in the drawings and the like. DETAILED DESCRIPTION

[0034] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0035] The following description of at least one exemplary embodiment is in fact merely illustrative and is in no way intended to limit the present disclosure and its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in the present disclosure. However, those skilled in the art will appreciate that they merely illustrate exemplary ways of the present disclosure that can be implemented, rather than exhaustive ways. In addition, the drawings need not be drawn to scale, and some features may be enlarged to illustrate the details of specific components.

[0036] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0037] The present disclosure provides a method for generating a garden style design image, which is based on an image filling (inpaint) model, analyzes historical data related to user preferences and uses a token to enhance context awareness to guide the model to generate images, thereby improving the accuracy of the image generation model's recommended design, satisfying the user's preferences, and generating a more reasonable filling effect in the overall picture. The garden design drawing generation method according to the present disclosure will be described in detail below in conjunction with the accompanying drawings. It should be understood that the actual method may also include other additional steps, but in order to avoid blurring the main points of the present disclosure, these other additional steps are not discussed herein and are not shown in the accompanying drawings.

[0038] Figure 1 FIG. 1 is a flow chart of a garden design method 1000 (hereinafter referred to as method 1000 ) based on an image generation model according to an embodiment of the present disclosure. Figure 1 As shown, the method 1000 mainly includes:

[0039] In step S101, in response to obtaining a scene image of a garden from a user, the scene image is segmented to determine a range where garden design is required as a mask image;

[0040] In step S102, a personalized design recommendation list is generated based on at least one of the user's preference data and scene data, and the user selects a reference image of the garden design from the design recommendation list according to expectations;

[0041] In step S103, the reference image is analyzed to generate a text description of the garden design, and the scene image, mask image, reference image and text description are used as input conditions of the image generation model to generate an expected garden design effect image.

[0042] Specifically, step S101 is mainly used to determine the scope of garden design to be performed from the scene image input by the user. Generally, the user uploads the image of the overall scene where the garden to be designed is located, wherein the overall scene includes physical elements such as the courtyard where the garden is located, the roads and buildings around the garden, and the environmental elements such as time, season and weather that may be contained in the image during the imaging process. In response to obtaining the scene image uploaded by the user, the region segmentation technology is used to determine the scope of garden design from the scene image. In a non-limiting embodiment, the region segmentation technology uses an image segmentation model (such as a Segmentation model) to process the scene image to automatically identify the scope of land that can be transformed into a garden. The model achieves efficient and accurate region segmentation by pixel-level classification of the input image. In addition, the image segmentation model can also be optimized for different application scenarios. For example, for the pixel range that has been identified as garden land, the pixels at the edge part may have defects or errors, and the edge part is optimized using an improved image segmentation model to improve the accuracy of edge recognition.

[0043] Additionally or alternatively, the area segmentation in step S101 can also be customized by the user's input to set the scope of the garden design. In a non-limiting embodiment, the user can also customize the area that needs to be transformed by manually painting in the scene image. In particular, the area segmentation method based on user input can also be used as a supplement to the image segmentation algorithm, such as when optimizing the edge portion of the garden land, if it is found that the identification of the edge portion has errors or mistakes, the computer results can be reviewed and adjusted based on the user input. In some embodiments, the scope of the garden design is visualized in the mask image in the scene graph image, such as representing the mask area in a way that is different from the transparency, brightness or color scheme of the original scene image, and the user can directly interact with the mask area, such as performing a confirmation operation to confirm that the edge division of the garden is correct and / or an adjustment operation to reset the correct edge.

[0044] In some embodiments, step S102 mainly involves determining the garden design reference image. On the one hand, considering the diversity of garden design styles and the personalized preferences of users, intelligent screening can be performed through the user's preference tags to automatically recommend garden design solutions suitable for users. Specifically, the user's preference data is collected, including but not limited to the user's past design solution selections, and / or the historical data of the user using similar appearance design applications, so as to summarize the user's preference tags for different styles based on the analysis of these historical data, and the preference tags can be set to correspond to the user's aesthetic level, in other words, the user's preference level can be intuitively reflected. In particular, when recommending a suitable garden design solution to a user, a personalized design recommendation list can be generated based on the analysis of the user's historical data by a machine learning algorithm, and the design recommendation list can be sorted based on the user's preference tags to ensure that each user can find a garden design style that meets their aesthetics.

[0045] Additionally or alternatively, in step S102, a personalized design recommendation list may be generated based on the scene data. Generally, the scene data includes the user's geographic location information and environmental condition information. The geographic location information is mainly used to evaluate the architectural / landscape style of the geographic area where the garden to be designed is located, and the environmental condition information at least includes climate data and soil data corresponding to the geographic location information, which are used to determine the plant species that meet the conditions in the design, so as to ensure that the user is provided with a garden design plan that is adapted to the scene.

[0046] In some embodiments, a text description of the reference image is obtained in step S103. In order to make the design scheme more instructive and operable, the selected reference image is analyzed using a contrastive language-image pretraining (CLIP) model, which can accurately capture the image content and generate a natural language description by training the connection between the image and the text, thereby providing a more detailed design basis for the garden design. Further, based on the obtained text description, it is used together with the scene image, the mask image, and the reference image selected by the user from the design recommendation list as input conditions of the image generation model, thereby generating a garden design rendering that meets the user's expectations.

[0047] Please refer to the following Figure 2, which shows an exemplary block diagram of generating a garden design rendering according to an image generation model of some embodiments of the present disclosure. First, the user inputs a scene image 201, and a mask image 202 is generated after determining the scope of the garden land to be designed based on the regional segmentation of the scene image 201; the user selects a reference image 204 from a design recommendation list generated based on user preferences and / or scene data, and analyzes the reference image 204 to obtain its corresponding text description 203; then, the input conditions 200 including the scene image 201, the mask image 202, the text description 203 and the reference image 204 are input into the garden design model 220. Specifically, the input conditions 200 of the garden design model 220 can refer to Figure 3 , which shows a garden effect image 360 ​​obtained by sequentially processing two groups of scene images 300 input by users.

[0048] Exemplarily, in the first group of images with sequence number 1, the scene image 300 is an empty courtyard in front of the floor-to-ceiling windows, the reference image 320 is an example of a garden containing various plants, and specifically, a text description 340 in natural language form is obtained from the reference image 320: "a garden with various plants and flowers, swirling gardens, the empress' swirlinggardens, gardens with flower beds, beautiful english garden, beautifulborder, whitespace border, beautiful garden, blue and purple plants, largepatches of plain colours, twisted gardens, garden landscape, blue border,lush flowery outdoors, garden background, herbs and flowers". As mentioned above, the reference image 320 is then segmented to obtain a mask image (not shown), which is used together with the scene image 300, the reference image 320, and the text description 340 as input conditions of the image generation model to ultimately generate a garden design rendering 360 in which an "empty courtyard is designed to be filled with various plants."

[0049] Return to reference Figure 2 In addition, the input conditions 200 may be processed in a refined manner. In a non-limiting embodiment, the garden design model 220 generates an expected garden design rendering including adaptively processing the reference image 204 to enhance the image quality, thereby obtaining an adaptive reference image. Specifically, an image processing adaptability (IPA) model may be used to process the image signal input by the garden design model 220 to obtain an adaptive reference image with enhanced image quality and details.

[0050] In another non-limiting embodiment, the local redrawing model 221 can be used to perform local redrawing processing on the mask image 202 based on the reference image 204 and the text description 203 to obtain a local redrawn image. In particular, the local redrawing processing can also use the adaptive reference image 204 after adaptive processing as input. Since the mask image 202 is a part of the scene image 201, the local redrawing processing can be understood as performing a fine processing on the local area of ​​the scene image 201 corresponding to the mask image 202, and then combining it with the unmasked part of the scene image 201 after processing, so as to form a complete image corresponding to the scene image 201.

[0051] Specifically, please refer to Figure 4 , which shows an example diagram of a local redrawing of a mask image of a garden range by an image generation model according to some embodiments of the present disclosure. Figure 4 4 shows three groups of local redrawing processes based on the reference image 400, wherein the circular frame is the garden land area originally belonging to the scene image 201, that is, the mask area corresponding to the mask image 202. In a non-limiting embodiment, the local redrawing model 221 may include an open source Powerpaint model, which can achieve a better and more natural redrawing effect in the local redrawing area. Figure 4 Two types of local redrawing are shown in the figure, in which the image obtained by the ordinary local redrawing 420 also looks consistent with the expectation of "an image of a courtyard with a garden (at the corresponding position of the mask area)", while the result of the Powerpaint redrawing 440 looks more consistent with "an image of a courtyard with a garden, and the garden is more similar to the reference image 400" as a whole, that is, a better and more natural redrawing effect can be achieved.

[0052] Continue to return to reference Figure 2, a machine learning model including a diffusion model (such as Stable Diffusion 1.5 (SD1.5)) is combined with the above processing results to generate a complete garden design rendering 240. Based on the above processing method, the rendering 240 can fully consider the ecological environment for plant growth and the rationality of spatial layout while ensuring the aesthetics of the design. Finally, the effect of migrating from the reference image 204 for garden design to the scene image 201 input by the user, especially to the mask image 202 of the scope to be designed, is achieved. It should be understood that, unlike the above Figure 4 Similar to the description of , the Powerpaint model can be used to make the transition between the redrawn area and the stitched edge in the design rendering 240 natural.

[0053] In summary, the garden design method disclosed herein makes full use of image processing technology and machine learning algorithms to provide users with personalized and high-quality garden design solutions in an automated and intelligent manner.

[0054] Specifically, please refer to Figure 5 , which shows an exemplary block diagram of using a prompt context Pctxt to guide model output in an image generation model according to some embodiments of the present disclosure. Pctxt is trained as the word-meta token 501 of the aforementioned image generation model to guide the model to perform context-aware image processing, wherein Pctxt 5012 is an additional task prompt specifically used to guide context awareness compared to the text prompt 5011 obtained from the reference image. Image generation in garden design belongs to the type used for image restoration or image filling, and its main purpose is to fill the specified range (i.e., the mask area) in the scene image in accordance with the input conditions to reconstruct the completed image. However, the aforementioned traditional image filling models such as sd1.5 have certain limitations on the effects of reasonable local filling and object removal, including the transition of the filled part into the background outside the mask area is not natural enough.

[0055] In some embodiments, the additional task prompt Pctxt 5012 is specifically used to guide the model to perform context-aware image processing. The learnable Pctxt 5012 enables the model to focus on filling the user-specified area according to the contextual information of the image, and generate a repair result that is integrated with the surrounding image content. Among them, the training process of Pctxt 5012 may include the local redrawing model using a randomly masked area of ​​the scene image for local filling processing, and optimizing the model to obtain a reconstructed unmasked image. This training strategy encourages the model to pay attention to the context of the image and use this information to fill in the missing areas, thereby producing a result in which the regional edges of the unmasked image are coordinated with the surrounding content of the scene image. When optimizing the model, it is necessary to find the best Pctxt so that the model accurately predicts the noise under given input conditions, so that the generated image results are more in line with expectations:

[0056]

[0057] Where t represents the denoising time step, p is randomly initialized as a set of tokens and used as a text encoder Input; Represents the concatenation of the noise latent 502, the masked image 503, and the mask 504. This formulation enables the user to seamlessly fill in the masked region without explicitly specifying the desired content, making its content coherent with the surrounding content of the scene image.

[0058] In some embodiments, during the training process, Pctxt 5012 and other text prompts 5011 obtained by the text encoder are merged (concat) to obtain the spliced ​​text, which is input into the base model of the sd1.5 model as a text condition. It should be understood that this prompt is optimized to minimize the difference between the model output and the original image, so that the repaired area can be seamlessly integrated into the surrounding image context. Further, the noise latent space 502, the mask image 503 and the mask 504 are input into the denoising u-net model 520, and the denoised image is output into the variational encoder VAE540, and finally the output image 560 of the garden design rendering with higher image quality is obtained.

[0059] Additionally or alternatively, in actual applications, in response to the user's desire to design a specific location in the garden and to have a good integration of the design content with the surrounding environment, that is, to perform image repair or filling processing on a specific area in the scene image, the filled content should match the surrounding context in the scene image, that is, Pctxt can be used to guide the model to generate appropriate content. In a non-limiting embodiment, if an area in the scene image is masked, Pctxt can help the model understand and generate content that matches the surrounding environment of the area, including but not limited to textures, colors, and styles in the environmental context.

[0060] It should be understood that the division of steps in the aforementioned flowcharts is only for illustration and not for limitation of the embodiments of the present application. These steps can be completed by each illustrated subject individually, two or more of them are completed together, or by an external system not shown in cooperation with the illustrated subject, and are not limited to the illustrated distribution method, and can be completed in sequence according to the illustrated order or in parallel in accordance with the data flow direction and system operation rules, and can be completed step by step or in combination.

[0061] The present disclosure also provides a garden design device 6000 based on an image generation model, which may include a processor 6010 and a memory 6020 storing computer executable instructions, which, when executed by the processor 6010, enables the processor 6010 to perform the garden design method based on the image generation model according to any of the aforementioned embodiments of the present disclosure. The processor 6010 may be a central processing unit (CPU) of an image generation device, which may be any type of general-purpose processor, or may be a processor specially designed for the garden design method based on the image generation model, such as an application-specific integrated circuit (“ASIC”). The memory 6020 may include various computer-readable media accessible by the processor 6010.

[0062] The present disclosure also provides a non-transitory computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, enables the processor to implement the garden design method based on the image generation model according to any of the aforementioned embodiments of the present disclosure.

[0063] The present disclosure also provides a computer program product, comprising computer executable instructions, which, when executed by a processor, enable the processor to implement the garden design method based on the image generation model according to any of the aforementioned embodiments of the present disclosure.

[0064] Figure 7An exemplary configuration of a computing device that can implement an embodiment of the present disclosure is shown. The computing device includes one or more processors 701, an input / output interface 705 connected to the processor 701 via a bus 704, and memories 702 and 703 connected to the bus 704. In some embodiments, the memory 702 may be a read-only memory (ROM), and the memory 703 may be a random access memory (RAM).

[0065] Processor 701 may be any kind of processor, and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). Memories 702 and 703 may be any non-transitory storage device that can implement data storage, and may include, but is not limited to, disk drives, optical storage devices, solid-state memories, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, compressed disks or any other optical media, cache memories and / or any other storage chips or modules, and / or any other media from which a computer can read data, instructions and / or codes.

[0066] The bus 704 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0067] In some embodiments, the input / output interface 705 is connected to an input unit 706 configured by an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 707 that outputs a processing operation screen and an image of a processing result to a display device, a storage unit 708 including a hard disk drive and the like for storing programs and various data, and a communication unit 709 including a local area network (LAN) adapter and the like and performing communication processing via a network represented by the Internet. In addition, a drive 710 that reads and writes data from and on a removable storage medium 711 is connected.

[0068] Various aspects, embodiments, implementations or features of the foregoing embodiments may be used alone or in any combination. Various aspects of the foregoing embodiments may be implemented by software, hardware or a combination of hardware and software.

[0069] For example, the foregoing embodiments may be embodied as computer readable code on a computer readable medium. A computer readable medium is any data storage device that can store data that can thereafter be read by a computer system. Examples of computer readable media include read-only memory, random access memory, CD-ROM, DVD, magnetic tape, hard drive, solid state drive, and optical data storage device. The computer readable medium may also be distributed in a network coupled computer system so that the computer readable code is stored and executed in a distributed manner.

[0070] For example, the aforementioned embodiments may be in the form of hardware circuits. The hardware circuits may include any combination of combinational logic circuits, clock storage devices (such as floppy disks, flip-flops, latches, etc.), finite state machines, memories such as static random access memories or embedded dynamic random access memories, custom designed circuits, programmable logic arrays, etc.

[0071] In one embodiment, hardware circuits according to the present disclosure may be implemented by coding and designing one or more integrated circuits using a hardware description language (HDL) such as Verilog or VHDL, or in combination using discrete circuits.

[0072] The terms used herein are only for describing specific exemplary embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, the singular forms of "one" and "the" used herein are intended to include plural forms as well. It should also be understood that when the word "including" is used in this article, it indicates the existence of the indicated features, wholes, steps, operations, units and / or components, but does not exclude the existence or increase of one or more other features, wholes, steps, operations, units and / or components and / or their combinations. In addition, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or order. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more.

[0073] References to "embodiments" or similar expressions in this specification refer to specific features, structures, or characteristics described in conjunction with the embodiment as being included in at least one specific embodiment of the present disclosure. Therefore, the appearance of "in an embodiment of the present disclosure" and similar expressions in this specification does not necessarily refer to the same embodiment.

[0074] Those skilled in the art will appreciate that the present disclosure may be implemented in various forms, such as a complete hardware embodiment, a complete software embodiment (including firmware, resident software, microprogram code, etc.), or a combination of software and hardware, which will be referred to below as a "circuit," "module," "unit," or "system." In addition, the present disclosure may also be implemented in any tangible media form as a computer program product having a computer-usable program code stored thereon.

[0075] The relevant description of the present disclosure is described with reference to the flowcharts and / or block diagrams of the systems, devices, methods and computer program products according to the specific embodiments of the present disclosure. It can be understood that each block in each flowchart and / or block diagram, and any combination of blocks in the flowchart and / or block diagram, can be implemented using computer program instructions. These computer program instructions can be executed by a machine composed of a processor of a general-purpose computer or a special computer or other programmable data processing device, and the instructions are processed by a computer or other programmable data processing device to implement the functions or operations described in the flowchart and / or block diagram.

[0076] The accompanying drawings show the flowcharts and block diagrams of the architecture, functions and operations that can be implemented by the systems, devices, methods and computer program products according to various embodiments of the present disclosure. Therefore, each block in the flowchart or block diagram may represent a module, segment, or part of the program code, which includes one or more executable instructions to implement the specified logical function. It should also be noted that in some other embodiments, the functions described in the blocks may not be performed in the order shown in the figure. For example, two blocks connected to each other in the figure may in fact be executed simultaneously, or in some cases according to the functions involved, they may be executed in the opposite order of the icons. It should also be noted that each block diagram and / or flowchart block, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a system based on dedicated hardware, or by a combination of dedicated hardware and computer instructions to perform specific functions or operations.

[0077] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to market technologies, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A garden design method based on an image generation model, comprising: In response to obtaining a scene image of a garden from a user, performing region segmentation on the scene image to determine a range requiring garden design as a mask image; generating a personalized design recommendation list based on at least one of the user's preference data and scenario data, and the user selecting a reference image of a garden design from the design recommendation list according to expectations; as well as The reference image is analyzed to generate a text description of the garden design, and the scene image, the mask image, the reference image and the text description are used as input conditions of the image generation model to generate a garden design rendering that meets the expectations.

2. The garden design method according to claim 1, wherein: The region segmentation includes performing image processing using an image segmentation model to automatically identify a pixel range in the scene image that is garden land.

3. The garden design method according to claim 2, wherein: The image segmentation model optimizes the edge portion of the identified garden land.

4. The garden design method according to claim 1 or 2, wherein: The area segmentation further includes customizing the range of the garden design based on the user's input; Preferably, the scope of the garden design is visualized in the mask image so that the user can perform at least one of a confirmation operation and an adjustment operation.

5. The garden design method according to claim 1, wherein: The user's preference data includes a preference tag based on an analysis of the user's historical data, wherein the design recommendation list is sorted based on the preference tag; The scene data includes the user's geographical location information and environmental condition information, wherein the environmental condition information includes climate data and soil quality data corresponding to the geographical location information; The image generation model generates a garden design rendering that meets the expectations, including: Performing adaptive processing on the reference image to enhance image quality, to obtain an adaptive reference image; Performing a partial redrawing process on the mask image based on the mask image and the text description to obtain a partial redrawn image; and The garden design rendering is generated based on the adaptive reference image and the local redrawn image.

6. The garden design method according to claim 1, wherein the image generation model generates a garden design rendering that meets the expectation, comprising: Training the prompt context as a word element of the image generation model, thereby guiding the image generation model to perform context-aware image processing, wherein the concatenated text obtained by merging the prompt context with the text description is used as a text input of the image generation model; Preferably, training the prompt context comprises: Performing a local redrawing process on the scene image with a random mask to obtain a reconstructed unmasked image, wherein the prompt context is trained so that the edge of the region of the unmasked image corresponding to the random mask is coordinated with the scene image; and In response to the user wanting to design a specific area of ​​the garden, the prompt context is used to guide the image generation model to understand and generate a garden design rendering for the specific area.

7. The garden design method according to claim 1, wherein the image generation model generates a garden design rendering that meets the expectation, comprising: An image prompt adapter is introduced into the image generation model to convert the reference image into an additional prompt input, wherein the additional prompt input and the text description are used together as word elements of the image generation model.

8. A garden design device based on an image generation model, comprising: processor; as well as A memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the garden design method based on the image generation model according to any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor implements the garden design method based on an image generation model according to any one of claims 1 to 7.

10. A computer program product, comprising computer executable instructions, which, when executed by a processor, enable the processor to implement the garden design method based on an image generation model according to any one of claims 1 to 7.

Images