Vector drawing generation model training method and system based on diffusion model
Patent Information
- Application Number
- CN202511084678.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-04
- Publication Date
- 2025-11-21
AI Technical Summary
现有建筑平面设计中，尺寸标注依赖设计师经验，导致效率低下且质量不一致，增加审查和沟通成本。
采用基于扩散模型的矢量图纸生成模型训练方法，通过对原始图像进行预处理、正向扩散、构建条件输入和损失函数、反向传播优化，生成带标注的矢量图纸。
实现了高效、准确地生成满足审查和施工要求的建筑平面尺寸标注图像，提高设计质量和效率。
Smart Images
Figure CN120997868A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for training vector paper generation models based on a diffusion model. Background Technology
[0002] In architectural floor plan design, dimensioning is not only a supplementary explanation to graphics but also a core component for realizing the design. Standardized and clear dimensioning effectively connects design and construction, ensuring the efficient, safe, and cost-effective completion of building projects. Wall dimensioning clarifies the wall's thickness, length, axial positioning, and relationship to other components. Door and window dimensioning determines the location, size, and installation method of doors and windows. Spatial dimensioning defines the usable area and functional layout of rooms or areas. However, in traditional architectural design processes, architectural floor plan dimensioning relies primarily on the designer's professional knowledge and experience, using hand-drawn sketches and relevant design software. This traditional approach is inefficient, susceptible to the designer's subjective judgment, and prone to inconsistent dimensioning quality, increasing the difficulty of review by the housing and construction bureau and communication costs with construction companies.
[0003] Therefore, existing technologies still have shortcomings. Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide a method and system for training a vector paper generation model based on a diffusion model, addressing the aforementioned deficiencies of the prior art. The technical solution adopted by this invention is as follows:
[0005] In a first aspect, the present invention provides a method for training a vector paper generation model based on a diffusion model, wherein the method includes:
[0006] After preprocessing the original image, a target image is obtained. The target image and editing instructions are then input into a diffusion model. Forward diffusion is performed based on the diffusion model to obtain a noisy spatial image. The editing instructions are text descriptions used to add building dimension annotations.
[0007] A conditional input is constructed based on editing instructions, the original image, and a noisy spatial image, and the loss function of the diffusion model is determined based on the conditional input.
[0008] The model parameters of the diffusion model are optimized by backpropagation algorithm, and the weights in the diffusion model are updated by an optimizer. Based on iterative training and validation, a vector paper generation model is obtained.
[0009] In one implementation, obtaining the target image after preprocessing the original image includes:
[0010] The original image is normalized.
[0011] The original image after normalization is subjected to size unification processing to obtain the target image.
[0012] In one implementation, the step of inputting the target image and editing instructions into a diffusion model, and performing forward diffusion based on the diffusion model to obtain a noisy spatial image includes:
[0013] The target image and editing instructions are input into the diffusion model, and the variational autoencoder in the diffusion model converts the target image from pixel space into latent space.
[0014] For each target image, a random time step is sampled, and random noise is added to the latent space according to the noise intensity of each time step to obtain a noisy spatial image.
[0015] In one implementation, the conditional input based on editing instructions, the original image, and the noisy spatial image includes:
[0016] The editing instructions are converted into text embeddings based on the CLIP encoder in the diffusion model.
[0017] The image embedding corresponding to the original image is determined based on the variational autoencoder in the diffusion model;
[0018] The conditional input is obtained by performing feature concatenation on the text embedding, the image embedding, and the noisy spatial image.
[0019] In one implementation, determining the loss function of the diffusion model based on the conditional input includes:
[0020] The conditions are input into the UNet model to predict the noise distribution in the noisy spatial image, thus obtaining the predicted noise.
[0021] The mean squared error between the predicted noise and the actual noise is used as the loss function of the diffusion model, where the actual noise is random noise added to the latent space.
[0022] In one implementation, the method further includes:
[0023] The original image is back-diffused based on the vector paper generation model to output an annotated image, and the annotated image is evaluated based on a preset evaluation index.
[0024] In one implementation, the back-diffusion of the original image based on the vector paper generation model to output an annotated image includes:
[0025] The original image and the prompt words are input into the vector paper generation model, and the conditional embedding is determined based on the original image and the prompt words;
[0026] Starting with random noise, and combining the conditional embedding, the noise is predicted and gradually removed using the UNet model;
[0027] The inference process is optimized using the DDIM sampler, and after multiple rounds of iterative denoising, labeled images are gradually generated.
[0028] Secondly, embodiments of the present invention also provide a vector paper generation model training system based on a diffusion model, wherein the system is used to implement the steps of the vector paper generation model training method based on a diffusion model as described in any of the above solutions, and the system includes:
[0029] The image preprocessing module is used to preprocess the original image to obtain the target image, and input the target image and editing instructions into the diffusion model. Based on the diffusion model, forward diffusion is performed to obtain a noisy spatial image. The editing instructions are text descriptions used to add building dimension annotations.
[0030] A conditional input construction module is used to construct conditional inputs based on editing instructions, the original image, and a noisy spatial image, and to determine the loss function of the diffusion model based on the conditional inputs;
[0031] The backpropagation optimization module is used to optimize the model parameters of the diffusion model through the backpropagation algorithm, update the weights in the diffusion model using the optimizer, and obtain the vector paper generation model based on iterative training and validation.
[0032] Thirdly, embodiments of the present invention also provide a terminal, wherein the terminal includes a memory, a processor, and a vector paper generation model training program based on a diffusion model stored in the memory and executable on the processor. When the processor executes the vector paper generation model training program based on a diffusion model, it implements the steps of the vector paper generation model training method based on a diffusion model of any of the above-mentioned schemes.
[0033] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a vector paper generation model training program based on a diffusion model, the vector paper generation model training program based on a diffusion model implementing the steps of the vector paper generation model training method based on a diffusion model as described in any of the above schemes on the computer-readable storage medium.
[0034] Beneficial Effects: Compared with existing technologies, this invention provides a vector drawing generation model training method based on a diffusion model. First, the original image is preprocessed to obtain a target image. This target image, along with editing instructions, is then input into a diffusion model. Forward diffusion is performed based on the diffusion model to obtain a noisy spatial image. The editing instructions are text descriptions used to add architectural dimension annotations. Next, a conditional input is constructed based on the editing instructions, the original image, and the noisy spatial image. The loss function of the diffusion model is then determined based on this conditional input. Finally, the model parameters of the diffusion model are optimized using a backpropagation algorithm, and the weights in the diffusion model are updated using an optimizer. A vector drawing generation model is obtained through iterative training and validation. This invention enables the dimensioning of architectural floor plans by training a diffusion model, efficiently, accurately, and flexibly generating architectural floor plan dimensioned images that meet the requirements of reviewers and construction parties, thus improving design quality and efficiency. Attached Figure Description
[0035] Figure 1 A flowchart of a preferred embodiment of the vector paper generation model training method based on a diffusion model provided in this invention.
[0036] Figure 2 This is an overall flowchart illustrating a specific application scenario of the vector paper generation model training method based on the diffusion model provided in this embodiment of the invention.
[0037] Figure 3 This is a schematic diagram of the architecture of a vector paper generation model training system based on a diffusion model provided in an embodiment of the present invention.
[0038] Figure 4 A schematic diagram of a terminal provided in an embodiment of the present invention. Detailed Implementation
[0039] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0040] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0041] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0042] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, "first control information" and "second control information" are only used to distinguish different control information and do not limit their order.
[0043] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.
[0044] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0045] Early AI algorithms were mostly based on rules and logical reasoning, which had limitations in solving specific domain problems and struggled to handle complex and ever-changing real-world tasks. With technological innovation, data-driven generative artificial intelligence has emerged, with image diffusion models representing a key breakthrough in the image processing field, capable of generating high-quality images from noise. Research has been conducted on generating architectural renderings based on diffusion models, generating building staircases using AI rule-based algorithms, and automatically generating fire safety designs for high-rise residential areas based on deep learning. These studies demonstrate the significant research in applying computer vision and other AI technologies to the architectural field. However, a method for automatically generating architectural plan dimensioning images based on diffusion models is lacking. Since plan dimensioning is a necessary step in the design process for measuring the dimensions of buildings and objects, a method for generating architectural plan dimensioning images based on diffusion models is of great significance.
[0046] Based on this, the present invention provides a method for training a vector drawing generation model based on a diffusion model. The method in this embodiment trains a diffusion model to obtain a vector drawing generation model, which can automatically generate labeled images from original architectural floor plans. In specific applications, this embodiment first preprocesses the original image to obtain a target image, and then inputs the target image and editing instructions into the diffusion model. Forward diffusion is performed based on the diffusion model to obtain a noisy spatial image, where the editing instructions are text descriptions for adding architectural dimension annotations. Then, a conditional input is constructed based on the editing instructions, the original image, and the noisy spatial image, and the loss function of the diffusion model is determined based on the conditional input. Finally, the model parameters of the diffusion model are optimized using a backpropagation algorithm, and the weights in the diffusion model are updated using an optimizer. Based on iterative training and validation, the vector drawing generation model is obtained. This invention achieves dimension annotation of architectural floor plans by training a diffusion model, which can efficiently, accurately, and flexibly generate architectural floor plan dimension annotation images that meet the requirements of reviewers and construction parties, improving design quality and efficiency.
[0047] The vector paper generation model training method based on the diffusion model in this embodiment can be applied to terminals, including intelligent product terminals such as computers, smart TVs, and mobile phones. In this embodiment, as shown... Figure 1 As shown, the vector paper generation model training method based on the diffusion model includes the following steps:
[0048] Step S100: After preprocessing the original image, a target image is obtained. The target image and editing instructions are input into the diffusion model. Forward diffusion is performed based on the diffusion model to obtain a noisy spatial image. The editing instructions are text descriptions used to add building dimension annotations.
[0049] In this embodiment, the original image is a building floor plan without dimension annotations. After obtaining the original image, in order to unify the input format of the diffusion model, this embodiment needs to preprocess the original image, including normalization and dimension unification. Specifically, combined with... Figure 2As shown, this embodiment normalizes the original image, standardizing the pixel values to the [0,1] range to facilitate stable model computation. Next, the normalized image is resized, adjusting the resolution to 768×768 or higher to ensure the diffusion model's ability to capture image details, thus obtaining the target image. Furthermore, in other implementations, this embodiment can also perform data augmentation on the original image, such as by adding geometric transformations (rotation, scaling, cropping) and color perturbations (random adjustment of brightness, contrast, and saturation), to expand the diversity of the original image and improve model robustness. When there are many original images, this embodiment can also filter low-quality images to improve the purity of the training data for subsequent diffusion models.
[0050] In addition, in other implementations, this embodiment also supports using sketches or CAD line drawings as input to the diffusion model, which can be encoded by an additional feature encoder (such as a graph encoder for vector graphics), thereby expanding the adaptability of the diffusion model to data at different design stages.
[0051] Furthermore, after obtaining the target image, this embodiment performs forward diffusion based on a diffusion model, specifically including the following steps:
[0052] Step S101: Input the target image and editing instructions into the diffusion model, and use the variational autoencoder in the diffusion model to convert the target image from pixel space into latent space;
[0053] Step S102: Randomly sample the time step for each target image, and add random noise to the latent space according to the noise intensity of each time step to obtain a noisy spatial image.
[0054] In this embodiment, the target image and editing instructions are input into the diffusion model. In this embodiment, the editing instructions are text descriptions used to add building dimension annotations. The variational autoencoder (VAE) in the diffusion model transforms the target image from pixel space to latent space to reduce the computational cost in subsequent training processes. Next, this embodiment generates random noise following a normal distribution (mean = 0, variance = 1) as a noise source for the diffusion process. A random time step is sampled for each target image, and the random noise is added to the latent space according to the noise intensity at each time step, thereby obtaining a noisy spatial image. The noisy spatial image can simulate the noisy state of the diffusion process.
[0055] The forward diffusion process in this embodiment is represented as follows:
[0056]
[0057] Where q(x) t|x t-1 ) represents the image state x at the previous time step (t-1). t-1 Image state x at the current time (time step t) t The probability distribution is the conditional probability of forward diffusion. Represents a normal distribution, β t This represents the diffusion coefficient that varies with time step t, and is usually a pre-defined sequence that increases with t. I is the identity matrix.
[0058] Step S200: Construct conditional input based on editing instructions, the original image, and the noisy spatial image, and determine the loss function of the diffusion model based on the conditional input.
[0059] In practical applications, this embodiment includes the following steps when constructing conditional inputs:
[0060] Step S201: Convert the editing instructions into text embeddings based on the CLIP encoder in the diffusion model;
[0061] Step S202: Determine the image embedding corresponding to the original image based on the variational autoencoder in the diffusion model;
[0062] Step S203: Perform feature concatenation on the text embedding, the image embedding, and the noisy spatial image to obtain the conditional input.
[0063] In this embodiment, the CLIP (Contrastive Language-Image Pretraining) encoder in the diffusion model converts editing instructions into text embeddings to capture the textual semantics within the instructions. Next, the variational autoencoder in the diffusion model determines the image embedding corresponding to the original image. Then, the text embedding, the image embedding, and the noisy spatial image are feature-connected to obtain the conditional input, which serves as the input to the subsequent UNet model. Specifically, this embodiment feeds the conditional input into the UNet model to predict the noise distribution in the noisy spatial image, obtaining the predicted noise. Next, the mean squared error between the predicted noise and the true noise is calculated, and this error is used as the loss function of the diffusion model, where the true noise is random noise added to the latent space.
[0064] Step S300: Optimize the model parameters of the diffusion model using the backpropagation algorithm, update the weights in the diffusion model using the optimizer, and obtain the vector paper generation model based on iterative training and validation.
[0065] This embodiment calculates the gradient of the loss with respect to the model parameters using the backpropagation algorithm, updates the weights of the UNet model using the AdamW optimizer (a variant of the Adam optimizer), gradually reduces the difference between the predicted noise and the actual noise, and enables the diffusion model to learn to predict noise more accurately, i.e., the denoising capability during backpropagation.
[0066] In this embodiment, the reverse diffusion process is represented as follows:
[0067]
[0068] Where, p θ (x t-1 |x t ) represents the conditional probability of backdiffusion, and represents the noisy image x at a known time t. t The denoised image x at time t-1 predicted by the model t-1 The probability distribution. μ represents a normal distribution. θ It is the mean of the model predictions, Σ θ θ is the variance, and θ is the model parameter.
[0069] This embodiment calculates the gradient of the loss with respect to the model parameters using the backpropagation algorithm, updates the weights of the UNet model using the AdamW optimizer, and gradually reduces the difference between predicted noise and real noise, allowing the diffusion model to learn to predict noise more accurately, i.e., the denoising capability during backpropagation. Steps S100 to S300 can be executed cyclically, gradually reducing the difference between predicted noise and real noise through iterative training and validation until the model loss stabilizes, ultimately obtaining a trained diffusion model, which is also the vector graphics generation model. This vector graphics generation model can automatically generate images to be labeled.
[0070] Furthermore, if distributed training is used in this embodiment, loss logs for each process need to be collected to ensure the consistency and monitorability of the training process. In addition, during training, this embodiment can perform multiple training runs on different training datasets by adjusting resolution, learning rate, training epochs, batch size, and batch quantity, and then compare the results using a test set to obtain a score. Table 1 below shows the scoring table of the diffusion model trained with different dataset sizes, resolutions, learning rates, and training epochs.
[0071] Table 1
[0072] Number of datasets Training set resolution Training resolution Training learning rate Training rounds score 1180 512x512 256x256 0.000065 20 0 1180 1024x1024 256x256 0.000065 20 0 1180 1024x1024 512x512 0.000065 20 1.5 2080 1024x1024 512x512 0.000065 20 1.8 3100 1024x1024 768x768 0.000050 15 3.2 4000 1024x1024 768x768 0.000050 15 3.0 4500 1024x1024 1024x1024 0.000035 10 3.8 5000 1024x1024 1024x1024 0.000035 10 4
[0073] Furthermore, to address overfitting issues in vector paper generation models, adjustments to training parameters are necessary. For instance, when generation distortion occurs, the dataset can be processed with higher resolution. Since the training resolution is low, such as 256x256 or 512x512, the image precision is poor, and the images obtained through computer vision convolutional learning are not refined enough. Therefore, the resolution can be increased to 768x768 or 1024x1024 to improve image resolution and avoid generation distortion. Another example is if the generated planar dimension annotations appear as a grid, indicating that the annotation data is too redundant, the learning rate is too high, and the number of training epochs is too high. In this case, single-sided annotations can be applied to walls, doors, windows, and spaces, with only one annotation line appearing on the X and Y axes. The learning rate should be reduced to below 0.00005, and the number of training epochs reduced to below 20.
[0074] After training the vector drawing generation model, this embodiment can perform backdiffusion on the original images in the test set based on the vector drawing generation model to output labeled images, and evaluate the labeled images based on a preset evaluation metric. Specifically, this embodiment can input the original image and the prompt word into the vector drawing generation model, and determine the conditional embedding based on the original image and the prompt word. In this embodiment, the prompt word can be a text instruction for architectural plan dimension annotation. Similarly, this embodiment can convert the prompt word into CLIP text embedding, convert the original image into latent spatial embedding, and obtain the conditional embedding based on the CLIP text embedding and latent spatial embedding. Next, this embodiment starts with random noise, combines the conditional embedding, and uses the UNet model to predict and gradually remove the noise. Then, the DDIM (Denoising Diffusion Implicit Models) sampler is used to optimize the inference process, thereby accelerating convergence and improving image quality, and after multiple rounds of iterative denoising, labeled images are gradually generated. Finally, this embodiment evaluates the labeled images based on a preset evaluation metric.
[0075] To evaluate the feasibility of the generated labeled image, this embodiment sets the following evaluation metrics for scheme evaluation: Label line range metric, used to determine whether the labels exceed the drawing area, such as penetrating walls to the outside; spatial labeling metric, used to measure whether the spatial labels are concise and clear, and do not obstruct other elements within the space; wall labeling metric, used to evaluate whether the label line offset covers walls, avoiding a large number of unlabeled walls; and door and window labeling metric, used to measure whether the label lines cover numerous but small-area objects such as doors and windows. Furthermore, in practical applications, this embodiment can also use the Inception network to extract features from the real image and the generated labeled image, calculate the Fréchet distance (a metric for measuring the shape similarity between two curves), and use this Fréchet distance to measure the similarity between the generated labeled image and the label distribution in the real image. A higher similarity indicates that the generated labeled image is very close to the label distribution in the real image. This embodiment can also use the CLIP model to calculate the cosine similarity between the image and the text, measuring the matching degree between the generated labeled image and the prompt words. A higher matching degree indicates a better matching degree between the generated labeled image and the prompt words.
[0076] In other implementations, this embodiment can also introduce a manual evaluation method to assess the quality of the generated labeled images. During manual evaluation, scores can be given based on the following dimensions:
[0077] Visual quality rating: The sharpness, artifacts, and naturalness of the generated labeled images are scored by humans.
[0078] Condition matching score: Manual evaluation of whether the labeled image matches the prompt or input image (e.g., "Does it contain the specified object"), and scoring of the labeled image.
[0079] Diversity scoring: Manually judge the diversity (e.g., whether it is repeated) of labeled images generated under the same prompt word, and score the labeled images based on the judged diversity.
[0080] Failure case: Manually analyze artifacts (such as blurring and distortion) and parts that do not match the prompt words in the generated labeled image, and score the labeled image based on the identified artifacts or non-matching parts.
[0081] Boundary condition testing: Manually test the generation effect under extreme conditions (such as complex prompts, low-quality input images) and score the labeled images.
[0082] After scoring the labeled images using the above-mentioned dimensions, an average score is obtained. When the average score is higher than the preset value, the quality of the labeled images can be judged to be excellent.
[0083] In other implementations, this embodiment can introduce adversarial training during the training of the vector paper generation model. A discriminator is added to distinguish between real labeled images and generated labeled images, improving the generation effect. Furthermore, for low-resolution generated labeled images, a super-resolution module can be used for post-processing to optimize the details of the labeled images. Further, this embodiment can also add label consistency verification, automatically correcting logical conflicts in the generated labeled images through graph structure analysis (such as dimensional chain constraints in architectural floor plans), for example, contradictory dimension annotations on the same wall. This embodiment can also preset multiple labeling styles (such as architectural codes and hand-drawn styles), controlling the visual appearance of the generated results through additional style embedding.
[0084] Based on the above embodiments, the present invention also provides a vector paper generation model training system based on a diffusion model, which is used to implement the steps of the above scheme. Specifically, as Figure 3 As shown, the system in this embodiment includes: an image preprocessing module 10, a conditional input construction module 20, and a backpropagation optimization module 30. Specifically, the image preprocessing module 10 is used to preprocess the original image to obtain a target image, and input the target image and editing instructions into a diffusion model. Forward diffusion is performed based on the diffusion model to obtain a noisy spatial image, wherein the editing instructions are text descriptions for adding building dimension annotations. The conditional input construction module 20 is used to construct conditional inputs based on the editing instructions, the original image, and the noisy spatial image, and determine the loss function of the diffusion model based on the conditional inputs. The backpropagation optimization module 30 is used to optimize the model parameters of the diffusion model through a backpropagation algorithm, update the weights in the diffusion model using an optimizer, and obtain a vector paper generation model based on iterative training and validation.
[0085] The working principle of each module in the vector paper generation model training system based on the diffusion model in this embodiment is the same as that of each step in the above method embodiment, and will not be repeated here.
[0086] The modules in the aforementioned vector paper generation model training system based on the diffusion model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the terminal in hardware form or independent of it, or stored in the terminal's memory in software form, so that the processor can call and execute the corresponding operations of each module.
[0087] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 4 As shown. The terminal may include one or more processors 100 ( Figure 4(Only one is shown in the image), a memory 101, and a computer program 102 stored in the memory 101 and executable on one or more processors 100. For example, a vector paper generation model training program based on a diffusion model. When one or more processors 100 execute the computer program 102, they can implement the various steps in the embodiment of the vector paper generation model training method based on a diffusion model. Alternatively, when one or more processors 100 execute the computer program 102, they can implement the functions of each module / unit in the embodiment of the vector paper generation model training system based on a diffusion model, which is not limited here.
[0088] In one embodiment, the processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0089] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 can also be used to temporarily store data that has been output or will be output.
[0090] Those skilled in the art will understand that Figure 4 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, operational databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAM bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for training a vector paper generation model based on a diffusion model, characterized in that, The method includes: After preprocessing the original image, a target image is obtained. The target image and editing instructions are then input into a diffusion model. Forward diffusion is performed based on the diffusion model to obtain a noisy spatial image. The editing instructions are text descriptions used to add building dimension annotations. A conditional input is constructed based on editing instructions, the original image, and a noisy spatial image, and the loss function of the diffusion model is determined based on the conditional input. The model parameters of the diffusion model are optimized by backpropagation algorithm, and the weights in the diffusion model are updated by an optimizer. Based on iterative training and validation, a vector paper generation model is obtained.
2. The vector paper generation model training method based on the diffusion model according to claim 1, characterized in that, The process of preprocessing the original image to obtain the target image includes: The original image is normalized. The original image after normalization is subjected to size unification processing to obtain the target image.
3. The vector paper generation model training method based on the diffusion model according to claim 1, characterized in that, The step of inputting the target image and editing instructions into the diffusion model, and performing forward diffusion based on the diffusion model to obtain a noisy spatial image includes: The target image and editing instructions are input into the diffusion model, and the variational autoencoder in the diffusion model converts the target image from pixel space into latent space. For each target image, a random time step is sampled, and random noise is added to the latent space according to the noise intensity of each time step to obtain a noisy spatial image.
4. The vector paper generation model training method based on the diffusion model according to claim 3, characterized in that, The conditional inputs for constructing the system based on editing instructions, the original image, and the noisy spatial image include: The editing instructions are converted into text embeddings based on the CLIP encoder in the diffusion model. The image embedding corresponding to the original image is determined based on the variational autoencoder in the diffusion model; The conditional input is obtained by performing feature concatenation on the text embedding, the image embedding, and the noisy spatial image.
5. The vector paper generation model training method based on the diffusion model according to claim 4, characterized in that, The step of determining the loss function of the diffusion model based on the conditional input includes: The conditions are input into the UNet model to predict the noise distribution in the noisy spatial image, thus obtaining the predicted noise. The mean squared error between the predicted noise and the actual noise is used as the loss function of the diffusion model, where the actual noise is random noise added to the latent space.
6. The vector paper generation model training method based on the diffusion model according to claim 1, characterized in that, The method further includes: The original image is back-diffused based on the vector paper generation model to output an annotated image, and the annotated image is evaluated based on a preset evaluation index.
7. The vector paper generation model training method based on the diffusion model according to claim 6, characterized in that, The vector paper generation model performs back-diffusion on the original image to output an annotated image, including: The original image and the prompt words are input into the vector paper generation model, and the conditional embedding is determined based on the original image and the prompt words; Starting with random noise, and combining the conditional embedding, the noise is predicted and gradually removed using the UNet model; The inference process is optimized using the DDIM sampler, and after multiple rounds of iterative denoising, labeled images are gradually generated.
8. A vector paper generation model training system based on a diffusion model, characterized in that, The system is used to implement the steps of the vector paper generation model training method based on the diffusion model according to any one of claims 1-7, and the system includes: The image preprocessing module is used to preprocess the original image to obtain the target image, and input the target image and editing instructions into the diffusion model. Based on the diffusion model, forward diffusion is performed to obtain a noisy spatial image. The editing instructions are text descriptions used to add building dimension annotations. A conditional input construction module is used to construct conditional inputs based on editing instructions, the original image, and a noisy spatial image, and to determine the loss function of the diffusion model based on the conditional inputs; The backpropagation optimization module is used to optimize the model parameters of the diffusion model through the backpropagation algorithm, update the weights in the diffusion model using the optimizer, and obtain the vector paper generation model based on iterative training and validation.
9. A terminal, characterized in that, The terminal includes a memory, a processor, and a vector paper generation model training program based on a diffusion model stored in the memory and executable on the processor. When the processor executes the vector paper generation model training program based on a diffusion model, it implements the steps of the vector paper generation model training method based on a diffusion model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a vector paper generation model training program based on a diffusion model, the vector paper generation model training program based on a diffusion model implementing the steps of the vector paper generation model training method based on a diffusion model as described in any one of claims 1-7 on the computer-readable storage medium.