Method for artistic recycling and redesign of construction waste material resources

Abstract

The application discloses a method for artistic recycling and redesign of construction waste material resources, which is based on a multi-modal data set of construction waste combining vision and language, and constructs a diffusion generation model for artistic redesign. The model mainly includes an image-text encoding module, a diffusion generation module, and a request encoding module. By inputting construction waste materials as elements, combining multi-modal data with artistic design styles and design requirements, and using the diffusion generation model, a new generation method of artistic works of construction waste is provided. The simulation experiment results show that the generative artificial intelligence technology can flexibly use different types of construction waste materials, and generate corresponding artistic works by combining various artistic design styles and design requirements. This innovative method not only provides new ideas and tools for artistic creation, but also provides effective technical support for the recycling management and sustainable utilization of construction waste materials, and helps to further promote the recycling of resources and ecological sustainable development.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of building sustainability and artistic design, and provides an innovative solution for the artistic redesign and reuse of building waste materials. BACKGROUND

[0002] The artistic redesign and reuse of building waste materials refers to the process of giving artistic value to waste materials generated during the construction process through artistic design and transformation, and applying them to creative and production activities.

[0003] In order to further promote the artistic redesign of building waste materials, it is particularly important to establish a complete waste material recycling system. The classification, collection, transportation and treatment of building waste will provide a foundation for the redesign and reuse of materials. At the same time, strengthening the research and development of related technologies and exploring new reuse processes and technologies are also the core driving force for the development of this field.

[0004] In recent years, the development of generative artificial intelligence (Generative AI) technology has brought new opportunities for the artistic redesign of building waste materials. Generative AI can automatically generate new content such as images, text, music, etc., and has achieved success in many fields. The application of this technology in the artistic design of building waste materials can improve design efficiency and reduce costs in many ways.

[0005] From the perspective of artistic design, generative AI can significantly improve design efficiency and shorten the development cycle of artistic products. For example, designers can quickly generate a large number of design schemes through generative AI and screen and optimize them to speed up the creative process. In addition, generative AI can help designers break through their inherent thinking and stimulate new creative inspiration. It can generate completely different waste material redesign images from traditional design ideas, enrich design materials, and even create new artistic forms such as algorithmic art and data art, broadening people's understanding and cognition of art.

[0006] In terms of building waste material redesign, generative AI can quickly identify the reuse value of materials and propose optimized design schemes. For example, through generative models, it can identify potential design elements in waste materials and integrate them into new artistic works. At the same time, generative AI can simulate the effects of different design schemes to help designers choose the best scheme, thereby reducing production costs and material waste. More importantly, this technology can also help popularize waste material art, making this design form more attractive and promoting social acceptance of waste material artistic redesign.

[0007] In summary, generative artificial intelligence, as a promising technology, is deeply influencing the artistic redesign and reuse of construction waste materials. In the future, with the continuous advancement of technology, this field will face more innovation and challenges, further promoting the sustainable development of construction waste materials. SUMMARY

[0008] Currently, the main disposal methods for construction waste are landfilling and secondary use. While secondary use alleviates the environmental pollution caused by construction waste to some extent, its economic benefits are relatively limited. Landfilling not only fails to fully utilize resources, but also causes serious environmental pollution and generates a large amount of greenhouse gases, further exacerbating ecological problems. In addition, the invention makes it possible to transform waste into art by introducing generative AI technology, creating new economic value, and providing an innovative solution for construction waste disposal. The system exhibits good adaptability and scalability in practical applications, opening up new avenues for the resource utilization of construction waste.

[0009] The invention proposes a method for artistic recycling and redesign of construction waste materials based on a conditional generative model. This model encodes the input multi-modal construction waste materials, combines specific condition inputs, and uses a generative diffusion model to achieve artistic recycling and redesign of construction waste materials based on condition input requirements. This method has strong feasibility and high efficiency.

[0010] The core feature of this method is that, relying on a multi-modal dataset of construction waste, a text encoder and an image encoder are first used to extract and encode the text and image data of construction waste. Then, a condition input encoder encodes the design conditions (such as material type, artistic style, etc.). Finally, these text and image features, as well as condition features, are input into a conditional diffusion model to achieve artistic redesign and reuse of construction waste materials, achieving the effect of "turning waste into treasure".

[0011] Based on this, the technical solution adopted by the invention is: a method for artistic recycling and redesign of construction waste materials, comprising the following steps:

[0012] Extract the construction waste description word features and image features of the construction waste text dataset and image dataset using a text encoder and an image encoder, respectively;

[0013] Use a condition encoder to encode condition information (such as material type, artistic style, etc.);

[0014] The artistic resource recycling and redesign of construction waste are realized by diffusion generation model, and the construction waste material work image is output.

[0015] Further, the image encoder encodes the image input I by using a pre-trained image encoder f img The input image features are captured; the text encoder uses a pre-trained parameter-fixed text encoder f text The input text information is encoded.

[0016] Specifically, the image encoder encodes the image input I includes: given an image I containing k inputs, the image encoder f img Each input image is encoded respectively and Where N I represents the number of tokens contained in each image, represents the feature of the i-th image input, that is, the low-dimensional vector representation. represents the feature of the N I token of the input image I i represents the input image i, f img (I) represents the total feature of the image. The final feature representation of the image input is to add the features of the k input images, that is,

[0017] Specifically, the text encoder f text The input text information is encoded, which includes defining Where N T represents the number of words in the text information, f text (T) represents the total feature of the input image for the text, represents the feature of the N T word of the input text h T represents the total feature of the input image for the text.

[0018] Further, it also includes using a linear mapping function to map the image feature dimension d I to the text feature dimension d T .

[0019]

[0020] f represents the mapping function, which can convert the d I corresponding dimension vector into a R d dimension vector, so that it is equal to the text feature dimension.

[0021] Further, the conditional encoder encodes the conditional text or artistic style image through the encoder according to the input, respectively encodes the text request information or artistic style image through the text or image encoder, and obtains the request feature q through splicing, then further processes the request feature q by using the mirror trainable CDM encoder, and finally links the input of the mirror CDM encoder to the original CDM model. Wherein the image encoder adopts CNN, and the text encoder adopts Transformer.

[0022] Further, the diffusion generative model obtains a sample sequence with less noise from the noise z t Starting from the conditional representation I, T of each time step t, the sample sequence with less noise z t-1 ,…,z0is obtained in sequence through denoising; in order to construct the model f θ with θ as the parameter, at each time step t, the training objective function of the extended model aims to obtain the Gaussian noise representation z t Solve the model denoising problem, which is defined as:

[0023]

[0024] Indicates the objective function to be optimized, θ represents the parameters to be learned by the encoder, Indicates the expectation, ∈ represents the noise added by the forward process, z t I and T represent image features and text features respectively.

[0025] The model is trained in a large scale, mainly taking text information as the main control source, and denoising z θ through the trained model f t .

[0026] Specifically, the model f θ includes an encoder, an intermediate module and a decoder based on residual connection, that is, CDMEncoder, CDM Mid and CDM Decoder.

[0027] Further, it further includes a modified cross-attention mechanism defined as taking the noise feature z t as a request, and taking the text feature h T as the representation of the key K and the value V, which can be defined as:

[0028] z t =z t +CrossAtt(Q=z t ,K=V=[h I ,h T ])

[0029] where [h I ,h T ] denotes the concatenation of image and text features, Q denotes the request, CrossAtt denotes the cross-attention mechanism for interactive learning of image and text features, h I denotes the feature of the input image, h T denotes the text feature corresponding to the input image.

[0030] Further, a multi-task training strategy is also included, and a pre-trained image generator is used to align with the image input information, and the request information feature is introduced, and the training objective function is further defined as:

[0031]

[0032] y denotes the input building waste material image and text features, and q denotes the input image and text condition input, such as style control and task description.

[0033] Compared with existing processing methods, the advantages of the present application lie in that it can efficiently process building waste through generative artificial intelligence technology, creating new products with artistic value and commercial potential. At the same time, this method not only reduces the environmental pollution caused by landfill, but also promotes the recycling management and sustainable use of building waste, providing an environmentally friendly and innovative solution for waste disposal. The specific working principle is shown in Figure 1 The advantages of the present application are:

[0034] (1) Using generative artificial intelligence technology to solve the recycling, reuse and redesign of building waste can more quickly and efficiently realize the artistic transformation of building waste resources. Compared with traditional building waste disposal methods, this technology is more flexible and efficient.

[0035] (2) By encoding the text and image condition input condition encoder, the artistic style in the generation process can be flexibly controlled, so that the artistic transformation of building waste is not limited by a specific style, showing strong flexibility and adaptability.

[0036] (3) The application of generative artificial intelligence technology in building waste resource recycling and redesign effectively shortens the cycle of traditional artistic design, significantly improves design efficiency, and shows innovation and uniqueness, bringing new possibilities for the artistic processing of waste.

[0037] As Figure 3As shown, this system uses a Conditional Diffusion Model (CDM) as its core and employs a dual encoder-decoder structure. At the encoding end, a vision-language pre-trained model architecture (such as CLIP, a text-image encoder) is used to extract multimodal features of construction waste materials, including physical attributes such as material, shape, and texture. The conditional input module also utilizes the image-text encoder to generate conditional embedding vectors from style conditions and textual task descriptions. The denoising process of the diffusion model is guided by these conditional vectors, gradually refining random noise into artistic works. In the inference stage, the model can generate works with artistic value and environmental significance based on the input textual and image information of the construction waste materials and the target style description, realizing the creative reuse of construction waste. This method based on deep generative models not only preserves the characteristics of raw materials but also endows them with new aesthetic value, providing an innovative solution for the treatment of construction waste. Therefore, these innovative points work together to form an efficient, stable, and reliable artistic treatment system for construction waste. The overall design of the system not only solves the technical pain points of traditional treatment methods but also opens up new avenues for the high-value utilization of construction waste. Attached Figure Description

[0038] Figure 1 This is a forward process in the diffusion model;

[0039] Figure 2 This is the reverse process of the diffusion model;

[0040] Figure 3 Flowchart for transforming construction waste into art;

[0041] Figure 4 Flowchart for cross-creation of building waste materials - bricks;

[0042] Figure 5 Artwork created using construction waste materials (bricks). Detailed Implementation

[0043] 1. Mathematical Principles of the Diffusion Model

[0044] As the most typical model of generative AI, the diffusion model (reference [1]) gradually transforms random noise into the target data distribution by simulating the random diffusion process, thereby generating new data samples. The working principle of the diffusion model can be divided into two stages: the forward process and the reverse process. The forward process mainly starts from the real data sample and continuously adds noise until the data becomes pure Gaussian noise. This process can be regarded as a data corruption process. In contrast, the reverse process starts from pure Gaussian noise and gradually denoises to finally restore the real data sample. This process can be regarded as a data repair process. At present, the diffusion model has been widely used in image generation, text generation, audio generation and video generation. It should be noted that the diffusion model has the following advantages: generating high-quality samples, relatively simple training process, and can be applied to various data types.

[0045] Forward process: such as Figure 1 As shown, given a set of real data samples x0~q(x0), the forward process adds Gaussian noise to the sample data T times. The q-process is shown in the figure, where the magnitude of each step is determined by the Gaussian distribution variance hyperparameter. The forward process is controlled by T, where T represents an adjustable parameter. Since each time step t depends only on time step t-1, the forward process can be defined as the process from data sample x0 to the latent variable x. T Markov chains:

[0046]

[0047] In the formula, — represents the Gaussian distribution variance hyperparameter, where β1 < β2 < ... < β T As t increases, x... t It's getting closer and closer to pure noise. 1:T q(x) represents the total conditional probability after adding noise at each step of the input x0. t |x t-1 ) represents the conditional probability after denoising. Let x represent a Gaussian distribution, I be a unit vector, and T represent the time step. As T→∞, x... T Represents Gaussian noise, and is related to The choice doesn't matter, β t Let x represent the noise level at time step t. Considering that random samples in the distribution cannot be propagated using back gradient, a reparameterization technique is needed to make it differentiable (see reference [2]). Meanwhile, x t It can be defined as

[0048]

[0049] In the formula, At the same time, ∈ represents noise sampling, β i represents the noise level at the t-th time step.

[0050] Assume that z is sampled from a Gaussian distribution The reparameterization technique can be defined as:

[0051]

[0052] In the formula, z is a variable accompanied by randomness, and satisfies a Gaussian distribution with mean μ θ and variance . In particular, the parameters μ θ and can be obtained by a neural network parameterized by θ. z represents the sample after resampling, μ θ represents the mean of the Gaussian parameters, σ θ represents the standard deviation of the Gaussian parameters, ⊙ represents multiplication, and ∈ represents Gaussian sampling. The process is still gradient-derivable through the “sampling” process, and randomness is transferred to the parameter ∈.

[0053] Reverse process: as shown in Figure 2 , the reverse process of the diffusion model, also known as the denoising inference process, aims to recover the original data distribution from the noisy data distribution . Similarly, this process can be defined as a Markov chain as follows:

[0054]

[0055] In the formula, μ θ (x t ,t) represents the mean, σ θ (x t ,t) represents the variance, and both the mean and the variance are learned by a neural network parameterized by θ. p θ (x 0:T ) represents the “integral” of all possible paths of the data sample starting from pure noise, p(x T ) represents the probability of x T , p θ (x t-1 |x t ) represents the conditional probability of obtaining x t from x t-1 , and t represents the time step. Based on reference [3], for any β t represents the noise level at the t-th time step. At the same time the parameters μ θ and σ θ are defined as:

[0056]

[0057] ∈ θ denotes the noise size learned through the network.

[0058] 2 Mathematical principles of diffusion model

[0059] Diffusion model is a kind of generative model, which can convert Gaussian noise into samples from the learned data distribution through denoising in an iterative manner. The present invention further studies the diffusion model for text-to-image generation of architectural waste artistic redesign and reuse based on multi-modal architectural waste material features. That is, starting from noise z t , based on the conditional representation I, T at each time step t, the sample sequence with smaller noise can be obtained in turn through denoising, such as z t-1 ,…,z0. In order to construct the model f θ with θ as the parameter, at each time step t, the training objective function of the extended model aims to solve the model denoising problem based on the Gaussian noise representation z t , which can be defined as:

[0060]

[0061] denotes the objective function, θ denotes the learnable parameters of the network, denotes the expectation.

[0062] It should be pointed out that through large-scale training, the present invention mainly takes text information as the main control source, and through training the model f θ to denoise z t . Figure 3 The model framework for architectural waste artistic redesign and reuse is described, which uses image and text encoders to encode input images and text information respectively. The CDM model is mainly used for image generation, and the additional CDM encoder is mainly used to capture the information of the text generation condition, that is, the request style information and the text information. It is worth noting that, Figure 3 only the encoder for single multi-modal architectural waste information is shown, such as bricks. In fact, the model proposed in the present invention can simultaneously encode different types of architectural waste image and text information.

[0063] Inspired by the UNet structure (reference [4]), the model f θ proposed in the present invention includes an encoder, an intermediate module and a decoder based on residual connection, which correspond to Figure 3CDM Encoder, Mid, and Decoder. Moreover, these models are also based on ResNet (Reference [5]) and Transformer (Reference [6]) structures. At time step t, the conditional diffusion model aims to encode the conditional text information based on Gaussian noise z t , and then generate the corresponding style artwork by understanding the semantics of the text information and combining the features of the multi-modal construction waste input. As shown in Figure 3 , the context information includes the construction waste image input and the text information. The purpose of designing this module is to introduce construction waste materials so that the model learns to master the features of different construction waste materials, and defines the condition as y = (I, T), where I represents the image features and T represents the text features. At the same time, the invention also introduces the style control and the task description of the corresponding construction waste artistic design as the request information q and defines the form of the output artwork.

[0064] 2.1 Image and text information encoding

[0065] As shown in Figure 3 , for the encoding of the image input I, the invention uses a pre-trained image encoder f img to capture the input image features. Given an image I containing k inputs (such as bricks, wood), the image encoder f img encodes each input image separately and obtains where N I represents the number of tokens contained in each image, and represents the feature of the i-th image input, i.e., a low-dimensional vector representation. Where d I represents the dimension of a single word encoding. represents the feature of the N I th token of the input image, and N I represents the Nth token of the image. The final feature representation of the image input is the sum of the features of the k input images, i.e., Similarly, in order to better capture the feature information of the text prompt words, the invention uses a pre-trained text encoder f text to encode the input text information. In particular, define where N T represents the number of words in the text information, and represents the feature of the i-th input text. Where d T represents the dimension of a single word encoding. In addition, a linear mapping function is further used to map the image feature dimension d IMapping to text feature dimension d T In addition, the image and text encoders are used to encode the style and text request information, respectively, and the request feature q is obtained by concatenation. Then, the mirror CDM encoder is used to further process the request feature q. Finally, the input of the mirror CDM encoder is linked to the original CDM model.

[0066] 2.2 Modified cross-attention mechanism

[0067] Figure 3 The detailed modified cross-attention mechanism is given in the lower left corner. Unlike the standard cross-attention mechanism in the CDM model, which takes the request Q, the keyword K, and the value V, here, the modified cross-attention mechanism is defined as the representation of the noise feature z t as the request, the text feature h T as the keyword K and the value V, which can be defined as:

[0068] z t = z t + CrossAtt (Q = z t , K = V = [h I , h T ]) (10)

[0069] where [h I , h T ] represents the concatenation of image and text features. CrossAtt represents the cross-attention mechanism, which is used for interactive learning of image and text features, h I represents the feature of the input image.

[0070] 2.3 Multi-task training strategy

[0071] The pre-trained image generator is used to align the image input information. The request information feature is introduced, and formula (9) can be further defined as:

[0072]

[0073] y represents the input building waste material image text feature, and q represents the input image text condition input, such as style control and task description.

[0074] 2.4 Experimental setup

[0075] All data set processing experiments of the present application are completed in the Python 3.6 environment, and are implemented using the deep learning library Pytorch 1.4 installed on the server. When training the deep learning classifier, 8 Nvidia TITAN RTX GPUs are used to train the artistic diffusion model of construction waste. The pre-training of the model is based on 310,000 pairs of synthesized image-text pairs, and is performed using the image-text retrieval alignment method. Subsequently, the VL-CSW method proposed in reference [7] is used to fine-tune the model, so that it can learn the knowledge related to construction waste materials. When training the CDM module, only the mirror model is trained, while the original model is frozen. To encode the image and text features, a ViT-L / 14 encoder is used, which is similar in structure to the Transformer (reference [6]). We use the output of the last layer of ViT-L / 14 as the input of the image-text feature. It should be noted that the present application uses generative AI models to realize the artistic transformation of construction waste, combining artistic categories (such as oil painting) and artistic styles (such as classicism). Since the output is an artistic work based on construction waste, the present application does not use traditional evaluation criteria to evaluate the generated results. In particular, by combining various artistic forms (such as oil painting, printmaking, sculpture, Chinese painting, and integrated materials) and various artistic styles (such as classicism, romanticism, realism, and realism) in a variety of cross combinations as shown in Figure 4 , the features of the construction waste material bricks can be extracted for secondary artistic creation, which can further promote the sustainable reuse of construction waste, and the results are shown in Figure 5 .

[0076] References

[0077] [1] Rethmeier N, Augenstein I. A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned & Perspectives [J]. ACM Computing Surveys (CSUR), 2021.

[0078] [2] Kingma D, Welling M. Auto-encoding variational bayes [J]. arXiv preprint arXiv:1312.6114, 2013.

[0079] [3] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models [C] / / Proceedings of the Advances in neural information processing systems, 2020, 33: 6840-6851.

[0080] [4] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation [C] / / Proceedings of the Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.

[0081] [5] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.

[0082] [6] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding [J]. arXiv preprint arXiv:1810.04805, 2018.

[0083] [7] Sun Y, Gu ZL, and Yang SB. Probing Vision and Language Models for Construction Waste Material Recognition [J]. Automation in Construction, 2024: 166: 105629.

[0084] [8] Radford A, Kim J, Hallacy C, et al. Learning transferable visual models from natural language supervision [C] / / Proceedings of the International conference on machine learning. 2021: 8748-8763.

Claims

1. A method for artistic recycling and redesign of construction waste material resources, characterized in that, The method comprises the following steps: extracting the construction waste description word features and image features of the construction waste text data set and image data set through a text encoder and an image encoder respectively; using a conditional encoder to encode the condition information; the conditional encoder encodes the condition text or artistic style image through the encoder, encodes the text request information or artistic style image through the text or image encoder respectively, and obtains the request feature q through splicing, then further processes the request feature q by using the mirror trainable CDM encoder, and finally links the input of the mirror CDM encoder to the original CDM model; combining the construction waste description word features, image features and condition features, outputting the construction waste material work image through the diffusion generative model; The diffusion generation model generates a sequence of samples from noise z t Starting from the conditional representation I, T at each time step t, a sequence of samples with less noise z t-1 ,…, z0 can be obtained in turn by denoising, in order to construct the model f θ At each time step t, the training objective function of the extended model aims to represent the Gaussian noise z t Solve the model denoising problem, which is defined as: represents the objective function that needs to be optimized, θ represents the parameters that the encoder needs to learn, represents the expectation, ∈ represents the target image, z t The table is Gaussian noise, I, T respectively represent image features and text features; The model is trained on a large scale, mainly taking text information as the main control source, and training the model f θ z t de-noising; The modified cross-attention mechanism is defined as in Equation 1 with the noise feature z t As a request, in text feature h T As a representation of the key K and the value V, it is defined as: z t = z t + CrossAtt(Q = z t , K = V = [h I , h T ]) In the formula, [h I ,h T ] represents splicing of image and text features, Q represents a request, CrossAtt represents a cross-attention mechanism for interactive learning of image and text features, h I represents a feature of an input image, and h T represents a text feature corresponding to the input image.

2. The method for artistic recycling and redesigning of construction waste material resources according to claim 1, characterized in that: The image encoder encodes the image input I, adopts a pre-trained image encoder f with fixed parameters c img The input image features are captured; the text encoder adopts a pre-trained text encoder f with fixed parameters text The input text information is encoded.

3. The method for artistic reusing and redesigning of construction waste material resources according to claim 2, characterized in that: The image encoder encodes the image input I comprises: given an image I containing k inputs, the image encoder f img Each input image is encoded separately and the result is where N I denotes the number of tokens contained in each image, denotes the feature of the i-th image input, denotes the feature of the N I token of the input image i, i denotes the input image i, f img (I) denotes the total feature of the image, the final feature representation of the image input is the sum of the features of the k input images, i.e.

4. The method for artistic reusing and redesigning of construction waste material resources according to claim 2, characterized in that: The text encoder f text Encoding the input text information includes defining where N T represents the number of words in the text information, f rext (T) represents the total feature of the input image for the text, represents the feature of the N T th word of the input text, h T represents the total feature of the input image for the text.

5. The method for artistic reusing and redesigning of construction waste material resources according to claim 3 or 4, characterized in that: Also included is employing a linear mapping function to map image feature dimension d I to text feature dimension d T .

6. The method for artistic reusing and redesigning of construction waste material resources according to claim 1, characterized in that: The model f θ comprising an encoder, an intermediate module and a decoder based on residual connections, i.e. CDM Encoder, CDM Mid and CDM Decoder.

7. The method for artistic reusing and redesigning of construction waste material resources according to claim 1, characterized in that: Further comprising a multi-task training strategy, a pre-trained image generator is used to align with the image input information, the request information feature is introduced, and the training target function is further defined as: y represents the input construction waste material image text feature, and q represents the input image text condition input.

Images