Generation method and device of assembled building block model, equipment and storage medium

Abstract

The invention provides an assembled building block model generation method and device, equipment and a storage medium, and the method comprises the steps: obtaining a source picture and a to-be-mapped model rendering graph, and generating an initial mapping rendering graph according to the source picture and the to-be-mapped model rendering graph through a deep learning diffusion model; taking the initial chartlet rendering image as a target chartlet rendering image, and carrying out matting processing on the target chartlet rendering image to obtain a matting bitmap; and taking the matting bitmap as a projection map, and carrying out projection mapping on the projection map according to a preset block structure of an assembled building block model so as to realize automatic mapping of the assembled building block model and obtain the assembled building block model after mapping is completed. According to the method, the deep learning technology and the image processing technology are combined, the full-automatic generation process from user input to the final chartlet model is achieved, the efficiency and quality of building block chartlet design are remarkably improved, and the problem that time and labor are consumed in a traditional manual design method is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating an assembled building block model. Background Art

[0002] In the field of building block model design, texture design is a key link, which can significantly improve the visual effect and expressiveness of building block models. The traditional building block texture design method mainly relies on the manual operation of designers. Designers usually need to use professional graphic design software to draw the texture of each building block one by one according to user needs and internal specifications. This process involves multiple steps, including pattern design, color adjustment, texture addition, and matching with the shape of the building blocks.

[0003] However, this traditional manual design method has a significant drawback: it is inefficient and time-consuming. For a set of complex building block products, designers often need to spend a lot of time to complete the texture design. Especially when dealing with complex textures or fine patterns, the design process is more time-consuming and sometimes even difficult to achieve the desired effect. This inefficiency not only increases the product development cycle, but also limits the innovation and diversity of the design. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problems that the existing mapping design method in the process of building block model design is inefficient and time-consuming and labor-intensive; A first aspect of the present invention provides a method for generating an assembled building block model, the method comprising: Obtain a source image and a model rendering of a to-be-mapped image, and generate an initial mapping rendering according to the source image and the model rendering of the to-be-mapped image through a deep learning diffusion model; Using the initial texture rendering as a target texture rendering, and performing a cutout process on the target texture rendering to obtain a cutout bitmap; The cutout bitmap is used as a projection map, and projection mapping is performed on the projection map according to a preset block structure of the assembled building block model, so as to realize automatic mapping of the assembled building block model and obtain the assembled building block model with the mapping completed.

[0005] Optionally, in a first implementation of the first aspect of the present invention, obtaining a source image and a model rendering to be mapped, and generating an initial mapping rendering according to the source image and the model rendering to be mapped by a deep learning diffusion model includes: Obtaining a source image and a model rendering to be mapped, and performing multi-scale feature extraction on the source image to obtain a feature map set containing multi-scale visual features; According to the geometric structure of the model rendering to be mapped, the visual features in the feature map set are adaptively spatially transformed to obtain a transformed feature map set aligned with the model rendering; Combining the transformed feature map set and the model rendering map as conditional input, and inputting them into a pre-trained deep learning diffusion model to obtain an initial noise map; An iterative denoising process is performed on the initial noise image, and a denoised image is generated or updated in each iteration until a preset number of iterations is reached, and the denoised image corresponding to the number of iterations is used as the initial texture rendering image.

[0006] Optionally, in a second implementation of the first aspect of the present invention, combining the transformed feature map set and the model rendering map as conditional input and inputting them into a pre-trained deep learning diffusion model to obtain an initial noise map includes: Combining the transformed feature map set and the model rendering map as conditional input, and inputting the conditional input into a pre-trained deep learning diffusion model, wherein the deep learning diffusion model includes a conditional embedding module and a U-Net denoising network; Using a conditional embedding module to encode the transformed feature map set and the model rendering map to obtain a conditional embedding vector, and generating a random noise map with the same size as the model rendering map according to a preset initial noise distribution; The conditional embedding vector and the random noise map are input into a U-Net denoising network to obtain predicted noise, and the random noise map is sampled and updated according to the predicted noise and a preset noise schedule to obtain an initial noise map.

[0007] Optionally, in a third implementation of the first aspect of the present invention, after using the initial texture rendering as the target texture rendering, the method further includes: Determining whether the initial texture rendering as the target texture rendering is a high-resolution rendering; If not, performing multi-scale decomposition on the initial texture rendering to obtain a set of sub-images with different frequency components, and performing up-sampling processing on each sub-image in the set of sub-images according to a preset super-resolution factor to obtain an amplified set of sub-images; Inputting the enlarged sub-image set into a pre-trained super-resolution diffusion model, performing iterative denoising processing on each enlarged sub-image to obtain a sub-image set with enhanced high-frequency details; Adaptively fusing the high-frequency detail-enhanced sub-image set to obtain a preliminary super-resolution image, and processing the preliminary super-resolution image using an edge-aware detail enhancement network to obtain a high-resolution texture image; The target texture rendering is updated according to the high-resolution texture image.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, performing a cutout process on the target texture rendering to obtain a cutout bitmap includes: Generate a rendering of the model to be mapped with the same shape and perspective according to the shape and perspective of the high-resolution mapping image; Perform pixel-by-pixel difference calculation on the high-resolution texture image and the model rendering image to be textured, and sum the difference results on multiple color channels after taking absolute values ​​to obtain a difference matrix; Binarization is performed on the difference matrix according to a preset threshold value, pixels greater than the threshold value are retained, and pixels less than the threshold value are set to be transparent, so as to obtain a preliminary cutout bitmap; The Sobel operator is applied to the preliminary cutout bitmap for edge detection, and abnormal edges are removed to obtain a cutout bitmap.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the step of using the cutout bitmap as a projection map and performing projection mapping on the projection map according to a preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model, and obtaining the assembled building block model with the mapping completed includes: The cutout bitmap is used as a projection map, and whether there is an adjustment instruction for the cutout bitmap used as the projection map is detected. If there is, the cutout bitmap is converted into a vector map to obtain a corresponding map vector map, and the projection map is updated according to the map vector map; According to the block structure of the preset assembled building block model, the projection map is segmented to obtain a sub-projection map set corresponding to each building block of the block structure; Performing geometric transformation and projection calculation on each sub-projection map in the sub-projection map set to obtain a projection map that matches the surface of the corresponding building block; According to the three-dimensional geometric information of the building block, UV coordinate mapping is performed on the projection map to obtain mapping coordinate information, and the mapping coordinate information is associated with the three-dimensional model of the corresponding building block to obtain a building block model set with mapping information; The building block model set with the texture information is assembled and rendered to obtain an assembled building block model with the texture completed.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, performing geometric transformation and projection calculation on each sub-projection map in the sub-projection map set to obtain a projection map that matches the surface of the corresponding building block includes: Performing edge detection and contour analysis on each sub-projection map in the sub-projection map set to obtain a control point set representing a main contour of the sub-projection map; According to the three-dimensional geometric information of the corresponding building block, the normal vector and main direction of the building block surface are calculated to obtain the surface parameterized description; Using the control point set and the surface parameterization description, a projection mapping function is constructed to calculate the mapping relationship between the sub-projection map and the surface of the building block; According to the mapping relationship, geometric transformation and projection calculation are performed on the sub-projection map to obtain a projection map that matches the surface of the corresponding building block.

[0011] A second aspect of the present invention provides a device for generating an assembled building block model, the device for generating an assembled building block model comprising: An image diffusion module is used to obtain a source image and a model rendering to be mapped, and generate an initial mapping rendering according to the source image and the model rendering to be mapped through a deep learning diffusion model; A cutout module, used to use the initial texture rendering as a target texture rendering, and perform a cutout process on the target texture rendering to obtain a cutout bitmap; The mapping projection module is used to use the cutout bitmap as a projection map and perform projection mapping on the projection map according to a preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model and obtain an assembled building block model with mapping completed.

[0012] A third aspect of the present invention provides a device for generating an assembled building block model, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the device for generating the assembled building block model executes the steps of the above-mentioned method for generating the assembled building block model.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the above-mentioned method for generating an assembled building block model.

[0014] The above-mentioned method, device, equipment and storage medium for generating the assembled building block model obtains the source image and the model rendering to be mapped, and generates an initial mapping rendering according to the source image and the model rendering to be mapped through a deep learning diffusion model; the initial mapping rendering is used as the target mapping rendering, and the target mapping rendering is cut out to obtain a cutout bitmap; the cutout bitmap is used as a projection map, and the projection map is projected according to the preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model and obtain the assembled building block model with the mapping completed. This method realizes the fully automatic generation process from user input to the final mapping model by combining deep learning technology and image processing technology, significantly improving the efficiency and quality of building block mapping design, and solving the problem of time-consuming and labor-intensive traditional manual design methods.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of a first embodiment of a method for generating an assembled building block model in an embodiment of the present invention; Figure 2 Schematic diagram of an embodiment of a device for generating an assembled building block model according to an embodiment of the present invention; Figure 3 Schematic diagram of an embodiment of a device for generating an assembled building block model in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0020] To facilitate understanding of this embodiment, a method for generating an assembled building block model disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: 101. Obtain a source image and a model rendering to be mapped, and generate an initial mapping rendering according to the source image and the model rendering to be mapped by using a deep learning diffusion model; In one embodiment of the present invention, the step of obtaining a source image and a model rendering image to be mapped, and generating an initial mapping rendering image based on the source image and the model rendering image to be mapped through a deep learning diffusion model includes: obtaining a source image and a model rendering image to be mapped, and performing multi-scale feature extraction on the source image to obtain a feature map set containing multi-scale visual features; performing adaptive spatial transformation on the visual features in the feature map set according to the geometric structure of the model rendering image to be mapped, and obtaining a transformed feature map set aligned with the model rendering image; combining the transformed feature map set and the model rendering image as conditional inputs, and inputting them into a pre-trained deep learning diffusion model to obtain an initial noise map; performing an iterative denoising process on the initial noise map, generating or updating a denoised image in each iteration, until a preset number of iterations is reached, and using the denoised image corresponding to the number of iterations as the initial mapping rendering image.

[0021] Specifically, first, obtain the source image and the model rendering to be mapped. The source image is usually a high-resolution real-life photo or other reference image uploaded by the user, and the format may include common image formats such as JPEG and PNG. The system will preprocess these images, including unifying the size, adjusting the color space (such as converting from RGB to Lab color space), and performing basic image enhancement (such as contrast adjustment). The model rendering to be mapped is a three-dimensional rendering result of the building block model pre-generated by 3D modeling software (such as Blender or Maya). This rendering contains the geometric information, material properties and lighting information of the building block model, and is usually saved in high-resolution PNG format to retain transparency information. After obtaining these two inputs, the system will perform a preliminary analysis on them, including extracting basic image properties (such as size, color range) and performing simple quality checks (such as detecting whether the image is complete, whether there is severe noise or blur).

[0022] Specifically, multi-scale feature extraction is performed on the source image to obtain a set of feature maps containing multi-scale visual features. This step uses a pre-trained deep convolutional neural network, such as VGG19 or ResNet50. Each layer of the network extracts features of different scales and abstraction levels. Specifically, shallow networks (such as the first few convolutional layers) mainly capture low-level visual features, such as edge, texture, and color information; intermediate-level networks capture more complex structures, such as shape and local patterns; and deep networks extract high-level semantic features, such as partial or overall structures of objects. In practice, multiple key layers of the network are selected as feature extraction points, usually including the last layer of each convolutional block. The feature maps extracted from these layers are saved to form a set of multi-scale feature maps. Each feature map is a three-dimensional tensor whose spatial dimension gradually decreases with the depth of the network, while the number of channels gradually increases. For the convenience of subsequent processing, these feature maps are normalized to the same numerical range (such as between 0 and 1). In addition, the feature maps are spatially upsampled to make their spatial dimensions consistent with the original image, which makes it easier to align and fusion features in subsequent steps.

[0023] Specifically, according to the geometric structure of the model rendering to be mapped, the visual features in the feature map set are adaptively spatially transformed to obtain a set of transformed feature maps aligned with the model rendering. This step uses a spatial transformer network (STN) or a deformable convolutional network (DCN). First, the system analyzes the geometric structure of the model rendering and extracts key points and contour information. This is usually achieved through edge detection algorithms (such as the Canny edge detector) and contour extraction algorithms (such as the findContours function in OpenCV). The extracted geometric information is used to guide the parameter prediction of the transformation network. The spatial transformer network consists of three main parts: a localization network, a grid generator, and a sampler. The localization network receives the model rendering and the current feature map as input and outputs transformation parameters (such as an affine transformation matrix). The grid generator creates a sampling grid based on these parameters, and the sampler uses this grid to resample the feature map. This process is performed for each feature map in the feature map set, and the result is a set of transformed feature maps that are spatially aligned with the model rendering. In the implementation, residual connections are also added to retain the original feature information and prevent information loss caused by excessive deformation. In addition, in order to process features of different scales, a multi-scale spatial transformer network may be used, where each scale is responsible for processing feature maps of the corresponding resolution.

[0024] Specifically, the transformed feature map set and the model rendering are combined as conditional inputs and input into the pre-trained deep learning diffusion model to obtain the initial noise map. This step first fuses the transformed feature map set and the model rendering, usually using channel splicing or attention mechanism. The fused data is used as conditional input to guide the generation process of the diffusion model. The diffusion model is a generative model based on gradual noise removal. Its core idea is to regard the image generation process as a process of gradually reducing noise. In the training phase, the model learns how to gradually add noise to the clear image until it becomes pure noise; in the generation phase, this process is performed in reverse. Specifically in this application, the diffusion model receives the fused conditional input and generates an initial noise map with the same size as the target map. This initial noise map is a random noise distribution, usually following a Gaussian distribution, and its variance is determined according to a predefined noise schedule. The generation of the initial noise map marks the beginning of the denoising process and provides a starting point for subsequent iterative denoising.

[0025] Specifically, the initial noise map is subjected to an iterative denoising process, and a denoised image is generated or updated in each iteration until a preset number of iterations is reached, and the final denoised image is used as the initial map rendering. This process is the core of the diffusion model to generate images. In each iteration, the model performs the following steps: First, based on the current noise level (determined by the preset noise schedule) and the conditional input, a neural network with a U-Net structure is used to predict the noise at the current time step. This U-Net network is the main component of the diffusion model, which processes inputs at different spatial scales and can capture the multi-scale features of the image. The predicted noise is then subtracted from the current image to obtain an updated, less noisy image. This process is repeated many times, usually tens to hundreds of iterations. The choice of the number of iterations requires a trade-off between generation quality and computational efficiency. Each iteration slightly reduces the noise in the image while gradually introducing more details and structures. The conditional input plays a key role in the whole process, ensuring that the generated image matches the visual features of the source image and the geometric structure of the model. The image obtained from the final iteration is used as the initial texture rendering, which combines the visual features of the source image and the geometric structure of the model, laying the foundation for the subsequent super-resolution processing and cutout process.

[0026] Furthermore, the method of combining the transformed feature map set and the model rendering map as conditional input and inputting them into a pre-trained deep learning diffusion model to obtain an initial noise map includes: combining the transformed feature map set and the model rendering map as conditional input and inputting them into a pre-trained deep learning diffusion model, wherein the deep learning diffusion model includes a conditional embedding module and a U-Net denoising network; using the conditional embedding module to encode the transformed feature map set and the model rendering map to obtain a conditional embedding vector, and generating a random noise map with the same size as the model rendering map according to a preset initial noise distribution; inputting the conditional embedding vector and the random noise map into a U-Net denoising network to obtain predicted noise, and sampling and updating the random noise map according to the predicted noise and a preset noise schedule to obtain an initial noise map.

[0027] Specifically, first, the transformed feature map set and the model rendering are combined as conditional inputs. This step first preprocesses the feature map set. Each feature map passes through a 1x1 convolution layer to adjust the number of channels so that all feature maps have the same number of channels, usually set to 256 or 512. Then, the spatial resolution of all feature maps is adjusted to the same as the model rendering using a bilinear interpolation method. For the model rendering, a small convolutional network (for example, containing 3 3x3 convolutional layers, each followed by a BatchNorm and ReLU activation function) is first used for feature extraction to obtain a feature representation with the same number of channels as the adjusted feature map. Next, the channel concatenation method is used to concatenate the processed feature map set and the model rendering features in the channel dimension. Specifically, if there are N feature maps, and the number of channels of each feature map and the model rendering feature is C, then the number of channels of the concatenated tensor will be (N+1)*C. This concatenation operation preserves the spatial information of all inputs, allowing the subsequent network to fully utilize multi-scale features and model geometry information. Finally, a global average pooling operation is applied to the concatenated tensor to obtain a global feature vector, which will serve as the input of the conditional embedding module.

[0028] Specifically, next, the conditional embedding module is used to encode the transformed feature map set and the model rendering map to obtain the conditional embedding vector. The conditional embedding module is a multi-layer perceptron (MLP) network, which usually contains 3 to 4 fully connected layers, each followed by LayerNorm and GELU activation functions. The first fully connected layer maps the input global feature vector to a higher dimension (such as 1024 or 2048), the middle layer keeps this dimension unchanged, and the last layer compresses the features to the required embedding dimension (such as 256 or 512). Between each fully connected layer, a Dropout layer (the dropout rate is usually set to 0.1) is inserted to prevent overfitting. In addition, in order to enhance the sensitivity of the model to the input, a self-attention mechanism is introduced in the conditional embedding module. Specifically, before the last fully connected layer, a multi-head self-attention layer (usually with 8 attention heads) is used to allow the model to establish long-distance dependencies between different feature dimensions. Finally, the conditional embedding vector is normalized by a LayerNorm layer to stabilize the subsequent training process. At the same time, a random noise map is generated according to the preset initial noise distribution. Here, the standard normal distribution N(0, 1) is used as the initial noise distribution. PyTorch's torch.randn function is used to generate a random tensor with the same spatial dimensions (height H and width W) and number of channels (usually 3, corresponding to the three RGB channels) as the model rendering. Each element of this tensor is independently sampled from the standard normal distribution. In order to ensure that the generated noise is compatible with the rest of the model in terms of numerical range, the noise is scaled, usually multiplied by a small coefficient (such as 0.1).

[0029] Finally, the conditional embedding vector and the random noise map are input into the U-Net denoising network. The encoder part of the U-Net network contains 4 downsampling blocks, each of which contains two 3x3 convolutional layers, each of which is followed by BatchNorm and LeakyReLU activation functions, and then a 2x2 maximum pooling layer. The decoder part has 4 upsampling blocks accordingly, each of which is first upsampled using transposed convolution, then concatenated with the feature map of the corresponding layer of the encoder, followed by two 3x3 convolutional layers, also followed by BatchNorm and LeakyReLU. The conditional embedding vector is injected into each convolution block by addition or multiplication. The specific implementation is to map the embedding vector to the same number of channels as the feature map through a linear layer, and then perform element-by-element addition or multiplication. The output of U-Net is the predicted noise, which has the same dimension as the input noise map. Then, the random noise map is sampled and updated according to the predicted noise and the preset noise schedule. The noise schedule uses a cosine decay function: ; Where t is the current step number, T is the total number of steps, and s is a small constant (such as 0.008). The update formula is: ; ,in is the predicted noise, is additional random noise. This updating process is repeated many times (usually 50 to 100 times) to finally get the initial noise map.

[0030] 102. Using the initial texture rendering as a target texture rendering, and performing a cutout process on the target texture rendering to obtain a cutout bitmap; In one embodiment of the present invention, after taking the initial texture rendering as the target texture rendering, the method further includes: determining whether the initial texture rendering as the target texture rendering is a high-resolution rendering; if not, performing multi-scale decomposition on the initial texture rendering to obtain a set of sub-images of different frequency components, and upsampling each sub-image in the sub-image set according to a preset super-resolution factor to obtain a set of enlarged sub-images; inputting the set of enlarged sub-images into a pre-trained super-resolution diffusion model, performing iterative denoising on each enlarged sub-image to obtain a set of sub-images with high-frequency detail enhancement; adaptively fusing the set of sub-images with high-frequency detail enhancement to obtain a preliminary super-resolution image, and processing the preliminary super-resolution image using an edge-aware detail enhancement network to obtain a high-resolution texture image; and updating the target texture rendering according to the high-resolution texture image.

[0031] Specifically, the initial texture rendering generated by the deep learning diffusion model based on the source image and the model rendering to be mapped can be directly a high-resolution texture. However, in the case of directly outputting a high-resolution texture, the cost of texture generation is relatively high. Therefore, the initial texture rendering can be output as a high-resolution texture and a low-resolution texture based on cost considerations. When the output is a low-resolution texture, the initial texture rendering needs to be processed with ultra-high resolution, and the new texture rendering obtained by processing is used as the target texture rendering.

[0032] First, the initial texture rendering is decomposed at multiple scales to obtain a set of sub-images with different frequency components. This step uses the Laplacian Pyramid algorithm for multi-scale decomposition. In specific implementation, the original image is first blurred using a 5x5 Gaussian kernel, and then downsampled by 2 times to obtain a low-frequency sub-image. This low-frequency sub-image is upsampled back to the original size and subtracted from the original image to obtain a high-frequency sub-image. This process is recursive, usually resulting in 4 to 5 sub-images of different scales. Each sub-image represents the detail information of the original image within a specific frequency range. Next, each sub-image is upsampled according to a preset super-resolution factor (usually 2x or 4x). Here, the bicubic interpolation method is used for upsampling, which can better preserve edge information while maintaining image smoothness. For each sub-image, the interpolation kernel is first calculated, and then the image is convolved to obtain the enlarged sub-image. This process retains the unique frequency characteristics of each sub-image, providing a basis for subsequent detail enhancement.

[0033] Next, the set of enlarged sub-images is input into the pre-trained super-resolution diffusion model. This model adopts the Conditional Diffusion Probabilistic Model structure, which includes a U-Net backbone network and a temporal embedding module. The U-Net network contains 4 downsampling blocks and 4 upsampling blocks, each of which contains 3 residual blocks, and each residual block contains two 3x3 convolutional layers with SiLU activation function in the middle. The temporal embedding module uses the sinusoidal position encoding method to convert the time step into a 512-dimensional embedding vector. For each enlarged sub-image, the model performs an iterative denoising process of 200 steps. In each step, the model predicts the noise residual and then updates the image state according to the predefined noise schedule (using the cosine decay function). Through this iterative process, the model gradually enhances the high-frequency details of each sub-image and generates a set of sub-images with enhanced high-frequency details.

[0034] Finally, the sub-images with high-frequency detail enhancement are adaptively fused to obtain a preliminary super-resolution image. This step uses a multi-scale attention fusion network. The network first applies a 3x3 convolution layer to each sub-image to extract features, and then uses the channel attention mechanism and spatial attention mechanism to calculate the channel weight and spatial weight respectively. Channel attention is implemented by global average pooling and maximum pooling followed by two layers of fully connected networks, while spatial attention is implemented by convolutional layers and sigmoid activation functions. The two attention weights are multiplied with the feature map to obtain a weighted feature map. Then, all weighted feature maps are upsampled to the same spatial resolution using a deconvolution layer and fused by element-wise addition. Finally, a 1x1 convolution layer is applied to adjust the number of channels to obtain a preliminary super-resolution image. Next, an edge-aware detail enhancement network is used for processing. This network adopts a residual learning structure and contains 20 residual blocks, each of which contains two 3x3 convolutional layers and a LeakyReLU activation function. An edge detection branch is introduced in the middle layer of the network, and the Sobel operator is used to extract edge information and merge it with the backbone feature map. The last layer uses the Tanh activation function to ensure that the output value is in the range of [-1, 1]. This network can further enhance the edge and texture details of the image and ultimately output a high-quality high-resolution texture image.

[0035] In one embodiment of the present invention, the target texture rendering is subjected to cutout processing to obtain a cutout bitmap, including: generating a rendering of a model to be mapped with the same shape and perspective according to the shape and perspective of the high-resolution texture image; performing pixel-by-pixel difference calculation on the high-resolution texture image and the rendering of the model to be mapped, taking the absolute value of the difference result and summing multiple color channels to obtain a difference matrix; binarizing the difference matrix according to a preset threshold, retaining pixels greater than the threshold and setting pixels less than the threshold to transparent to obtain a preliminary cutout bitmap; applying the Sobel operator to the preliminary cutout bitmap for edge detection, removing abnormal edges, and obtaining a cutout bitmap.

[0036] Specifically, first, according to the shape and perspective of the high-resolution texture image, a rendering of the model to be textured with the same shape and perspective is generated. This step is implemented using the OpenGL rendering engine. In the specific process, the metadata of the high-resolution texture image is first read to extract its resolution and camera parameters. Then, the pre-prepared 3D building block model is loaded into the OpenGL environment. The virtual camera parameters are set to match the perspective of the texture image, including the camera position, orientation, and field of view. To ensure the quality of the rendering, multi-sampling anti-aliasing and anisotropic filtering are enabled. Parallel light sources are used to simulate natural lighting, and ambient occlusion is applied to enhance the sense of depth. No texture is applied during the rendering process, and only the geometric information of the model is retained. The purpose of this is to generate a rendering that completely corresponds to the original texture image in terms of spatial structure, but only contains geometric information. This rendering will serve as a reference for the subsequent cutout process to help identify which areas belong to the model itself and which areas are content added by the texture. In this way, the texture content can be accurately separated from the geometric shape of the model, laying the foundation for subsequent cutout processing.

[0037] Next, the pixel-by-pixel difference calculation is performed on the high-resolution texture image and the rendering of the model to be textured. First, the two images are converted to Lab color space because Lab space is more consistent with human eye perception. Then, the pixel difference of the two images is calculated, and the absolute value of the difference result is taken to eliminate the influence of positive and negative differences. Subsequently, the three channels of Lab space are weighted and summed, and the three channels are combined into a scalar value representing the overall degree of difference at each pixel position. The weighted sum is used here instead of a simple sum because the human eye has different sensitivities to brightness and different colors. Usually, a higher weight is given to the L channel because the human eye is more sensitive to brightness changes. The final difference matrix has the same spatial dimensions as the input image, but only one channel. This difference matrix clearly shows the difference between the original texture image and the pure geometric model rendering, effectively highlighting the content added by the texture. In this way, the modification and addition of the texture to the original model can be accurately identified, providing a key information basis for the subsequent cutout process.

[0038] Then, the difference matrix is ​​binarized according to the preset threshold. The selection of the threshold is adaptively determined by Otsu's method. The core idea of ​​Otsu's method is to find the optimal threshold by maximizing the inter-class variance. The specific implementation process includes calculating the histogram of the difference matrix to obtain the distribution of the number of pixels in the grayscale level. Then the total number of pixels, the cumulative sum array and the cumulative mean array are calculated. Next, all possible thresholds are traversed, and for each threshold, the image is divided into foreground and background parts, and the inter-class variance is calculated. The threshold that maximizes the inter-class variance is selected as the final binarization threshold. The advantage of this method is that it can automatically select the optimal threshold according to the actual grayscale distribution of the image without manual intervention, and is suitable for images with different lighting conditions and contents. The difference matrix is ​​binarized using the determined threshold, and pixels greater than the threshold are retained (set to 1), and pixels less than the threshold are set to transparent (set to 0), and a preliminary cutout bitmap is obtained. This preliminary cutout bitmap clearly distinguishes the map area that needs to be retained and the background area that needs to be removed.

[0039] Finally, the Sobel operator is applied to the preliminary cutout bitmap for edge detection, and the abnormal edges are removed to obtain the cutout bitmap. The Sobel operator includes two 3x3 convolution kernels, which are used to detect edges in the horizontal and vertical directions respectively. By performing a convolution operation on the preliminary cutout bitmap and calculating the gradient amplitude, significant edges can be identified. A gradient threshold is set, and edge pixels with a gradient amplitude less than the threshold are set to transparent in the original mask, thereby removing abnormal edges and obtaining the final cutout bitmap. This process can effectively eliminate some noise and discontinuous edges caused by thresholding, making the cutout result smoother and more accurate.

[0040] 103. Use the cutout bitmap as a projection map, and perform projection mapping on the projection map according to a preset block structure of the assembled building block model, so as to realize automatic mapping of the assembled building block model and obtain an assembled building block model with mapping completed.

[0041] In one embodiment of the present invention, the cutout bitmap is used as a projection map, and the projection map is projected and mapped according to a preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model, and obtaining the assembled building block model after mapping includes: using the cutout bitmap as a projection map, and detecting whether there is an adjustment instruction for the cutout bitmap used as the projection map, if so, converting and adjusting the cutout bitmap according to the adjustment instruction to obtain a corresponding map vector map, and updating the projection map according to the map vector map; according to the preset block structure of the assembled building block model, The method comprises the following steps: first, performing a segmentation process on the projection map to obtain a sub-projection map set corresponding to each building block of the segmented structure; second, performing a geometric transformation and projection calculation on each sub-projection map in the sub-projection map set to obtain a projection map matching the surface of the corresponding building block; third, performing UV coordinate mapping on the projection map according to the three-dimensional geometric information of the building block to obtain map coordinate information, and then associating the map coordinate information with the three-dimensional model of the corresponding building block to obtain a building block model set with map information; fourth, performing an assembly and rendering process on the building block model set with map information to obtain an assembled building block model with map completed.

[0042] Specifically, first, the cutout bitmap is used as a projection map, and whether there is an adjustment instruction for the cutout bitmap as a projection map. The purpose of this step is to provide opportunities for manual intervention and adjustment, and increase the flexibility of the generation process. The process of detecting adjustment instructions involves the design of the user interface and the implementation of interactive logic. The system will continue to monitor user input, including interactive behaviors such as mouse clicks, keyboard operations, or touch screens. When relevant adjustment instructions are detected, the system will trigger the corresponding processing flow. If there is an adjustment instruction, the cutout bitmap is converted and adjusted according to the instruction. Vector map conversion usually uses contour tracking algorithms, such as the Moore-Neighbor algorithm, to extract the edge contour of the bitmap, and then use the Douglas-Peucker algorithm to simplify the contour. The simplified contour points are fitted by Bezier curves to generate a smooth vector path. The adjustment process may include path editing, color modification, adding or deleting elements, etc., all of which require the support of specialized vector map editing tools. After the adjustment is completed, the system will regenerate the bitmap based on the modified vector map and update the projection map. This process not only allows designers to make fine adjustments, but also ensures the continuity and consistency of subsequent processing.

[0043] Specifically, according to the preset block structure of the assembled building block model, the map projection map is segmented to obtain a sub-projection map set corresponding to each building block of the block structure. The implementation of this step involves complex geometric calculations and space partitioning algorithms. Specifically, it is necessary to first parse the block structure information of the assembled building block model, which usually includes the position, size, shape and relative relationship of each building block. Then, these three-dimensional spatial information are projected onto a two-dimensional plane to form a plane segmentation scheme corresponding to the map projection map. Next, the map projection map is segmented using a computational geometry algorithm, such as a plane segmentation tree or a region growing algorithm. During the segmentation process, it is necessary to consider the connection relationship and overlap between the building blocks to ensure that the segmentation result can accurately reflect the structure of the three-dimensional model. For complex curved surfaces or special-shaped building blocks, a more advanced surface segmentation algorithm may be required. After the segmentation is completed, each sub-projection map corresponds to a specific building block, retaining some information of the original map. The key to this step is to maintain the accuracy and integrity of the segmentation to ensure that the subsequent projection mapping can accurately correspond to each building block.

[0044] Specifically, each sub-projection map in the sub-projection map set is geometrically transformed and projected to obtain a projection map that matches the surface of the corresponding building block. This process involves a complex three-dimensional to two-dimensional projection transformation. First, the three-dimensional geometric information of each building block needs to be obtained, including its position, direction, and surface normal vector in the entire model. Then, a series of geometric transformations, including rotation, scaling, and translation, are applied to each sub-projection map to make it consistent with the surface shape and direction of the corresponding building block. This step is usually implemented using an affine transformation matrix. Next, a projection calculation is performed to project the transformed two-dimensional projection map onto the surface of the building block in three-dimensional space. This projection process needs to consider factors such as viewing angle and lighting to ensure that the projected map visually matches the surface of the building block perfectly. For curved or irregularly shaped building blocks, more complex nonlinear projection methods such as spherical projection or cylindrical projection may need to be used. Throughout the process, special attention should be paid to maintaining the continuity of the map, especially at the edges and corners of the building blocks, to avoid obvious breaks or distortions. The result of this step is a set of projection maps that exactly match the surfaces of the individual building blocks, ready for subsequent UV coordinate mapping.

[0045] Specifically, according to the three-dimensional geometric information of the building block, the projection map is UV-coordinate mapped to obtain the map coordinate information, and the map coordinate information is associated with the three-dimensional model of the corresponding building block to obtain a set of building block models with map information. UV coordinate mapping is the process of unfolding the surface of a three-dimensional model to a two-dimensional plane. This step is crucial to ensure that the map is correctly applied to the three-dimensional model. First, a UV unfolding map needs to be created for each building block. This process usually involves complex surface parameterization algorithms, such as the minimum stretch energy method or the spectral method. For building blocks with regular shapes, simple planar projection can be used, while for complex shapes, more advanced methods such as spherical projection or cylindrical projection may be required. Once the UV unfolding map is created, the pixels of the projection map need to be mapped into this UV space. This process needs to take into account the resolution of the map and the surface area of ​​the building block to ensure that the map does not appear stretched or compressed. During the mapping process, the edges of the map also need to be processed to ensure that the map can smoothly transition at the seams of the building blocks. After the UV coordinate mapping is completed, the obtained map coordinate information is associated with the three-dimensional model of the building block. This is usually achieved by adding UV coordinates to the vertex data of the 3D model. Ultimately, each building block becomes a 3D model with accurate texture information, forming a complete set of building block models with texture information.

[0046] Specifically, the set of building block models with texture information is assembled and rendered to obtain the assembled building block model with texture information. This step involves the construction of a three-dimensional scene and high-quality rendering. First, according to the preset assembly structure, each building block model with texture information is placed in the correct position and direction. This process requires accurate spatial transformation calculations to ensure that each building block can be perfectly spliced ​​together. During the assembly process, the contact surfaces between the building blocks need to be processed to ensure that there are no visual gaps or overlaps. Next, the lighting environment of the scene is set, including ambient light, direct light, and indirect light to create realistic light and shadow effects. Ambient occlusion may also need to be added to enhance the three-dimensional sense of the model. Then, advanced rendering techniques such as physically based rendering (PBR) are applied to consider the reflectivity, roughness and other properties of the material to make the texture and building block material look more realistic. For transparent or translucent building blocks, the refraction and scattering of light also need to be handled correctly. Finally, use high-quality anti-aliasing technology, such as MSAA or FXAA, to render the entire scene to obtain the final high-quality image or interactive 3D model. This final assembly building block model not only contains accurate geometric information, but also incorporates highly customized texture design to show a unique visual effect.

[0047] Furthermore, the step of performing geometric transformation and projection calculation on each sub-projection map in the sub-projection map set to obtain a projection map that matches the surface of the corresponding building block includes: performing edge detection and contour analysis on each sub-projection map in the sub-projection map set to obtain a control point set representing the main contour of the sub-projection map; calculating the normal vector and main direction of the building block surface based on the three-dimensional geometric information of the corresponding building block to obtain a parametric description of the surface; constructing a projection mapping function using the control point set and the parametric description of the surface to calculate the mapping relationship from the sub-projection map to the building block surface; and performing geometric transformation and projection calculation on the sub-projection map based on the mapping relationship to obtain a projection map that matches the surface of the corresponding building block.

[0048] Specifically, first, edge detection and contour analysis are performed on each sub-projection map in the sub-projection map set to obtain a set of control points representing the main contours of the sub-projection map. The implementation of this step involves complex image processing and geometric analysis techniques. Specifically, the projection map first needs to be converted into a high-resolution bitmap format so that traditional image processing algorithms can be applied. Then, the Canny edge detection operator is used to detect the edge of the image. The Canny operator first performs Gaussian blur on the image to reduce noise, then calculates the gradient amplitude and direction of the image, then performs non-maximum suppression to refine the edge, and finally obtains the final edge map through the double threshold method and edge connection. After obtaining the edge map, a contour tracking algorithm, such as the Moore-Neighbor algorithm, is used to extract continuous contour lines. In order to reduce the amount of data and retain key features, the extracted contour is approximated by polygons, which is usually implemented using the Douglas-Peucker algorithm. The algorithm simplifies the contour by recursively splitting the curve into line segments and removing points with small deviations. Finally, the simplified contour is curve fitted, usually using cubic spline interpolation, to obtain a set of control points that can accurately represent the main contours of the sub-projection map. These control points not only retain the key features of the image, but also greatly reduce the amount of data, providing efficient input for subsequent projection mapping.

[0049] Specifically, according to the three-dimensional geometric information of the corresponding building blocks, the normal vector and main direction of the building block surface are calculated to obtain the surface parametric description, which involves complex three-dimensional geometric calculation and surface analysis technology. First, the geometric information of a single building block needs to be extracted from the three-dimensional model data, including vertex coordinates, patch information, etc. Then, the normal vector of each patch is calculated, which is usually achieved by using the vector cross product of three non-collinear vertices on the patch. For curved blocks, the average normal vector needs to be calculated at each vertex, which can be achieved by weighted averaging the normal vectors of adjacent patches. Next, the principal component analysis (PCA) method is used to determine the main direction of the building block surface. PCA calculates the covariance matrix of the vertex coordinates and performs eigenvalue decomposition on it to obtain the main direction vectors of the surface. These main direction vectors are crucial for the subsequent parameterization process. Surface parameterization is the process of mapping a three-dimensional surface to a two-dimensional plane, usually using isometric parameterization or minimum distortion parameterization methods. For complex shapes, it may be necessary to divide the surface into multiple fragments, each of which is parameterized separately and then spliced. The parameterization process requires solving an optimization problem, the goal of which is to minimize the deformation energy while keeping the distortion of area and angle to a minimum. The resulting surface parameterization description contains the mapping relationship from three-dimensional space to two-dimensional parameter space, providing a basis for subsequent mapping projection.

[0050] Specifically, using the control point set and the surface parameterization description, constructing the projection mapping function, and calculating the mapping relationship between the sub-projection map and the building block surface involve complex mathematical modeling and optimization calculations. First, it is necessary to locate the control points of the sub-projection map in the parameterized two-dimensional space. This is usually achieved by minimizing the correspondence error between the control points in the two-dimensional space and the three-dimensional surface, and a nonlinear least squares method such as the Levenberg-Marquardt algorithm can be used. Once the position of the control points in the parameter space is determined, a mapping function from the two-dimensional parameter space to the three-dimensional surface can be constructed. This mapping function is usually represented by spline interpolation or radial basis function (RBF). For spline interpolation, bicubic splines or NURBS surfaces can be used; for RBF, commonly used basis functions include multi-quadratic functions or Gaussian functions. The process of constructing the mapping function is actually solving an interpolation problem, which requires ensuring that the mapping function matches exactly at the control points and transitions smoothly in other areas. In addition, the bijectivity of the mapping needs to be considered to ensure that there is no folding or overlap. In order to improve the quality of the mapping, additional constraints may need to be introduced, such as maintaining area ratios or minimizing distortions. The final projection mapping function should not only accurately reflect the correspondence between the sub-projection map and the surface of the building block, but also ensure the continuity and smoothness of the mapping, laying the foundation for subsequent map projection.

[0051] Specifically, according to the mapping relationship, the sub-projection map is geometrically transformed and projected to obtain a projection map that matches the surface of the corresponding building block, which involves complex graphics transformation and rendering technology. First, the sub-projection map needs to be discretized into a high-resolution bitmap for pixel-level transformation. Then, for each pixel in the bitmap, the corresponding position on the surface of the building block is calculated using the previously constructed projection mapping function. This process is actually a reverse mapping, and numerical methods such as Newton's method are required to solve nonlinear equations. For the mapped position, interpolation calculation is required to obtain the accurate color value, usually using bilinear interpolation or bicubic interpolation. When projecting, the curvature and viewing angle of the building block surface also need to be considered, and the projection result is appropriately deformed and distorted to ensure that it looks correct in three-dimensional space. In addition, occlusion and edge problems need to be handled to ensure that the map can smoothly transition at the edges and corners of the building block. In order to improve the rendering quality, anti-aliasing techniques such as super sampling anti-aliasing (SSAA) or multi-sampling anti-aliasing (MSAA) may also need to be applied. Finally, the processed texture information is combined with the geometric information of the building block to generate a complete, 3D building block model with accurate textures. This final projection map must not only visually match the surface of the building block perfectly, but also ensure that it maintains the correct appearance at different angles and lighting conditions.

[0052] In this embodiment, a source image and a model rendering to be mapped are obtained, and an initial mapping rendering is generated based on the source image and the model rendering to be mapped through a deep learning diffusion model; the initial mapping rendering is used as a target mapping rendering, and the target mapping rendering is subjected to a cutout processing to obtain a cutout bitmap; the cutout bitmap is used as a projection map, and the projection map is projected according to the preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model, and obtain a completed assembled building block model. This method realizes a fully automatic generation process from user input to the final mapping model by combining deep learning technology and image processing technology, significantly improving the efficiency and quality of building block mapping design, and solving the time-consuming and labor-intensive problem of traditional manual design methods.

[0053] The above describes the method for generating the assembled building block model in the embodiment of the present invention. The following describes the device for generating the assembled building block model in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a device for generating an assembled building block model includes: An image diffusion module 201 is used to obtain a source image and a model rendering image to be mapped, and generate an initial mapping rendering image according to the source image and the model rendering image to be mapped by using a deep learning diffusion model; A cutout module 202 is used to use the initial texture rendering as a target texture rendering and perform a cutout process on the target texture rendering to obtain a cutout bitmap; The mapping projection module 203 is used to use the cutout bitmap as a projection map and perform projection mapping on the projection map according to a preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model and obtain an assembled building block model with mapping completed.

[0054] In an embodiment of the present invention, the device for generating the assembled building block model runs the above-mentioned method for generating the assembled building block model, and the device for generating the assembled building block model obtains a source image and a model rendering to be mapped, and generates an initial mapping rendering according to the source image and the model rendering to be mapped through a deep learning diffusion model; uses the initial mapping rendering as a target mapping rendering, and performs a cutout process on the target mapping rendering to obtain a cutout bitmap; uses the cutout bitmap as a projection map, and performs projection mapping on the projection map according to a preset block structure of the assembled building block model, so as to realize automatic mapping of the assembled building block model and obtain an assembled building block model with mapping completed. This method realizes a fully automatic generation process from user input to the final mapping model by combining deep learning technology and image processing technology, significantly improves the efficiency and quality of building block mapping design, and solves the problem of time-consuming and labor-intensive traditional manual design methods.

[0055] above Figure 2 The generating device of the assembled building block model in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The generating device of the assembled building block model in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0056] Figure 33 is a schematic diagram of the structure of a device for generating a building block model provided by an embodiment of the present invention. The device 300 for generating a building block model may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the device 300 for generating the building block model. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the device 300 for generating the building block model to implement the steps of the above-mentioned method for generating the building block model.

[0057] The device 300 for generating the building block model may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The structure of the device for generating the assembled building block model shown does not constitute a limitation on the device for generating the assembled building block model provided by the present invention, and may include more or less components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0058] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the method for generating an assembled building block model.

[0059] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0060] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0061] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating an assembled building block model, characterized in that: The method for generating the assembled building block model comprises: Obtain a source image and a model rendering of a to-be-mapped image, and generate an initial mapping rendering according to the source image and the model rendering of the to-be-mapped image through a deep learning diffusion model; Using the initial texture rendering as a target texture rendering, and performing a cutout process on the target texture rendering to obtain a cutout bitmap; The cutout bitmap is used as a projection map, and projection mapping is performed on the projection map according to a preset block structure of the assembled building block model, so as to realize automatic mapping of the assembled building block model and obtain the assembled building block model with the mapping completed.

2. The method for generating an assembled building block model according to claim 1, characterized in that: The step of obtaining a source image and a model rendering image to be mapped, and generating an initial mapping rendering image according to the source image and the model rendering image to be mapped by using a deep learning diffusion model includes: Obtaining a source image and a model rendering to be mapped, and performing multi-scale feature extraction on the source image to obtain a feature map set containing multi-scale visual features; According to the geometric structure of the model rendering to be mapped, the visual features in the feature map set are adaptively spatially transformed to obtain a transformed feature map set aligned with the model rendering; Combining the transformed feature map set and the model rendering map as conditional input, and inputting them into a pre-trained deep learning diffusion model to obtain an initial noise map; An iterative denoising process is performed on the initial noise image, and a denoised image is generated or updated in each iteration until a preset number of iterations is reached, and the denoised image corresponding to the number of iterations is used as the initial texture rendering image.

3. The method for generating an assembled building block model according to claim 2, characterized in that: The step of combining the transformed feature map set and the model rendering map as conditional input and inputting the transformed feature map into the pre-trained deep learning diffusion model to obtain the initial noise map includes: Combining the transformed feature map set and the model rendering map as conditional input, and inputting the conditional input into a pre-trained deep learning diffusion model, wherein the deep learning diffusion model includes a conditional embedding module and a U-Net denoising network; Encoding the transformed feature map set and the model rendering map using a conditional embedding module to obtain a conditional embedding vector, and generating a random noise map having a size consistent with the model rendering map according to a preset initial noise distribution; The conditional embedding vector and the random noise map are input into a U-Net denoising network to obtain predicted noise, and the random noise map is sampled and updated according to the predicted noise and a preset noise schedule to obtain an initial noise map.

4. The method for generating an assembled building block model according to claim 1, characterized in that: After the initial texture rendering is used as the target texture rendering, the method further includes: Determining whether the initial texture rendering as the target texture rendering is a high-resolution rendering; If not, performing multi-scale decomposition on the initial texture rendering to obtain a set of sub-images with different frequency components, and performing up-sampling processing on each sub-image in the set of sub-images according to a preset super-resolution factor to obtain an amplified set of sub-images; Inputting the enlarged sub-image set into a pre-trained super-resolution diffusion model, performing iterative denoising processing on each enlarged sub-image to obtain a sub-image set with enhanced high-frequency details; Adaptively fusing the high-frequency detail-enhanced sub-image set to obtain a preliminary super-resolution image, and processing the preliminary super-resolution image using an edge-aware detail enhancement network to obtain a high-resolution texture image; The target texture rendering is updated according to the high-resolution texture image.

5. The method for generating an assembled building block model according to claim 4, characterized in that: The performing a cutout process on the target texture rendering to obtain a cutout bitmap comprises: Generate a rendering of the model to be mapped with the same shape and perspective according to the shape and perspective of the high-resolution mapping image; Perform pixel-by-pixel difference calculation on the high-resolution texture image and the model rendering image to be textured, and sum the difference results on multiple color channels after taking absolute values ​​to obtain a difference matrix; Binarization is performed on the difference matrix according to a preset threshold value, pixels greater than the threshold value are retained, and pixels less than the threshold value are set to be transparent, so as to obtain a preliminary cutout bitmap; The Sobel operator is applied to the preliminary cutout bitmap for edge detection, and abnormal edges are removed to obtain a cutout bitmap.

6. The method for generating an assembled building block model according to claim 1, characterized in that: The method of using the cutout bitmap as a projection map and performing projection mapping on the projection map according to a preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model to obtain the assembled building block model with the mapping completed comprises: The cutout bitmap is used as a projection map, and whether there is an adjustment instruction for the cutout bitmap used as the projection map is detected. If there is, the cutout bitmap is converted and adjusted according to the adjustment instruction to obtain a corresponding map vector map, and the projection map is updated according to the map vector map; According to the block structure of the preset assembled building block model, the projection map is segmented to obtain a sub-projection map set corresponding to each building block of the block structure; Performing geometric transformation and projection calculation on each sub-projection map in the sub-projection map set to obtain a projection map that matches the surface of the corresponding building block; According to the three-dimensional geometric information of the building block, UV coordinate mapping is performed on the projection map to obtain mapping coordinate information, and the mapping coordinate information is associated with the three-dimensional model of the corresponding building block to obtain a building block model set with mapping information; The building block model set with the texture information is assembled and rendered to obtain an assembled building block model with the texture completed.

7. The method for generating an assembled building block model according to claim 6, characterized in that: The step of performing geometric transformation and projection calculation on each sub-projection map in the sub-projection map set to obtain a projection map matching the surface of the corresponding building block comprises: Performing edge detection and contour analysis on each sub-projection map in the sub-projection map set to obtain a control point set representing a main contour of the sub-projection map; According to the three-dimensional geometric information of the corresponding building block, the normal vector and main direction of the building block surface are calculated to obtain the surface parameterized description; Using the control point set and the surface parameterization description, a projection mapping function is constructed to calculate the mapping relationship between the sub-projection map and the surface of the building block; According to the mapping relationship, geometric transformation and projection calculation are performed on the sub-projection map to obtain a projection map that matches the surface of the corresponding building block.

8. A device for generating an assembled building block model, characterized in that: The generating device of the assembled building block model comprises: An image diffusion module is used to obtain a source image and a model rendering to be mapped, and generate an initial mapping rendering according to the source image and the model rendering to be mapped through a deep learning diffusion model; A cutout module, used to use the initial texture rendering as a target texture rendering, and perform a cutout process on the target texture rendering to obtain a cutout bitmap; The mapping projection module is used to use the cutout bitmap as a projection map and perform projection mapping on the projection map according to a preset block structure of the assembled building block model to achieve automatic mapping of the assembled building block model and obtain an assembled building block model with mapping completed.

9. A device for generating an assembled building block model, characterized in that: The device for generating the assembled building block model comprises: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the device for generating an assembled building block model to execute the steps of the method for generating an assembled building block model as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the method for generating an assembled building block model as described in any one of claims 1 to 7 are implemented.

Images