A decoration real scene effect picture generation method based on style transfer

Abstract

The present application relates to the technical field of decoration design, and more particularly to a decoration real scene effect picture generation method based on style transfer. The method obtains an indoor space original picture to be decorated and a decoration style reference picture, performs scene analysis on the indoor space original picture to generate a target semantic segmentation picture and a target depth estimation picture; extracts global visual features and structural factors of the decoration style reference picture, and local content features of each decoration semantic region in the target semantic segmentation picture, calculates a correlation weight matrix based on the structural factors, and weightedly fuses to generate regionalized style modulation parameters corresponding to each decoration semantic region; taking the style adaptability of each semantic region and the overall scene visual consistency as the target, generates an initial real scene effect picture through a conditional generative adversarial network, and then combines the structural factors, the target depth estimation picture and the original picture lighting characteristics to optimize rendering to obtain a target real scene effect picture. The method realizes accurate adaptation of style and scene, and improves the coordination and realism of the effect picture.

Description

Technical Field

[0001] This invention relates to the field of interior design technology, and in particular to a method for generating interior design renderings based on style transfer. Background Technology

[0002] Promoting intelligent and personalized interior design is a crucial direction for improving efficiency and user experience in the home furnishing industry. The automatic generation of interior design renderings based on style transfer has become a research hotspot in the intersection of computer vision and interior design in recent years. However, this technology still faces the dual challenges of realistic effects and design rationality in practical applications. For example, existing methods often rely on end-to-end image style transfer models, which frequently result in style misuse, structural distortion, or local misalignment in complex interior scenes, affecting the practical design reference value.

[0003] Currently, various image synthesis methods based on generative adversarial networks and attention mechanisms have been developed for interior design style transfer. However, in practical deployments, the generation effects and scene adaptability of existing methods still have significant limitations. Most methods use uniform style features for global transfer, ignoring the semantic differences and spatial constraints of different functional areas within the interior space. Furthermore, existing methods often fail to fully consider the geometric consistency of the interior space and the lighting characteristics of the original image, resulting in generated renderings that lack visual realism and spatial rationality, making them difficult to use directly for interior design decisions. When dealing with structurally complex scenes, fixed transfer patterns can easily lead to a mismatch between style and regional function; while when style features are strong, they may obscure the structural information of the original scene, causing distortions in spatial perspective or lighting logic. Summary of the Invention

[0004] This invention provides a method for generating interior decoration renderings based on style transfer, which solves the technical problem that existing methods ignore scene semantic structure and spatial constraints during style transfer, resulting in overall visual inconsistency in the generated renderings.

[0005] The first aspect of this invention provides a method for generating interior decoration renderings based on style transfer, comprising:

[0006] Obtain the original image of the interior space to be renovated and the reference image of the renovation style; perform scene analysis on the original image of the interior space to generate a target semantic segmentation map and a target depth estimation map;

[0007] Extract the global visual features and structural factors of the decoration style reference image, and extract the local content features of each decoration semantic region in the target semantic segmentation image;

[0008] Based on the structural factors, calculate the correlation weight matrix between the global visual features and each of the local content features;

[0009] Based on the relevance weight matrix, the global visual features are weighted and fused to generate a set of regionalized style modulation parameters that correspond one-to-one with the decoration semantic region.

[0010] Based on the original indoor space image, the target semantic segmentation image, the target depth estimation image, and the regional style modulation parameters, with the style adaptability of each decoration semantic region and the visual consistency of the overall scene as the collaborative optimization objective, forward inference and feature modulation are performed through a conditional generative adversarial network to generate an initial real-scene effect image.

[0011] Based on the structural factors, the target depth estimation map, and the lighting characteristics of the original indoor space map, the initial real-scene rendering is optimized for geometric and lighting consistency to generate the target real-scene rendering.

[0012] Optionally, the step of performing scene parsing on the original indoor space image to generate a target semantic segmentation map and a target depth estimation map includes:

[0013] The original indoor space image is fused with a feature pyramid through multi-scale convolution to generate an enhanced feature set.

[0014] The enhanced feature set is then subjected to pixel-by-pixel classification and pixel-by-pixel regression to generate an initial semantic segmentation map and an initial depth estimation map.

[0015] Extract the object edge geometric gradient map corresponding to the initial depth estimation map, and use the object edge geometric gradient map to perform boundary-guided optimization on the category probability distribution of the corresponding pixels in the initial semantic segmentation map to generate an intermediate semantic segmentation map;

[0016] Extract connected regions belonging to the same object from the intermediate semantic segmentation map, and use the connected regions to perform spatial smoothing optimization on the depth values ​​in the corresponding regions of the initial depth estimation map to generate an intermediate depth estimation map;

[0017] Based on preset indoor scene physical rules, the intermediate semantic segmentation map and the intermediate depth estimation map are logically consistent and a target semantic segmentation map and a target depth estimation map are generated.

[0018] Optionally, the step of extracting the global visual features and structural factors of the decoration style reference image, and extracting the local content features of each decoration semantic region in the target semantic segmentation image, includes:

[0019] The decoration style reference image is decoupled and encoded to separate style factors related to color and texture and structural factors related to spatial layout;

[0020] The style factors are aggregated into global visual features;

[0021] Based on the target semantic segmentation map, the mask of each decoration semantic region is extracted from the original indoor space image to generate a mask set;

[0022] The original indoor space image is focused using the mask set to generate multiple region feature maps;

[0023] Each region feature map is subjected to feature alignment extraction at the same spatial level as the style decoupling encoding to generate local content features corresponding to the decoration semantic region.

[0024] Optionally, the step of calculating the correlation weight matrix between the global visual features and each of the local content features based on the structural factor includes:

[0025] Based on the spatial location information corresponding to the structural factors and the local content features, a spatial soft mask is constructed.

[0026] The spatial soft mask is multiplied element-wise with the global visual features to generate candidate style features;

[0027] Calculate the similarity between each feature component in the local content feature and the candidate style feature, and arrange the obtained similarity values ​​according to the spatial dimension or channel dimension of the candidate style feature to obtain the initial attention weight matrix;

[0028] The enhanced attention weight matrix is ​​constructed by tensor concatenation and joint encoding of the initial attention weight matrix, the geometric continuity information of the corresponding region in the target depth estimation map, and the category encoding of the corresponding region in the target semantic segmentation map.

[0029] The enhanced attention weight matrix is ​​compressed and normalized in terms of channel dimension to generate a relevance weight matrix.

[0030] Optionally, the step of weightedly fusing the global visual features according to the relevance weight matrix to generate a set of regionalized style modulation parameters corresponding one-to-one with the decoration semantic regions includes:

[0031] The global visual features are dynamically weighted using the relevance weights corresponding to the decoration semantic regions to generate the initial style representation vectors corresponding to the decoration semantic regions.

[0032] Based on the local content features of the decoration semantic region, the initial style representation vector is conditionally adjusted and decoupled to generate a subset of affine transformation parameters and a subset of convolution modulation parameters.

[0033] The subset of affine transformation parameters and the subset of convolution modulation parameters are combined to form the regionalized style modulation parameters corresponding to the decoration semantic region.

[0034] Optionally, the step of generating an initial real-world rendering image based on the original indoor space image, the target semantic segmentation image, the target depth estimation image, and the regionalized style modulation parameters, with the style adaptability of each of the decoration semantic regions and the visual consistency of the overall scene as the collaborative optimization objective, and through conditional generative adversarial networks for forward inference and feature modulation, includes:

[0035] The original indoor space image, the target semantic segmentation image, and the target depth estimation image are concatenated along the channel dimension to generate a multi-source conditional feature map.

[0036] The multi-scale encoded feature map in the multi-source conditional feature map is extracted by the encoder of the generator in the conditional generative adversarial network;

[0037] The multi-scale encoded feature map is passed to the corresponding layer of the decoder in the generator via a skip connection;

[0038] The decoder features of each level of the decoder are superimposed with the corresponding encoder features of the same level in the multi-scale coding feature map obtained through skip connections to generate a fused feature map;

[0039] Based on the target semantic segmentation map, the fused feature map is divided into multiple semantic region feature blocks corresponding to each decoration semantic region;

[0040] Each semantic region feature block is used with the regionalized style modulation parameters corresponding to its respective semantic region feature block. Adaptive instance normalization and convolution operations are performed sequentially to output the region feature block corresponding to the semantic region feature block.

[0041] Based on the target depth estimation map, all the feature blocks of the region are weighted and integrated through a spatial attention mechanism to generate a geometrically consistent feature map;

[0042] The geometrically consistent feature map is upsampled and convolutionally rendered to generate an initial real-world rendering.

[0043] Optionally, the step of performing geometric and lighting consistency rendering optimization on the initial real-scene rendering based on the structural factor, the target depth estimation map, and the lighting characteristics of the original indoor space image to generate the target real-scene rendering includes:

[0044] Extract the luminance channel from the original indoor space image, calculate the gradient field and statistical histogram of the luminance channel, and generate a lighting characteristic map;

[0045] The target depth estimation map is registered and the spatial contour information corresponding to the structural factors is compared to generate a displacement vector field.

[0046] Based on the displacement vector field, a differentiable image distortion operation is performed on the initial real-scene rendering to correct the perspective distortion and contour misalignment of the objects in the image and generate a geometrically corrected image.

[0047] The illumination characteristic map is decomposed into a global illumination component and a high-frequency shadow component, which are then used to adjust the global hue and local brightness and darkness contrast of the geometrically corrected image, respectively, to generate the effect image.

[0048] The rendered image is post-processed with edge sharpening and noise suppression to generate a real-world rendering of the target scene.

[0049] A second aspect of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method for generating decoration scene effect images based on style transfer as described above.

[0050] The third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the method for generating interior decoration renderings based on style transfer as described above.

[0051] The fourth aspect of the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer performs the above-described method for generating interior decoration scene renderings based on style transfer.

[0052] As can be seen from the above technical solutions, the present invention has the following advantages:

[0053] This invention accurately captures the semantic structure and spatial geometric relationships of interior scenes through semantic segmentation and depth estimation, providing core spatial constraints and semantic basis for style transfer and preventing style transfer from deviating from the structural characteristics of the scene itself. By extracting structural factors and using them as a link to connect the global visual features of the reference image with the local content features of each decoration semantic region, a correlation weight matrix is ​​calculated to generate regionalized style modulation parameters, achieving accurate adaptation of style features in different semantic regions. This ensures that the style of each region matches the reference image while avoiding style fragmentation based on semantic association. The collaborative optimization objective of the conditional generative adversarial network further strengthens the balance between the style adaptability of each decoration semantic region and the overall visual consistency of the scene, avoiding local style conflicts and overall inconsistencies from the generation source. Subsequent rendering optimizations for geometric and lighting consistency, combined with structural factors, depth information, and original image lighting characteristics, compensate for the shortcomings of existing methods that ignore spatial lighting and geometric logic. This ensures that the generated renderings not only match the requirements in style but also conform to the actual structure and lighting patterns of the interior space. This guarantees a deep integration of style transfer with scene semantic structure and spatial constraints. The final generated target real-world renderings possess accurate style expression, semantic structure adaptability, and overall visual harmony, restoring the spatial texture and visual unity of real decoration scenes. This provides more valuable real-world effect presentations for decoration design and enhances the applicability of style transfer in the field of decoration design and the reliability of the generated effects. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating the steps of a method for generating interior decoration renderings based on style transfer, as provided in an embodiment of the present invention;

[0056] Figure 2 This is a structural block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0057] This invention provides a method for generating interior decoration renderings based on style transfer, which solves the technical problem that existing methods ignore scene semantic structure and spatial constraints during style transfer, resulting in overall visual inconsistency in the generated renderings.

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that in the optional embodiments of the present invention, the object information and other related data involved require the permission or consent of the object when the embodiments of the present invention are applied to specific products or technologies, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. That is to say, if the embodiments of the present invention involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations, and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0059] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a method for generating interior decoration renderings based on style transfer, as provided in an embodiment of the present invention.

[0060] This invention provides a method for generating interior decoration renderings based on style transfer, comprising:

[0061] Step 101: Obtain the original image of the interior space to be renovated and the reference image of the renovation style. Perform scene analysis on the original image of the interior space to generate a target semantic segmentation map and a target depth estimation map.

[0062] It should be noted that the original image of the interior space to be renovated refers to an RGB color image taken with a high-definition camera, containing the complete spatial form of the area to be renovated, and the shooting scene must cover the entire interior space from all angles. The decoration style reference image refers to the interior design drawing or real scene picture specified by the user that has the target decoration style. It can be a single picture or a collection of multiple pictures, and must contain color, texture, and spatial layout elements with clear style characteristics.

[0063] In practical applications, this method supports diverse input formats: spatial input images are not limited to original images of renovated interior spaces, but can also be extended to unfinished space images (actual photos of unrenovated houses), space sketches (hand-drawn spatial layout sketches by designers), and actual space images (actual photos of partially or fully renovated spaces); style reference images are not limited to interior design drawings, but can be compatible with various artworks with a clear visual style, such as oil paintings, watercolors, and photographs, as long as they possess extractable global visual features and structural factors, they can all serve as reference sources for style transfer. Through the input combination of "unfinished space image / space sketch / actual space image + reference style image (styled interior design drawing / a painting) = actual scene rendering," this method can generate stylized decoration effects for spaces at different stages and in different forms, broadening the applicability and practicality of the method.

[0064] Further, step 101 may include the following sub-steps:

[0065] S11. Perform multi-scale convolution on the original indoor space image and fuse it with the feature pyramid to generate an enhanced feature set.

[0066] In this embodiment of the invention, a ResNet-50-based backbone network is used for feature extraction. The multi-scale convolution is not an arbitrary combination of scales, but rather three core scale levels are preset, corresponding to different downsampling rates: 1 / 4 downsampling (capturing image detail features, such as wall textures and door / window edges), 1 / 8 downsampling (capturing local structural features, such as furniture outlines and spatial partitioning), and 1 / 16 downsampling (capturing global spatial features, such as the overall apartment layout). Each scale level performs feature convolution through two consecutive 3×3 convolution kernels, with a convolution stride of 2 and SAME padding (ensuring that the size of the feature map after convolution is in a fixed ratio to the input). Each convolutional layer is followed by a batch normalization (BN) layer and a ReLU activation function to suppress gradient vanishing and enhance the nonlinear expression of features.

[0067] Feature Pyramid Fusion (FPN) employs a bidirectional fusion structure of "top-down + bottom-up": the bottom-up path generates feature maps at each level through the aforementioned multi-scale convolutions; the top-down path upsamples the highest-level (1 / 16 downsampled) feature map by 2x (using bilinear interpolation) and laterally connects it with the corresponding scale (1 / 8 downsampled) feature map from the bottom-up path (adjusting the number of channels in the bottom-up feature map to 256 dimensions using a 1×1 convolution kernel, consistent with the number of channels in the top-down feature map), and then removes redundancy using a 3×3 convolution kernel to obtain the fused features; this process is repeated to fuse the 1 / 8 and 1 / 4 downsampled feature maps sequentially, ultimately outputting an enhanced feature set containing three scales. The enhanced feature set is a multi-channel feature map set (each scale feature map has 256 channels), preserving both low-level detailed features and fusing high-level semantic features.

[0068] S12. Perform pixel-by-pixel classification and pixel-by-pixel regression on the enhanced feature set to generate an initial semantic segmentation map and an initial depth estimation map.

[0069] In this embodiment of the invention, a parallel processing architecture is adopted to uniformly process the feature maps of the three scales in the enhanced feature set, ensuring that the size of the output result matches the original size of the indoor space. Figure 1 The pixel-wise classification is used to generate the initial semantic segmentation map. The specific process is as follows: The feature maps at three scales of the enhanced feature set are each passed through a 1×1 convolution kernel (the number of channels is set to the preset number of semantic categories; this scheme presets 15 categories, including walls, floors, ceilings, wooden doors, glass doors, windows, sofas, beds, dining tables, desks, wardrobes, bookshelves, coffee tables, lamps, and background walls), mapping the number of channels in each scale feature map to the number of semantic categories. Then, the feature maps corresponding to 1 / 8 and 1 / 16 downsampling are upsampled (using bilinear interpolation, magnified by 2x and 4x respectively), and then pixel-wise superimposed with the feature map corresponding to 1 / 4 downsampling (maintaining the original size) to obtain a feature map of the same size as the original indoor space image. Finally, the Softmax function is used to normalize the 15 semantic probabilities of each pixel, and the category label corresponding to the maximum probability is taken as the semantic category of that pixel, generating the initial semantic segmentation map. The initial semantic segmentation image is a single-channel 8-bit grayscale image with pixel values ​​ranging from 0 to 14 (corresponding to 15 semantic categories). It has a few issues with blurred boundaries and misclassification of categories, which need to be optimized further.

[0070] Pixel-wise regression is used to generate the initial depth estimation map. The specific process is as follows: The feature maps at three scales of the enhanced feature set are processed by using a 1×1 convolution kernel to map the number of channels in each scale feature map to a single dimension (corresponding to depth values). The 1 / 8 and 1 / 16 downsampled feature maps are upsampled (using bilinear interpolation, magnified by 2x and 4x respectively), and then superimposed with the 1 / 4 downsampled feature map to obtain a feature map of the same size. Subsequently, the initial depth prediction value for each pixel is output through the ReLU activation function (with a threshold set to 0 to filter negative depth values), generating the initial depth estimation map. The initial depth estimation map is a single-channel 32-bit floating-point grayscale image with pixel values ​​ranging from 0.5 to 10 meters (covering the depth range of common indoor spaces). It exhibits local depth value fluctuations and discontinuous depth at object edges, requiring further optimization.

[0071] S13. Extract the object edge geometric gradient map corresponding to the initial depth estimation map, and use the object edge geometric gradient map to perform boundary-guided optimization on the category probability distribution of the corresponding pixels in the initial semantic segmentation map to generate an intermediate semantic segmentation map.

[0072] In this embodiment of the invention, the extraction of the geometric gradient map of the object edge is implemented using the Sobel operator. A 3×3 Sobel convolution kernel is preset (horizontal kernel: [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], vertical kernel: [[-1, -2, -1], [0, 0, 0], [1, 2, 1]]). The horizontal and vertical gradients of the initial depth estimation map are calculated respectively to obtain the horizontal gradient map and the vertical gradient map. The two gradient maps are then subjected to a pixel-level sum-of-squares and square root operation to generate a comprehensive geometric gradient map of the object edge. The larger the pixel value in the gradient map, the higher the probability that the location is an object edge. The data format is a single-channel grayscale image of the same size as the initial depth estimation map.

[0073] The boundary-guided optimization process uses the object edge geometric gradient map, represented by the comprehensive object edge geometric gradient map, as the core constraint. It directly adjusts the category probability distribution of the initial semantic segmentation map based on this gradient map. Specifically, the comprehensive object edge geometric gradient map is first binarized (with a preset threshold of 50; pixels with values ​​≥50 are considered edge regions, and those <50 are considered non-edge regions), resulting in an edge mask. For the edge regions marked by the edge mask, the category probability distribution of the corresponding pixels in the initial semantic segmentation map is extracted. The difference in category probability between this pixel and its eight neighboring pixels is calculated, and a weighted smoothing algorithm (with a weight of 1.5 for edge regions and 1.0 for non-edge regions) is used to adjust the probability distribution, making the category probabilities of pixels on both sides of the edge more discriminative. For non-edge regions, the initial category probability distribution remains unchanged. After optimization, the semantic category of each pixel is determined again using the Softmax function, resulting in the target semantic segmentation map.

[0074] S14. Extract connected regions belonging to the same object in the target semantic segmentation map, and use the connected regions to perform spatial smoothing optimization on the depth values ​​in the corresponding regions of the initial depth estimation map to generate an intermediate depth estimation map.

[0075] In this embodiment of the invention, the connected region extraction of the same object adopts the 8-neighbor connected region labeling algorithm, based on the semantic category label of the target semantic segmentation map: traverse all pixels in the target semantic segmentation map, divide the pixels with the same semantic category label and adjacent (within the 8-neighbor range, i.e., up, down, left, right and four diagonal directions) into a connected region, assign a unique identifier to each connected region, and generate a connected region mask map.

[0076] Spatial smoothing optimization processes each connected region individually in the initial depth estimation map. The core logic is that depth values ​​within connected regions of the same object should remain continuous and smooth without significant abrupt changes. Specifically, the process involves calculating the mean μ and standard deviation σ for all pixel depth values ​​within each connected region. A preset outlier threshold of μ ± 2σ is used, replacing depth values ​​exceeding this threshold with the mean μ. Subsequently, a Gaussian filtering algorithm (preset convolution kernel size 3×3, standard deviation σ = 1.2, adaptively adjusted based on the area of ​​the connected region; σ is adjusted to 1.5 when the area is > 1000 pixels) is used to smooth the depth values ​​within the region, eliminating local fluctuations. Depth value differences are maintained at the boundaries of different connected regions (consistent with the spatial relationship of objects), and cross-region smoothing is not performed. The optimized intermediate depth estimation map has a depth value continuity error ≤ 0.05 meters, accurately reflecting the three-dimensional spatial relationship of objects within the room.

[0077] S15. Based on the preset indoor scene physical rules, perform logical consistency verification on the intermediate semantic segmentation map and the intermediate depth estimation map, and generate the target semantic segmentation map and the target depth estimation map.

[0078] In this embodiment of the invention, the preset indoor scene physical rules are rigid constraint rules based on common indoor spatial layouts and object physical characteristics. These rules are used to verify whether the category labels of the intermediate semantic segmentation map and the depth values ​​of the intermediate depth estimation map conform to real-world logic, avoiding semantic and depth contradictions. The preset indoor scene physical rules include three core judgment rules: 1. The depth values ​​of walls, floors, and ceilings are distributed in a layered and continuous manner, with the depth difference between adjacent pixels ≤ 0.1; 2. The depth value of furniture semantic regions must be greater than the depth value of the corresponding supporting surface (floor / countertop) to avoid floating; 3. Door and window semantic regions must seamlessly connect with wall semantic regions, and their depth values ​​must be consistent with the corresponding walls.

[0079] The logical consistency verification process is as follows: traverse all pixels in the intermediate semantic segmentation map and intermediate depth estimation map, and verify one by one whether the semantic category and corresponding depth value of the pixel conform to the above preset rules. Pixel regions are processed in different ways: if a region fully conforms to the preset rules, directly retain the intermediate semantic segmentation information and intermediate depth estimation information of that region as the basic valid data for the target semantic segmentation map and target depth estimation map; if a region violates the rules, first determine the error type: if it is an abnormal depth value (such as furniture floating), return to S14 to re-optimize the spatial smoothing of the region, adjust the depth value to conform to the rules, and replace the corresponding region data in the original intermediate depth estimation map with the corrected depth information as valid data for the target depth estimation map; if it is a semantic labeling error (such as mislabeling a wall as the ground), return to S13 to re-optimize the semantic category probability distribution of the region, correct the semantic labels, and replace the corresponding region data in the original intermediate semantic segmentation map with the corrected semantic information as valid data for the target semantic segmentation map.

[0080] After all pixel regions have passed the verification (including those that pass directly and those that pass after correction), the integrated semantic segmentation data and depth estimation data are subjected to a pixel-level consistency solidification process (the semantic map is solidified with category labels, and the depth map is smoothed with pixel differences) to directly generate a target semantic segmentation map and a target depth estimation map of the same size as the original indoor space image. If the verification fails, step 101 can be re-executed or the user can be prompted to check the quality of the input image.

[0081] Step 102: Extract the global visual features and structural factors of the decoration style reference image, and extract the local content features of each decoration semantic region in the target semantic segmentation image.

[0082] Furthermore, step 102 may include the following sub-steps:

[0083] S21. Decouple and encode the decoration style reference image to separate style factors related to color and texture and structural factors related to spatial layout.

[0084] In this embodiment of the invention, the feature maps of the second, third, and fourth convolutional sub-layers (pre-defined as the style decoupling core layers, corresponding to low, medium, and high spatial levels, respectively) of a pre-trained deep convolutional network are processed as follows: First, the decoration style reference image is input into the preprocessing module and normalized to a preset pixel size (keeping the aspect ratio unchanged and adding black borders at the edges). Then, it is input into the pre-trained deep convolutional network to extract the above three feature maps (each layer has 128, 256, and 512 channels, respectively). For each feature map, style factors and structural factors are separated by a channel mask: a preset texture channel mask (preserving the channels corresponding to color and texture information in the feature map; the second sub-layer of the second convolution retains the first 64 channels, the second sub-layer of the third convolution retains the first 128 channels, and the second sub-layer of the fourth convolution retains the first 256 channels) is used to filter the feature map channel by channel. Then, the Gram matrix is ​​calculated (window size 3×3, stride 1) to generate texture feature matrices for each level, which are then aggregated to obtain the style factors. Simultaneously, a pre-defined structural channel mask (preserving the channels corresponding to the spatial layout in the feature map, which is complementary to the texture channel mask) is used to filter and extract the spatial contour information in the feature map through an edge detection algorithm. Then, the contour continuity is optimized by morphological dilation to generate structural feature maps at each level. After aggregation, the structural factor is obtained.

[0085] S22. Aggregate style factors into global visual features.

[0086] In this embodiment of the invention, firstly, preset weights (0.3, 0.4, and 0.3 respectively) are assigned to the style factors of the second sub-layer of the second convolutional layer, the second sub-layer of the third convolutional layer, and the second sub-layer of the fourth convolutional layer. The lower weights at lower layers are used to weaken the influence of noise, while the higher weights at mid-to-higher layers are used to strengthen the core style features. The fused style matrix is ​​then obtained by weighted summation. Subsequently, the fused style matrix is ​​subjected to channel dimension compression (using a 1×1 convolutional kernel to reduce the number of channels from 512 dimensions to 256 dimensions). Then, it is processed by batch normalization (momentum parameter 0.9, precision parameter 1e-5) and linear rectified activation function (threshold 0) to eliminate dimensional redundancy and enhance nonlinear expression, ultimately generating global visual features.

[0087] S23. Based on the target semantic segmentation map, extract the mask of each decoration semantic region from the original indoor space map and generate a mask set.

[0088] In this embodiment of the invention, a list of semantic categories and corresponding mask identifiers are first preset (e.g., mask 1 for the wall and mask 2 for the ground, matching 15 semantic categories in sequence, corresponding one-to-one with the semantic category labels in step 101). All pixels in the target semantic segmentation image are traversed, and pixels corresponding to the same semantic category are marked as 1, while the rest are marked as 0, generating a single-channel binary mask image (with the same size as the original indoor space image of 3000×2000 pixels, and pixel values ​​of 0 or 1). Each semantic category corresponds to an independent mask image, ensuring that there is no deviation from the space size in step 101.

[0089] To avoid the impact of mask edge jaggedness on subsequent feature extraction, morphological closing operations are performed on each mask image to smooth the edge contours. Simultaneously, area filtering (with a preset minimum area threshold of 50 pixels) is used to remove minor noise regions (such as isolated pixels) caused by semantic annotation errors, consistent with the logical consistency verification objective in step 101. Finally, 15 semantic region masks are generated, forming a mask set, with the data format being a three-dimensional tensor (dimension: number of semantic categories × original image height × original image width).

[0090] S24. Use a mask set to focus attention on the original indoor space image and generate multiple regional feature maps.

[0091] In this embodiment of the invention, only valid pixels of each decoration semantic region in the original interior space image are retained, while background and other regional interference are masked. First, the original interior space image (RGB three channels, 3000×2000 pixels) is multiplied pixel by pixel with each mask image in the mask set (mask value 1 corresponds to retaining pixels from the original image, and mask value 0 corresponds to setting pixels to 0), resulting in 15 preliminary region images. Then, each preliminary region image is filtered at the pixel level, retaining valid pixels with RGB channel pixel values ​​in the range of 0-255 (removing extreme noise points), and then uniformly scaled to a preset pixel size (consistent with the preprocessed size of the style reference image, facilitating subsequent feature alignment) using a bilinear interpolation algorithm, generating multiple region feature maps. Each region feature map corresponds to a type of decoration semantic region, and the data form is a three-dimensional tensor, retaining the original color, texture, and contour information of the region.

[0092] S25. Perform feature alignment extraction on the feature maps of each region at the same spatial level as the style decoupling encoding to generate local content features corresponding to the decoration semantic region.

[0093] In this embodiment of the invention, feature alignment extraction is based on a three-layer feature layer of style decoupling encoding (second convolutional sub-layer, third convolutional sub-layer, and fourth convolutional sub-layer) to ensure spatial hierarchical matching between local content features and style factors. Each region feature map is input into a pre-trained deep convolutional network with the same architecture to extract the aforementioned three-layer feature maps (128, 256, and 512 channels respectively, with dimensions consistent with the corresponding layers of style decoupling encoding). The feature dimension design avoids conflicts with subsequent convolutional modulation parameter generation. For each feature map layer, global average pooling (with the pooling kernel size consistent with the feature map size) is used to compress the spatial dimensions, obtaining feature vectors for each layer (128 dimensions for the second convolutional sub-layer, 256 dimensions for the third convolutional sub-layer, and 512 dimensions for the fourth convolutional sub-layer). Subsequently, the three-layer feature vectors are weighted and fused with the same weights as the style factors (0.3, 0.4, and 0.3) to obtain a fused feature vector.

[0094] Finally, the number of channels in the fused feature vector is uniformly adjusted to 256 dimensions using a 1×1 convolution kernel, and then subjected to L2 normalization (normalization range 0-1) to generate local content features corresponding to a single region feature map. This process is repeated to generate local content features for each of the 15 region feature maps, forming a set of local content features.

[0095] Step 103: Based on the structural factor, calculate the correlation weight matrix between global visual features and each local content feature.

[0096] Furthermore, step 103 may include the following sub-steps:

[0097] S31. Construct a spatial soft mask based on the spatial location information corresponding to structural factors and local content features.

[0098] In this embodiment of the invention, the three-layer feature maps of the structural factors are uniformly scaled to a preset size using bilinear interpolation, and then weighted according to the layer weights (0.3, 0.4, 0.3, consistent with the feature fusion weights in step 102) to generate a fused structural map. Subsequently, based on the semantic region location corresponding to the local content features, the pixel values ​​at the corresponding positions in the fused structural map are adjusted by preset coefficients (0.8-1.2, 1.2 for edge regions and 0.8 for center regions) to strengthen the constraint effect of the edge structure. Finally, the adjusted pixel values ​​are normalized using the sigmoid function (output range 0-1) to generate a spatial soft mask. The larger the mask value, the stronger the structural constraint at that location; the data format is a two-dimensional tensor.

[0099] S32. Multiply the spatial soft mask element-wise with the global visual features to generate candidate style features.

[0100] In this embodiment of the invention, every pixel of the global visual features and spatial soft mask is traversed, and the values ​​at corresponding positions are multiplied to retain the style features of regions with strong structural constraints and weaken the style interference of regions without structural significance. The feature map obtained after multiplication is processed by a 1×1 convolution kernel to remove redundancy and generate candidate style features. The data form is still a two-dimensional tensor.

[0101] S33. Calculate the similarity between each feature component in the local content features and the candidate style features, and arrange the obtained similarity values ​​according to the spatial dimension or channel dimension of the candidate style features to obtain the initial attention weight matrix.

[0102] In this embodiment of the invention, for each local content feature, cosine similarity is calculated with each of the 256 one-dimensional feature vectors of the candidate style features, resulting in 256 similarity values ​​(range -1-1). Negative similarity values ​​are set to 0, while positively correlated features are retained. These similarity values ​​are then arranged according to the channel dimension (256 dimensions) of the candidate style features, generating a 1×256 dimensional single-row weight vector. This process is repeated to generate weight vectors for each of the 15 local content features. These weight vectors are then combined according to the semantic region order to obtain the initial attention weight matrix. The data format is a two-dimensional tensor (dimension: 15×256), with each row corresponding to the style association weight of a semantic region.

[0103] S34. Tensor concatenation and joint encoding are performed on the initial attention weight matrix, the geometric continuity information of the corresponding region in the target depth estimation map, and the category encoding of the corresponding region in the target semantic segmentation map to construct the enhanced attention weight matrix.

[0104] In this embodiment of the invention, geometric continuity information corresponding to each decoration semantic region is extracted and compressed into a 1×256-dimensional vector (consistent with the channel dimension of the initial weight matrix) through global average pooling. The semantic category encoding (1×15-dimensional) is expanded to 1×256-dimensional using a 1×1 convolutional kernel to achieve dimension alignment. Subsequently, the initial attention weight matrix (15×256), the geometric continuity vector set (15×256), and the expanded category encoding set (15×256) are concatenated according to the channel dimension to generate a 15×768-dimensional fusion tensor. Joint encoding is performed through a three-layer fully connected network (with 512, 256, and 256 neurons respectively) to strengthen the association of multi-dimensional information, and finally generate an enhanced attention weight matrix. The data form is a two-dimensional tensor (dimension: 15×256), with each row corresponding to the fusion association weight of a semantic region.

[0105] S35. Perform channel dimension compression and normalization on the enhanced attention weight matrix to generate a relevance weight matrix.

[0106] In this embodiment of the invention, the enhanced attention weight matrix (15×256) is compressed using a 1×1 convolutional kernel while maintaining the same number of channels, eliminating redundant features to generate a compressed matrix. Then, an L2 normalization algorithm (consistent with the local content feature normalization in step 102) is used to normalize the weight values ​​in each row, ensuring the sum of the weight values ​​in each row is 1, thus strengthening the relative proportion of the weights. Finally, a threshold filtering process (a preset threshold of 0.01 is used to remove weights below this value to avoid weak association interference) generates a relevance weight matrix. The data is in the form of a two-dimensional tensor (dimension: 15×256), with each row corresponding to a precise association weight between a semantic region and the global style.

[0107] Step 104: Based on the relevance weight matrix, the global visual features are weighted and fused to generate a set of regional style modulation parameters that correspond one-to-one with the semantic regions of decoration.

[0108] Furthermore, step 104 may include the following sub-steps:

[0109] S41. Dynamically weight the global visual features using the relevance weights corresponding to the decoration semantic regions to generate the initial style representation vectors corresponding to the decoration semantic regions.

[0110] In this embodiment of the invention, for each of the 15 categories of semantic regions related to decoration, the relevance weights (1×256 dimensions) of the corresponding rows are extracted one by one. These weights are then element-wise weighted with the global visual features, that is, the weight vector is multiplied element-wise with each column of the global visual features, strengthening high-weight style components and weakening low-weight components. After weighting, the spatial dimension is compressed through global average pooling (pooling kernel size 256×1) to generate an initial style representation vector of 1×256 dimensions. Each semantic region corresponds to a dedicated vector, ultimately forming a set of 15 initial style representation vectors (data format: 15×256-dimensional two-dimensional tensor).

[0111] S42. Based on the local content features of the decoration semantic region, conditionally adjust and decouple the initial style representation vector to generate a subset of affine transformation parameters and a subset of convolution modulation parameters.

[0112] In this embodiment of the invention, the local content features (1×256-dimensional) of each semantic region are concatenated with the corresponding initial style representation vector (1×256-dimensional) according to the channel dimension to generate a 1×512-dimensional fusion vector. The fusion vector is conditionally adjusted through a fully connected network (256 neurons) to eliminate the adaptation conflict between style and content, and outputs a 1×256-dimensional adjusted vector. This vector is input into a dual-branch decoupled encoding module: one branch generates an affine transformation parameter subset (containing 64 mean parameters and 64 variance parameters, adapted for subsequent adaptive instance normalization operations) through a 1×1 convolution kernel (128 channels), and the other branch generates a convolution modulation parameter subset (responsible for regional refinement of feature texture) through a 3×3 convolution kernel (128 channels, padding method is SAME). Both parameter subsets are 1×128-dimensional vectors, corresponding one-to-one with the semantic regions.

[0113] S43. Combine the subset of affine transformation parameters and the subset of convolution modulation parameters to form the regionalized style modulation parameters corresponding to the semantic region of decoration.

[0114] In this embodiment of the invention, for each type of decoration semantic region, its corresponding affine transformation parameter subset (1×128-dimensional) and convolution modulation parameter subset (1×128-dimensional) are concatenated along the channel dimension to generate a 1×256-dimensional regionalized style modulation parameter. After concatenation, batch normalization (momentum parameter 0.9, precision parameter 1e-5, consistent with step 102) is applied to unify the parameter range (0-1), eliminate redundant interference, and enhance parameter stability. This process is repeated to generate exclusive regionalized style modulation parameters for each of the 15 types of decoration semantic regions, ultimately forming a parameter set (data format: 15×256-dimensional two-dimensional tensor), with each semantic region corresponding to a set of parameters.

[0115] Step 105: Based on the original indoor space image, target semantic segmentation image, target depth estimation image, and regional style modulation parameters, with the style adaptability of each decoration semantic region and the visual consistency of the overall scene as the collaborative optimization goal, forward inference and feature modulation are performed through a conditional generative adversarial network to generate an initial real-scene effect image.

[0116] In this embodiment of the invention, the conditional generative adversarial network (GAN) employs joint optimization of adversarial loss, style reconstruction loss, and content reconstruction loss during the training phase. The adversarial loss, provided by a discriminator, distinguishes generated images from real images; the style reconstruction loss is calculated based on Gram matrix differences to ensure that the generated image is similar to the style reference in texture and color. Figure 1 The content reconstruction loss is based on perceptual feature differences, ensuring that the generated image maintains the same semantic structure as the original. Figure 1 During training, gradient descent is used to optimize network parameters until the loss converges.

[0117] Furthermore, step 105 may include the following sub-steps:

[0118] S51. The original indoor space image, the target semantic segmentation image, and the target depth estimation image are spliced ​​together along the channel dimension to generate a multi-source conditional feature map.

[0119] In this embodiment of the invention, the stitching process employs a channel-dimensional overlay method: the target semantic segmentation map and the target depth estimation map are respectively calibrated using a 1×1 convolutional kernel (maintaining a 1-dimensional channel count) to ensure pixel accuracy alignment with the original indoor space image (edge ​​error ≤ 1 pixel). Then, the images are stitched together in the order of "original indoor space image (3 channels) + target semantic segmentation map (1 channel) + target depth estimation map (1 channel)" to generate a 5-channel multi-source conditional feature map (data format: 5×3000×2000 dimensional tensor). After stitching, batch normalization (momentum parameter 0.9, precision parameter 1e-5, consistent with steps 102 and 104) is used to unify the feature range.

[0120] S52. Extract multi-scale encoded feature maps from multi-source conditional feature maps using the encoder of the generator in a conditional generative adversarial network.

[0121] It should be noted that the generator of the conditional generative adversarial network adopts an "encoder-decoder" architecture. The encoder is a multi-scale feature extraction module based on a deep convolutional network, with three preset core scale levels (corresponding to the style decoupling level in step 102, downsampling at 1 / 4, 1 / 8, and 1 / 16), which respectively capture details, local structure, and global layout features to ensure compatibility with the feature levels mentioned above.

[0122] In this embodiment of the invention, a multi-source conditional feature map is input into the encoder and subjected to three sets of convolutional blocks (each set containing two 3×3 convolutional kernels, a linear rectified activation function, and batch normalization) to achieve multi-scale downsampling. The convolution stride is set to 2 (to achieve downsampling), and the padding method is SAME padding (to maintain the size ratio). After downsampling at each layer, a corresponding scale encoded feature map is output, resulting in three multi-scale encoded feature maps (with 128, 256, and 512 channels, and sizes of 750×500, 375×250, and 188×125 pixels, respectively), constituting a multi-scale feature set.

[0123] In this embodiment of the invention, the encoded feature maps at scales of 1 / 4, 1 / 8, and 1 / 16 of the encoder are respectively passed to the third, second, and first levels of the decoder. Before passing them, the number of channels in the encoded feature maps is uniformly adjusted to 128, 256, and 512 dimensions (consistent with the input channels of the corresponding levels of the decoder) using a 1×1 convolution kernel to eliminate channel dimension conflicts while retaining the original feature information.

[0124] S54. The decoder features of each level of the decoder are superimposed with the corresponding encoder features of the same level in the multi-scale coding feature map obtained through skip connections to generate a fused feature map.

[0125] It should be noted that the decoder features are the feature maps of each level of the decoder after upsampling (bilinear interpolation, magnification by 2 times), which have the same size as the corresponding encoder feature maps. The superposition is performed using pixel-level addition operations to achieve the fusion of detailed features and high-level generated features.

[0126] In this embodiment of the invention, the overlay process is performed layer by layer: the first layer of the decoder (188×125 pixels, 512 channels) is overlaid with the corresponding encoder feature map, and after redundancy removal by a 3×3 convolution kernel, the fused feature is output; the second layer of the decoder (375×250 pixels, 256 channels) is overlaid with the corresponding encoder feature map, and the above convolution redundancy removal operation is repeated; the third layer of the decoder (750×500 pixels, 128 channels) is processed in the same way. Finally, three fused feature maps to be modulated are generated (the size is consistent with the corresponding layer, and the number of channels is 128, 256, and 512), which contain both the style features generated by the generator and the real-world detail features of the original image.

[0127] S55. Based on the target semantic segmentation map, the fused feature map is divided into multiple semantic region feature blocks corresponding to each decoration semantic region.

[0128] In this embodiment of the invention, the target semantic segmentation map is scaled to the size corresponding to the three fused feature maps (750×500, 375×250, and 188×125 pixels) using bilinear interpolation, preserving the semantic region contours. For each fused feature map, based on the scaled target semantic segmentation map, pixels of the corresponding region are extracted one by one according to 15 semantic categories to form semantic region feature blocks. Each semantic category corresponds to one feature block, and 15 feature blocks are generated from each of the three fused feature maps, resulting in 45 semantic region feature blocks (divided into 3 scale levels). The size of each feature block matches the pixel range of the corresponding region.

[0129] S56. Using the regionalized style modulation parameters corresponding to each decoration semantic region feature block, perform adaptive instance normalization and convolution operations in sequence, and output the region feature blocks corresponding to the semantic region feature blocks.

[0130] In this embodiment of the invention, for each semantic region feature block, the corresponding category's regional style modulation parameters are invoked. First, an adaptive instance normalization operation is performed (using 64 mean and 64 variance parameters from the parameter subset to adjust the pixel distribution of the feature block) to adapt the feature block to the target style. Then, a convolution operation is performed using a 3×3 convolution kernel (using 128-dimensional convolution modulation parameters from the parameter subset, with SAME padding) to refine the regional texture features and eliminate the feature blurring problem after normalization. After processing, each semantic region feature block outputs a corresponding region feature block, maintaining the same size as the input.

[0131] S57. Based on the target depth estimation map, the feature blocks of all regions are weighted and integrated through a spatial attention mechanism to generate a geometrically consistent feature map.

[0132] In this embodiment of the invention, the target depth estimation map is scaled to the corresponding scale, and the depth continuity information of each decoration semantic region is extracted (using the Sobel operator, 3×3 convolution kernel, consistent with step 101) to generate a depth attention mask. This mask is then multiplied element-wise with 15 region feature blocks at the corresponding scale, with the weight for depth-continuous regions set to 1.2 and the weight for discontinuous regions set to 0.8. Subsequently, the weighted feature blocks are concatenated and integrated according to the semantic region position to restore the complete feature map shape; finally, a 1×1 convolution kernel is used to unify the channel number to 384 dimensions, generating three geometrically consistent feature maps at different scales to ensure that the features of each region are consistent with the original in terms of spatial location and depth relationship. Figure 1 To.

[0133] S58. Upsample and convolutionally render the geometrically consistent feature map to generate the initial real-world rendering.

[0134] In this embodiment of the invention, three geometrically consistent feature maps are upsampled sequentially from largest to smallest scale (using bilinear interpolation, magnifying by a factor of 2 each time), gradually restoring them to 3000×2000 pixels. After upsampling, a 3×3 convolutional kernel is used for smoothing to eliminate jagged edges. Subsequently, a convolutional rendering module (containing two 3×3 convolutional kernels and one 1×1 convolutional kernel) adjusts the number of feature map channels from 384 dimensions to 3 dimensions (RGB channels), calibrating the color range to 0-255. Finally, a linear rectified activation function is used to filter invalid pixel values, generating an initial real-world effect image (3-channel RGB image, 3000×2000 pixels). This image not only matches the target decoration style but also preserves the original image's spatial layout and geometric relationships.

[0135] Step 106: Based on the structural factors, target depth estimation map, and lighting characteristics of the original indoor space image, perform geometric consistency and lighting consistency rendering optimization on the initial real-scene rendering to generate the target real-scene rendering.

[0136] Furthermore, step 106 may include the following sub-steps:

[0137] S61. Extract the brightness channel from the original indoor space image, calculate the gradient field and statistical histogram of the brightness channel, and generate a lighting characteristic map.

[0138] In this embodiment of the invention, the original indoor space image is converted to a luminance-chrominance space, and a luminance channel (single channel, pixel value range 0-255, representing the brightness of each location) is separated, maintaining the same size as the original image. Then, the Sobel operator (3×3 convolution kernel, consistent with steps 101 and 105) is used to calculate the gradient field of the luminance channel, obtaining gradient maps in both horizontal and vertical directions, quantifying the rate of brightness change (gradient value range 0-255, the larger the value, the more drastic the brightness change). Simultaneously, a statistical histogram of the luminance channel (pixel-level statistics, 256 intervals, corresponding to luminance values ​​0-255) is calculated to obtain the luminance distribution probability. The gradient field and the statistical histogram are concatenated by channel, and the dimensions are calibrated using a 1×1 convolution kernel to generate a 3-channel lighting characteristic map (data format: 3×3000×2000 dimensional tensor), completely preserving the original image's lighting characteristics.

[0139] S62. Register and compare the target depth estimation map with the corresponding spatial contour information in the structural factors to generate a displacement vector field.

[0140] In this embodiment of the invention, the spatial contour information of the structural factor is first scaled to 3000×2000 pixels using bilinear interpolation to align with the size of the target depth estimation map, and then smoothed using morphological closing operations (3×3 convolution kernel). Subsequently, a cross-correlation-based registration algorithm is employed, using the target depth estimation map as a reference, to compare the positional differences between the structural factor contour and the depth contour pixel by pixel, calculating the x and y axis offsets of each pixel (unit: pixel, offset threshold ±2 pixels; areas exceeding this threshold are considered areas requiring correction). The x and y axis offsets are combined into a two-dimensional vector to generate a displacement vector field (data format: 2×3000×2000 dimensional tensor), with each vector corresponding to the correction direction and distance of a pixel.

[0141] S63. Based on the displacement vector field, perform a differentiable image distortion operation on the initial real-scene rendering to correct the perspective distortion and contour misalignment of objects in the image and generate a geometrically corrected image.

[0142] In this embodiment of the invention, the initial real-scene rendering (3×3000×2000 dimensional) and the displacement vector field are input into the distortion module. The preset interpolation accuracy is bilinear interpolation (pixel error ≤ 0.5). The image position is adjusted pixel by pixel according to the displacement vector: for areas with perspective distortion (such as tilted walls or misaligned furniture), pixels are stretched and offset according to the vector offset; for areas with misaligned contours (such as the connection deviation between doors / windows and walls), the pixel positions are finely adjusted to achieve contour alignment. After correction, edge blanking is processed by boundary filling (using the average value of neighboring pixels) to generate a 3-channel geometrically corrected image (3000×2000 pixels) with a geometric deviation ≤ 1 pixel, ensuring consistency with the geometric relationship of the real scene.

[0143] S64. Decompose the lighting characteristic map into global illumination component and high-frequency shadow component, and use them to adjust the global tone and local brightness and darkness contrast of the geometric correction image respectively to generate the effect map.

[0144] It should be noted that the global illumination component represents the overall brightness and darkness of the room (such as the overall brightness of natural light and artificial light), while the high-frequency shadow component represents local shadow details (such as furniture shadows and corner shadows). Adjusting the two separately can achieve a natural fit between the lighting and the original image, avoiding the lighting incongruity after stylization.

[0145] In this embodiment of the invention, a Gaussian filter (15×15 kernel size, standard deviation σ=3.0) is used to decompose the illumination characteristic map. After filtering, a global illumination component is obtained (smoothing the brightness distribution and preserving the overall tone). The difference before and after filtering is the high-frequency shadow component (preserving local details). The global illumination component is multiplied pixel-by-pixel with the geometrically corrected image (brightness adjustment coefficient 0.8-1.2, adapting to different style brightness requirements) to unify the global tone. Then, the high-frequency shadow component is superimposed on the corresponding area to enhance the local contrast (shadow intensity weight 0.6, avoiding excessive shadows). After adjustment, the color range is calibrated to 0-255, generating a 3-channel effect image (3000×2000 pixels), with the illumination distribution similar to the original. Figure 1 To.

[0146] S65. Perform edge sharpening and noise suppression post-processing on the rendering to generate a real-world rendering of the target.

[0147] It should be noted that the purpose of post-processing is to optimize image quality, sharpen edges to enhance the clarity of object outlines, and suppress noise to eliminate slight noise introduced by previous operations, ensuring that the target image has both realism and detail, and meets the needs of presenting a real-life decoration scene.

[0148] In this embodiment of the invention, the Laplacian operator (3×3 convolution kernel, sharpening intensity coefficient 1.5) is used for edge sharpening to enhance the outline edges of objects such as doors, windows, and furniture, thereby improving the sense of depth in the image. Wavelet transform filtering is used for noise suppression to reduce high-frequency random noise (such as graininess introduced by lighting adjustments) while preserving edge details and avoiding image blurring. Finally, color saturation is fine-tuned (with an adjustment range of ±5% to match the color texture of the original image) to generate a 3-channel target real-world image (3000×2000 pixels, RGB format).

[0149] Please see Figure 2 , Figure 2 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0150] An electronic device according to an embodiment of the present invention includes: a memory 201 and a processor 202. The memory 201 stores a computer program. When the computer program is executed by the processor 202, the processor 202 executes the method for generating decoration scene effect diagrams based on style transfer as described in any of the above embodiments.

[0151] Memory 201 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 201 has storage space 203 for program code 213 for performing any of the method steps described above. For example, storage space 203 for program code may include individual program codes 213 for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When run by a computing processing device, this code causes the computing processing device to perform the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When this code is run by a computing device, it causes the device to perform the various steps in the style transfer-based method for generating realistic interior design renderings described above.

[0152] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for generating interior decoration renderings based on style transfer as described in any of the above embodiments.

[0153] This invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the method for generating decoration scene renderings based on style transfer as described in any of the above embodiments.

[0154] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating interior decoration renderings based on style transfer, characterized in that, include: Obtain the original image of the interior space to be renovated and the reference image of the renovation style; perform scene analysis on the original image of the interior space to generate a target semantic segmentation map and a target depth estimation map; Extract the global visual features and structural factors of the decoration style reference image, and extract the local content features of each decoration semantic region in the target semantic segmentation image; Based on the structural factors, calculate the correlation weight matrix between the global visual features and each of the local content features; Based on the relevance weight matrix, the global visual features are weighted and fused to generate a set of regionalized style modulation parameters that correspond one-to-one with the decoration semantic region. Based on the original indoor space image, the target semantic segmentation image, the target depth estimation image, and the regional style modulation parameters, with the style adaptability of each decoration semantic region and the visual consistency of the overall scene as the collaborative optimization objective, forward inference and feature modulation are performed through a conditional generative adversarial network to generate an initial real-scene effect image. Based on the structural factors, the target depth estimation map, and the lighting characteristics of the original indoor space map, the initial real-scene rendering is optimized for geometric and lighting consistency to generate the target real-scene rendering.

2. The method for generating interior decoration renderings based on style transfer according to claim 1, characterized in that, The steps of performing scene analysis on the original indoor space image to generate a target semantic segmentation map and a target depth estimation map include: The original indoor space image is fused with a feature pyramid through multi-scale convolution to generate an enhanced feature set. The enhanced feature set is then subjected to pixel-by-pixel classification and pixel-by-pixel regression to generate an initial semantic segmentation map and an initial depth estimation map. Extract the object edge geometric gradient map corresponding to the initial depth estimation map, and use the object edge geometric gradient map to perform boundary-guided optimization on the category probability distribution of the corresponding pixels in the initial semantic segmentation map to generate an intermediate semantic segmentation map; Extract connected regions belonging to the same object from the intermediate semantic segmentation map, and use the connected regions to perform spatial smoothing optimization on the depth values ​​in the corresponding regions of the initial depth estimation map to generate an intermediate depth estimation map; Based on preset indoor scene physical rules, the intermediate semantic segmentation map and the intermediate depth estimation map are logically consistent and a target semantic segmentation map and a target depth estimation map are generated.

3. The method for generating interior decoration renderings based on style transfer according to claim 1, characterized in that, The steps of extracting the global visual features and structural factors of the decoration style reference image and extracting the local content features of each decoration semantic region in the target semantic segmentation image include: The decoration style reference image is decoupled and encoded to separate style factors related to color and texture and structural factors related to spatial layout; The style factors are aggregated into global visual features; Based on the target semantic segmentation map, the mask of each decoration semantic region is extracted from the original indoor space image to generate a mask set; The original indoor space image is focused using the mask set to generate multiple region feature maps; Each region feature map is subjected to feature alignment extraction at the same spatial level as the style decoupling encoding to generate local content features corresponding to the decoration semantic region.

4. The method for generating interior decoration renderings based on style transfer according to claim 1, characterized in that, The step of calculating the correlation weight matrix between the global visual features and each of the local content features based on the structural factor includes: Based on the spatial location information corresponding to the structural factors and the local content features, a spatial soft mask is constructed. The spatial soft mask is multiplied element-wise with the global visual features to generate candidate style features; Calculate the similarity between each feature component in the local content feature and the candidate style feature, and arrange the obtained similarity values ​​according to the spatial dimension or channel dimension of the candidate style feature to obtain the initial attention weight matrix; The enhanced attention weight matrix is ​​constructed by tensor concatenation and joint encoding of the initial attention weight matrix, the geometric continuity information of the corresponding region in the target depth estimation map, and the category encoding of the corresponding region in the target semantic segmentation map. The enhanced attention weight matrix is ​​compressed and normalized in terms of channel dimension to generate a relevance weight matrix.

5. The method for generating interior decoration renderings based on style transfer according to claim 1, characterized in that, The step of weightedly fusing the global visual features according to the relevance weight matrix to generate a set of regionalized style modulation parameters that correspond one-to-one with the decoration semantic regions includes: The global visual features are dynamically weighted using the relevance weights corresponding to the decoration semantic regions to generate the initial style representation vectors corresponding to the decoration semantic regions. Based on the local content features of the decoration semantic region, the initial style representation vector is conditionally adjusted and decoupled to generate a subset of affine transformation parameters and a subset of convolution modulation parameters. The subset of affine transformation parameters and the subset of convolution modulation parameters are combined to form the regionalized style modulation parameters corresponding to the decoration semantic region.

6. The method for generating interior decoration renderings based on style transfer according to claim 1, characterized in that, The step of generating an initial real-world rendering image based on the original indoor space image, the target semantic segmentation image, the target depth estimation image, and the regionalized style modulation parameters, with the style adaptability of each of the decoration semantic regions and the visual consistency of the overall scene as the collaborative optimization objective, and through forward inference and feature modulation using a conditional generative adversarial network, includes: The original indoor space image, the target semantic segmentation image, and the target depth estimation image are concatenated along the channel dimension to generate a multi-source conditional feature map. The multi-scale encoded feature map in the multi-source conditional feature map is extracted by the encoder of the generator in the conditional generative adversarial network; The multi-scale encoded feature map is passed to the corresponding layer of the decoder in the generator via a skip connection; The decoder features of each level of the decoder are superimposed with the corresponding encoder features of the same level in the multi-scale coding feature map obtained through skip connections to generate a fused feature map; Based on the target semantic segmentation map, the fused feature map is divided into multiple semantic region feature blocks corresponding to each decoration semantic region; Each semantic region feature block is used with the regionalized style modulation parameters corresponding to its respective semantic region feature block. Adaptive instance normalization and convolution operations are performed sequentially to output the region feature block corresponding to the semantic region feature block. Based on the target depth estimation map, all the feature blocks of the region are weighted and integrated through a spatial attention mechanism to generate a geometrically consistent feature map; The geometrically consistent feature map is upsampled and convolutionally rendered to generate an initial real-world rendering.

7. The method for generating interior decoration renderings based on style transfer according to claim 1, characterized in that, The step of optimizing the initial real-scene rendering based on the structural factors, the target depth estimation map, and the lighting characteristics of the original indoor space image to achieve geometric and lighting consistency, and generating the target real-scene rendering, includes: Extract the luminance channel from the original indoor space image, calculate the gradient field and statistical histogram of the luminance channel, and generate a lighting characteristic map; The target depth estimation map is registered and the spatial contour information corresponding to the structural factors is compared to generate a displacement vector field. Based on the displacement vector field, a differentiable image distortion operation is performed on the initial real-scene rendering to correct the perspective distortion and contour misalignment of the objects in the image and generate a geometrically corrected image. The illumination characteristic map is decomposed into a global illumination component and a high-frequency shadow component, which are then used to adjust the global hue and local brightness and darkness contrast of the geometrically corrected image, respectively, to generate the effect image. The rendered image is post-processed with edge sharpening and noise suppression to generate a real-world rendering of the target scene.

8. An electronic device, characterized in that, The system includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method for generating interior decoration renderings based on style transfer as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the method for generating interior decoration renderings based on style transfer as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein when the program instructions are executed by a computer, the computer performs the method for generating interior decoration renderings based on style transfer as described in any one of claims 1-7.

Images