A decoration effect video generation system and method

Abstract

The present application relates to the technical field of computer, and especially relates to a decoration effect video generation system and method. A space original graph and a target decoration effect graph are acquired through an input module, and structured data is generated through feature extraction and semantic segmentation; a timing processing module fuses the structured data with preset decoration process timing rules, generates stage condition control signals, and generates an initial image sequence through frame-by-frame inference generation under the dual constraints of decoration fixed structure maintenance and regional texture continuity; a physical rendering module dynamically adjusts a physical light model and material reflection properties based on the process stage, performs frame-by-frame light rendering on the initial image sequence to generate a target image sequence; and a video synthesis module performs timing encoding and packaging on the target image sequence to generate a decoration effect video. The present application realizes dynamic control of decoration processes and space structures, and improves the process logic, visual coherence and physical realism of the video.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a system and method for generating decoration effect videos. Background Technology

[0002] Enhancing the visualization and dynamic demonstration of the interior design process is crucial for improving the presentation of design schemes and improving customer experience. Automatically generating interior design effect videos based on image generation models is an important way to achieve efficient display and immersive communication of design schemes. However, this technology still faces common technical bottlenecks and practical challenges in practical applications. For example, existing methods typically rely on static renderings or fixed-viewpoint 3D models, which are insufficient to reflect the gradual progress of the decoration process and hinder customers' understanding of the construction logic and phased results. Furthermore, existing video generation technologies, when applied to interior design scenarios, generally suffer from problems such as inter-frame content jumps, insufficient physical realism, and low generation efficiency, restricting their integrated application in actual design processes.

[0003] Currently, various techniques such as temporal compositing and neural rendering have been developed for generating dynamic effects. However, existing methods still have significant shortcomings in terms of generation quality and controllability when dealing with interior decoration scenes. Most methods rely on a uniform generation strategy, such as fixed-viewpoint sequence compositing, simple transition animation splicing, or globally consistent texture generation. This control mode is difficult to adapt to the complex and ever-changing temporal logic and spatial structural constraints of interior decoration processes. When generating interior decoration processes involving multiple steps and changes in multiple objects, a fixed generation strategy can easily lead to inconsistencies in spatial structure and abrupt changes in object positions or materials; while when generating simple scene changes, excessively complex model calculations may cause unnecessary resource consumption and time delays. Summary of the Invention

[0004] This invention provides a system and method for generating decoration effect videos, which solves the technical problem that existing methods for generating decoration effect videos cannot dynamically control the process by combining the timing rules of decoration procedures with the characteristics of spatial structure, making it difficult to adapt to the dynamic changes of different decoration schemes and process logic, thus failing to achieve reliable dynamic visualization of the decoration process.

[0005] The first aspect of the present invention provides a decoration effect video generation system, comprising: an input module, a timing processing module, a physical rendering module, and a video compositing module connected in sequence;

[0006] The input module is used to acquire the original spatial image and the target decoration rendering, and to perform feature extraction and semantic segmentation on the original spatial image and the target decoration rendering to generate structured data;

[0007] The timing processing module is used to fuse the structured data with the preset timing rules of the decoration process, generate a staged conditional control signal containing the mask of the area to be updated and the texture evolution path, and use the staged conditional control signal as a guide, with the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, to generate an initial image sequence frame by frame.

[0008] The physical rendering module is used to perform lighting rendering on each frame of the initial image sequence based on the physically adjusted lighting model and material reflection properties at each stage of the process, and generate the target image sequence.

[0009] The video synthesis module is used to perform time-series encoding and video encapsulation on the target image sequence to generate a decoration effect video.

[0010] Optionally, the input module includes: a spatial structure segmentation unit, a decoration object identification unit, a material attribute extraction unit, and a structured data integration unit;

[0011] The spatial structure segmentation unit is used to acquire the original spatial map and the target decoration rendering, and extract the fixed structures of walls, floors, ceilings, doors, windows, beams and columns from the original spatial map and the target decoration rendering to generate spatial structure data;

[0012] The decoration object identification unit is used to identify the unfinished objects in the original space map and the completed objects in the target decoration effect map, and generate decoration object feature data.

[0013] The material attribute extraction unit is used to extract the material parameters of various objects and spatial structures in the target decoration rendering, and associate them with the material base data of the corresponding area of ​​the original spatial rendering to generate material attribute association data;

[0014] The structured data integration unit is used to integrate the spatial structure data, the decoration object feature data, and the material attribute association data to generate structured data.

[0015] Optionally, the timing processing module performs the following steps:

[0016] The spatial structure features, decoration object features, and material features in the structured data are concatenated and weighted with the preset decoration process sequence rules to generate process guidance features;

[0017] Based on the process guidance features, a staged condition control signal corresponding to each decoration process is constructed; the staged condition control signal includes the mask of the area to be updated and the texture evolution path corresponding to the current process;

[0018] Guided by the staged conditional control signal, and constrained by the total loss obtained by weighting the fixed total structural loss value and the regional texture continuity loss, the initial image sequence is generated frame by frame iteratively.

[0019] Optionally, the step of iteratively generating an initial image sequence frame by frame, guided by the staged conditional control signal and constrained by the total loss obtained by weighting a fixed total structural loss value with a regional texture continuity loss, includes:

[0020] The staged conditional control signal is input into the image generation network to generate the initial generated image of the current frame;

[0021] Obtain the temporal context frame of the current frame; wherein the temporal context frame is the previous generated frame of the current frame; if the current frame is the first frame of the decoration process and there is no previous generated frame, then initialize and generate based on the spatial structure features in the structured data, and use the initialized and generated image as the temporal context frame.

[0022] The initial generated image and the temporal context frame are input into the optical flow estimation network to calculate the forward optical flow field;

[0023] The temporal context frame is distorted at the pixel level based on the forward optical flow field to generate the predicted image of the current frame;

[0024] Calculate the optical flow consistency loss in the fixed structure region and the texture continuity loss in the non-fixed structure region, and then weight and fuse the optical flow consistency loss and the texture continuity loss to obtain the total loss value;

[0025] When the total loss value is greater than or equal to the preset loss threshold and the current iteration number has not reached the preset maximum iteration number, the generation parameters corresponding to the current frame are iterated with the goal of minimizing the total loss, and the process jumps to the step of inputting the staged conditional control signal into the image generation network to generate the initial generated image of the current frame.

[0026] When the total loss value is less than the preset loss threshold, or when the current iteration number reaches the preset maximum iteration number, it is determined that the current frame has been generated and the current frame is used as the temporal context frame of the subsequent frame.

[0027] When all the image frames corresponding to the decoration process have been generated, the generated image frames are combined and output as the initial image sequence.

[0028] Optionally, the physical rendering module includes a physical lighting rendering unit and a material reflection property rendering unit;

[0029] The physical lighting rendering unit is used to calculate the global lighting distribution and generate a lighting distribution map by using a ray tracing algorithm through a physical lighting model based on the indoor space geometric parameters and light source parameters in the structured data.

[0030] The material reflection attribute rendering unit is used to dynamically extract the material reflection attribute parameters of the corresponding process stage according to the decoration process stage identifier of the frame, and combine the illumination distribution map to perform pixel-by-pixel illumination rendering on each frame of the initial image sequence through the physical illumination model to generate the target image sequence.

[0031] Optionally, the material reflection property rendering unit performs the following steps:

[0032] Read the decoration process stage identifier corresponding to each frame of the initial image sequence;

[0033] Based on the stage identifier of the decoration process, the material reflection attribute parameters corresponding to the current process stage are dynamically extracted from the material attribute association data in the structured data;

[0034] The extracted material reflection attribute parameters and the illumination distribution map are fused pixel by pixel according to the physically based rendering equation corresponding to the physical lighting model. Then, the physical lighting model is used to sequentially render the illumination of each frame in the initial image sequence to generate the target image sequence.

[0035] The second aspect of this invention provides a method for generating decoration effect videos, comprising:

[0036] Obtain the original spatial image and the target decoration rendering, and perform feature extraction and semantic segmentation on the original spatial image and the target decoration rendering to generate structured data;

[0037] The structured data is fused with the preset decoration process timing rules to generate a staged conditional control signal containing the mask of the area to be updated and the texture evolution path. Guided by the staged conditional control signal, and with the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, the initial image sequence is generated frame by frame through reasoning.

[0038] Based on the dynamically adjusted physical lighting model and material reflection properties at each stage of the process, lighting rendering is performed on each frame of the initial image sequence to generate the target image sequence.

[0039] The target image sequence is time-series encoded and video-encapsulated to generate a decoration effect video.

[0040] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the decoration effect video generation method described above.

[0041] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the decoration effect video generation method as described above.

[0042] The fifth aspect of the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer performs the decoration effect video generation method as described above.

[0043] As can be seen from the above technical solutions, the present invention has the following advantages:

[0044] This invention utilizes the collaborative operation of an input module, a timing processing module, a physical rendering module, and a video synthesis module. First, the input module acquires the original spatial image and the target decoration effect image, performing feature extraction and semantic segmentation to transform the image information into structured data containing spatial structure, decoration objects, and material attributes. Then, the timing processing module deeply integrates this structured data with preset decoration process timing rules, generating a staged conditional control signal containing the mask of the area to be updated and the texture evolution path. Simultaneously, using the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, it infers frame-by-frame to generate an initial image sequence. Subsequently, the physical rendering module dynamically adjusts the physical lighting model and material reflection attributes based on the process stages, performing lighting rendering on the initial image sequence frame-by-frame to generate the target image sequence. Finally, the video synthesis module performs timing encoding and encapsulation of the target image sequence, outputting the final decoration effect video. This solution specifically addresses the problem that existing decoration effect video generation methods cannot dynamically control the combination of decoration process timing rules and spatial structure characteristics, making them difficult to adapt to different... The dynamic changes in decoration plans and construction process logic present technical challenges in achieving reliable dynamic visualization of the decoration process. This solution addresses these challenges by integrating structured data with the sequential rules of decoration procedures. It ensures that the video generation process strictly adheres to real construction logic, adapting to the differentiated needs of various decoration plans and construction processes. The constraint of maintaining fixed structures ensures the stability and non-displacement of spatial structures such as walls, floors, doors, and windows between frames, while the constraint of regional texture continuity guarantees smooth and natural texture changes in the construction area. This effectively avoids common problems in traditional generation methods, such as structural misalignment and texture breaks. Furthermore, the physically rendered module dynamically adjusts lighting and material parameters according to the construction stage, ensuring a high degree of physical realism in the video that matches the construction progress. This allows users to intuitively and clearly perceive the complete decoration process from bare walls to finished product, without relying on complex 3D modeling technology. This lowers the technical barrier to entry while maintaining both generation efficiency and quality, providing an efficient and practical tool for real-world scenarios such as decoration design presentations, construction briefings, and customer communication. It possesses significant industrial application value and promising prospects for promotion. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a structural block diagram of a decoration effect video generation system provided in an embodiment of the present invention;

[0047] Figure 2This is a structural block diagram of the input module provided in an embodiment of the present invention;

[0048] Figure 3 This is a structural block diagram of the physical rendering module provided in an embodiment of the present invention;

[0049] Figure 4 A flowchart illustrating the steps of a method for generating a decoration effect video according to an embodiment of the present invention;

[0050] Figure 5 This is a structural block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0051] This invention provides a system and method for generating decoration effect videos, which solves the technical problem that existing decoration effect video generation methods cannot dynamically control the combination of decoration process sequence rules and spatial structure characteristics, making it difficult to adapt to the dynamic changes of different decoration schemes and process logic, thus failing to achieve reliable dynamic visualization of the decoration process.

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that in the optional embodiments of the present invention, the object information and other related data involved require the permission or consent of the object when the embodiments of the present invention are applied to specific products or technologies, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. That is to say, if the embodiments of the present invention involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations, and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0053] Please see Figure 1 , Figure 1 This is a structural block diagram of a decoration effect video generation system provided in an embodiment of the present invention.

[0054] The present invention provides a decoration effect video generation system, comprising: an input module 101, a timing processing module 102, a physical rendering module 103, and a video compositing module 104 connected in sequence.

[0055] Input module 101 is used to acquire the original spatial image and the target decoration rendering, and to perform feature extraction and semantic segmentation on the original spatial image and the target decoration rendering to generate structured data.

[0056] It should be noted that the original spatial plan refers to an image of the space before or after renovation, such as an image of a bare-bones apartment, an original floor plan, or an image of the site before construction, used to represent the initial state before renovation. The target renovation rendering, on the other hand, refers to an image of the design effect after renovation, used to represent the final desired renovation state.

[0057] In practical applications, the core function of the input module 101 is to transform unstructured image data into structured information, providing a foundation for subsequent processing. Specifically, the input module 101 acquires the original spatial image and the target decoration rendering provided by the user, extracts low-level visual information such as contours, textures, and colors from the images using feature extraction algorithms, and then distinguishes image pixels according to categories specific to the decoration scene using semantic segmentation algorithms, ultimately generating structured data containing information on spatial structure, decoration objects, and materials.

[0058] Furthermore, such as Figure 2 As shown, the input module 101 is further subdivided into a spatial structure segmentation unit, a decoration object identification unit, a material attribute extraction unit, and a structured data integration unit. Each unit works together to ensure the accuracy and completeness of the structured data.

[0059] The spatial structure segmentation unit focuses on extracting fixed structural information from images. It processes the original spatial image and the target decoration effect image, identifies structural edges in the image through edge detection algorithms, and then accurately extracts fixed structures such as walls, floors, ceilings, doors, windows, beams and columns whose positions and shapes remain basically unchanged during the decoration process by combining contour extraction algorithms. It generates spatial structure data containing information such as fixed structure dimensions, coordinates, and contours. This data is the foundation for ensuring the stability of the spatial skeleton in the subsequent generation process. For example, it can accurately extract the wall contours and door and window positions in the bedroom to avoid structural misalignment in the subsequent generation process.

[0060] The decoration object recognition unit is responsible for distinguishing the state of decoration objects in the image. Based on the spatial structure data generated by the spatial structure segmentation unit, it uses the object detection algorithm to identify the unfinished objects (such as cement walls, original floors, exposed pipes, etc.) in the original spatial image and the finished decoration objects (such as decorative walls, paved floors, finished furniture, etc.) in the target decoration effect image. It generates decoration object feature data containing information such as object category, location, and outline, clarifies the object change relationship before and after decoration, and provides object-level control basis for time series generation.

[0061] The core function of the material attribute extraction unit is to obtain the material parameters of the decoration object. Based on the feature data of the decoration object, the material parameters of various objects and spatial structures in the target decoration rendering are extracted through the material feature extraction algorithm, including color, reflectivity, roughness, metallicity, etc. At the same time, the material base data of the corresponding area of ​​the original space image (i.e. the material information in the rough state) are associated to generate material attribute association data. This data establishes the correspondence between the rough material and the finished material, providing data support for the gradual transition of materials.

[0062] The structured data integration unit is used to integrate the above three types of data. It uses a data normalization algorithm to convert spatial structure data, decoration object feature data, and material attribute association data into a unified format, removes redundant information and invalid data, ensures data consistency and usability, and finally generates structured data for direct use by the timing processing module 102 and the physical rendering module 103, ensuring the efficiency of data interaction between various modules of the system.

[0063] The timing processing module 102 is used to fuse structured data with preset timing rules for decoration procedures, generate a staged conditional control signal containing the mask of the area to be updated and the texture evolution path, and use the staged conditional control signal as a guide, with the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, to generate the initial image sequence frame by frame.

[0064] It should be noted that the preset decoration process sequence rule refers to the pre-set construction sequence according to the interior decoration construction logic, such as the basic renovation, water and electricity construction, wall base treatment, wall decoration construction, floor paving, ceiling installation, door and window installation, furniture and soft furnishing delivery, etc., to ensure that the video generation process is consistent with the actual construction process.

[0065] In this embodiment of the invention, the timing processing module 102, as the core control unit of the system, is responsible for implementing the timing generation logic of the decoration process. This module first integrates structured data with preset decoration process timing rules. These rules are based on standard construction procedures in the decoration industry, clearly defining the sequence and connection logic of each construction stage. After integration, a staged conditional control signal is generated, containing a mask of the area to be updated and a texture evolution path. The mask of the area to be updated identifies the image area that needs to change in the current construction stage, and the texture evolution path defines the texture change process of the decoration object from its initial state to completion. Simultaneously, the timing processing module 102 uses both fixed structure maintenance constraints and regional texture continuity constraints as dual constraints to generate an initial image sequence frame by frame. The fixed structure maintenance constraints ensure that the positions of fixed structures such as walls, floors, doors, and windows are stable in each frame, while the regional texture continuity constraints ensure that the texture changes in the construction area are smooth and natural, avoiding structural shifts or texture breaks.

[0066] Furthermore, the timing processing module 102 generates the initial image sequence through three core operations, with each step progressing in turn to ensure the orderliness and controllability of the generation process.

[0067] The first step involves the time-series processing module 102 performing feature concatenation and weight mapping between the spatial structure features, decoration object features, and material features in the structured data and the preset decoration process time-series rules. Feature concatenation achieves the fusion of multi-source information by splicing the three types of features along the feature dimensions. Weight mapping assigns adaptive weights to different features according to the needs of the current construction stage. For example, in the wall decoration stage, the weight of material features is appropriately increased, while in the structural modification stage, the constraint effect of spatial structure features is strengthened. Through this process, process-guided features that are highly matched with the current construction stage are generated.

[0068] The second step involves constructing staged conditional control signals corresponding to each decoration process based on the generated process guidance features. Each process corresponds to a set of exclusive staged conditional control signals, which include the mask of the area to be updated and the texture evolution path for the current process. The mask of the area to be updated accurately identifies the construction area of ​​the current process, ensuring that only this area changes while non-construction areas remain stable. The texture evolution path clarifies the texture change pattern of the decoration object in the current process. For example, in the process of laying floor tiles, the texture evolution path defines the laying order of tiles and the texture connection method. This signal enables fine-grained control of image generation.

[0069] The third step involves iteratively generating an initial image sequence frame by frame, guided by a staged conditional control signal and constrained by a total loss obtained by weighting the fixed structure optical flow consistency loss and the regional texture continuity loss. The fixed structure optical flow consistency loss measures the degree of offset of the fixed structure between frames and is obtained by calculating the optical flow error of the fixed structure region between the current frame and the previous frame. The regional texture continuity loss evaluates the smoothness of texture changes in the construction area and is obtained by calculating the texture similarity of the construction area in adjacent frames. The total loss is obtained by weighted fusion of the two types of losses and is used to comprehensively evaluate the generation quality of the current frame. When the total loss meets a preset threshold, the current frame is considered qualified, and the process proceeds to the next frame. This constraint mechanism ensures that the generated initial image sequence maintains both structural stability and natural texture variations.

[0070] It is worth mentioning that, in this embodiment of the invention, the process of iteratively generating the initial image sequence frame by frame includes several detailed steps. Through complete calculation, judgment, and optimization logic, the quality of the generated initial image sequence is ensured. First, the staged conditional control signal is input to the image generation network. The image generation network is a deep learning-based generation model, trained and optimized using a dataset specific to the decoration scene. It can generate an initial generated image that meets the requirements of the current process according to the input control signal. The image resolution is consistent with the input image, ensuring the continuity of the visual effect. Next, the temporal context frame of the current frame is obtained. The temporal context frame is the image of the previous frame that has been generated and is used to provide a temporal reference for the generation of the current frame. If the current frame is the first frame of a certain decoration process and there is no previous generated frame, initialization is performed based on the spatial structure features in the structured data to generate a rough state image containing only a fixed structure. This image is used as the temporal context frame to ensure the accuracy of the starting benchmark for temporal generation.

[0071] Then, the initially generated image and the temporal context frame are input into the optical flow estimation network. The optical flow estimation network is used to calculate the pixel motion relationship between the two frames and outputs a forward optical flow field. This optical flow field contains the motion vector of each pixel from the temporal context frame to the current frame, which can accurately reflect the change trend between frames. Based on the forward optical flow field, the temporal context frame is distorted at the pixel level. The pixels of the temporal context frame are mapped to the corresponding positions of the current frame according to the optical flow vector using a bilinear interpolation algorithm to generate the prediction image of the current frame. This prediction image retains the key information of the temporal context frame and provides a reference for subsequent loss calculation.

[0072] Next, the optical flow consistency loss in the fixed structure region and the texture continuity loss in the non-fixed structure region are calculated, and the two are weighted and fused to obtain the total loss value. The optical flow consistency loss in the fixed structure region is obtained by traversing all pixels in the fixed structure region, calculating the optical flow error of each pixel, and summing and averaging the results. This loss is used to determine whether the fixed structure has shifted. The texture continuity loss in the non-fixed structure region is derived by calculating the structural similarity index (SSIM) of the construction area. This loss is used to evaluate whether the texture changes are smooth. The total loss value is calculated by fusing the two types of losses after assigning appropriate weights. The weight assignment can be dynamically adjusted according to the process stage. For example, the weight of the optical flow consistency loss can be increased in important stages of fixed structure, and the weight of the texture continuity loss can be increased in important stages of texture changes.

[0073] When the total loss value is greater than or equal to the preset loss threshold and the current iteration number has not reached the preset maximum iteration number, it is determined that the quality of the current frame generation is not up to standard. With the goal of minimizing the total loss, the generation parameters corresponding to the current frame are iteratively optimized, including the texture intensity and optical flow constraint intensity of the image generation network. After optimization, the process jumps to the step of inputting the staged conditional control signal into the image generation network to regenerate the initial generated image of the current frame until the total loss value meets the requirements or the iteration number reaches the upper limit.

[0074] When the total loss value is less than the preset loss threshold, or the current iteration count reaches the preset maximum iteration count, the current frame is determined to be generated successfully and stored in the temporal buffer as the temporal context frame for subsequent frames. After all image frames corresponding to the decoration process have been generated, all qualified frames are combined in temporal order to output the initial image sequence, ensuring the temporal continuity and integrity of the sequence.

[0075] The physical rendering module 103 is used to perform lighting rendering on each frame of the initial image sequence based on the physically adjusted lighting model and material reflection properties at each stage of the process, and to generate the target image sequence.

[0076] It should be noted that dynamic adjustment of the process stage refers to adaptively adjusting the corresponding lighting and material performance parameters according to the current construction process stage of the frame, so that the rendering effect matches the construction progress. The physical lighting model is a lighting calculation model that follows the laws of physical optics and is built based on the conservation of light energy and the optical properties of materials. It is used to simulate physical phenomena such as reflection, refraction, and scattering of light on the surface of objects, so that the brightness, tonal transitions, and material texture of the rendered image are consistent with the real physical world, which is different from simple color filling or empirical rendering methods.

[0077] In this embodiment of the invention, the physically-rendered module 103 is used to enhance the realism of the initial image sequence. It performs frame-by-frame rendering based on a physically-based lighting model dynamically adjusted according to the construction stage and material reflection properties. The physically-based lighting model follows real optical physics laws and can simulate the reflection, refraction, and scattering behavior of light on the surface of an object. The material reflection properties are dynamically adjusted according to the characteristics of the current construction stage, so that different processes present corresponding material performances. For example, there are significant differences in reflectivity and roughness between the wet and dry stages of wall paint. Through the synergistic effect of the two, the rendering result conforms to the visual effect of a real physical scene.

[0078] Furthermore, such as Figure 3 As shown, the physical rendering module 103 includes a physical lighting rendering unit and a material reflection property rendering unit. The two units work together to achieve high-quality rendering that conforms to physical laws.

[0079] The physically based lighting rendering unit is responsible for calculating the global lighting distribution of the scene. Based on the indoor spatial geometric parameters (such as space dimensions, floor height, and door / window positions) and light source parameters (such as the solar altitude angle and color temperature of natural light, and the position, power, and illumination angle of artificial light sources) in the structured data, it uses a ray tracing algorithm based on a physically based lighting model to simulate the propagation, reflection, refraction, and scattering processes of light. It calculates the light intensity and color of each pixel in the scene, generating a lighting distribution map. Ray tracing algorithms can accurately simulate real-world lighting effects, such as simulating the gradual shadows created by sunlight shining through windows onto walls, or the highlight reflections created by light on floor tiles, ensuring that the lighting distribution map conforms to physical laws and providing a foundation for subsequent material rendering.

[0080] The material reflection attribute rendering unit is responsible for achieving dynamic material rendering effects. Based on the corresponding construction process stage identifier for each frame, it dynamically extracts the material reflection attribute parameters corresponding to the current process stage from the material attribute association data in the structured data. These parameters include reflectivity, roughness, metallicity, and refractive index. These parameters change dynamically with the construction stage; for example, cement material has low reflectivity and high roughness during the wall base treatment stage, while paint has high reflectivity and low roughness during the paint drying stage. After extracting the parameters, the material reflection attribute rendering unit combines these parameters with the lighting distribution map generated by the physically based lighting rendering unit. Through the physically based rendering equation corresponding to the physically based lighting model, it performs pixel-by-pixel lighting rendering on each frame of the initial image sequence, fully integrating the optical properties of the material with the scene lighting to generate a physically realistic target image sequence.

[0081] It is worth mentioning that, in this embodiment of the invention, the material reflection attribute rendering unit achieves process-driven dynamic material rendering through three specific operations, ensuring that the rendering effect is highly matched with the construction stage.

[0082] The first step is to read the corresponding construction process stage identifier from each frame of the initial image sequence. This identifier is stored in the frame header information and is used to clarify the specific construction stage of the current frame, such as basic renovation, water and electricity construction, wall painting, floor paving, etc., providing a clear basis for the subsequent extraction of material parameters. The reading process is implemented through a frame parsing algorithm to ensure the accuracy and efficiency of the parsing.

[0083] The second step involves dynamically extracting the material reflection attribute parameters corresponding to the current construction stage from the material attribute association data of the structured data, based on the read construction process stage identifier. The material attribute association data stores material parameter templates for different construction objects at each stage. The extraction process is achieved through index lookup, quickly locating the corresponding parameter template based on the construction stage identifier to ensure timely parameter extraction. The extracted parameters include core optical characteristic parameters such as reflectivity, roughness, and metallicity, accurately characterizing the material performance at the current stage.

[0084] The third step involves pixel-by-pixel fusion calculations of the extracted material reflection attribute parameters and the lighting distribution map, based on the physically based rendering equations corresponding to the physical lighting model. These equations adhere to the laws of energy conservation and optical propagation, accurately calculating the color representation of materials under current lighting conditions. Pixel-by-pixel calculations ensure that the color of each pixel aligns with the synergistic effect of material characteristics and the lighting environment. After calculation, the physical lighting model is used to sequentially render the lighting of each frame in the initial image sequence, generating a target image sequence. Each frame in this sequence accurately represents the material and lighting effects of the corresponding construction stage, significantly enhancing the realism and credibility of the decoration effect video.

[0085] The video synthesis module 104 is used to perform time-series encoding and video encapsulation on the target image sequence to generate a decoration effect video.

[0086] In this embodiment of the invention, the video synthesis module 104 receives the target image sequence output by the physical rendering module 103, compresses and encodes the image sequence using a time-series coding algorithm, and then encapsulates it according to a common video format standard to finally generate a decoration effect video that can be played directly. This video can be used in actual scenarios such as decoration design display and construction briefing to help users intuitively understand the entire decoration process.

[0087] This invention utilizes the collaborative operation of an input module 101, a timing processing module 102, a physical rendering module 103, and a video synthesis module 104. The input module 101 performs feature extraction, semantic segmentation, and structured data generation on the original spatial image and the target decoration effect image. The timing processing module 102 fuses the structured data with preset decoration process timing rules to generate a staged conditional control signal, and iteratively generates an initial image sequence frame by frame using both the decoration fixed structure maintenance constraint and the regional texture continuity constraint as dual constraints. The physically based rendering module 103 dynamically adjusts the physical lighting model and material reflection properties based on the process stage to perform pixel-by-pixel lighting rendering. The video compositing module 104 completes the temporal encoding and encapsulation to output the final video. This effectively solves the technical problem that existing methods for generating decoration effect videos cannot dynamically control the temporal rules and spatial structure characteristics of decoration processes, making it difficult to adapt to the dynamic changes of different decoration schemes and process logic, thus failing to achieve credible dynamic visualization of the decoration process. It not only ensures the stability of the spatial structure between frames and the natural texture changes in the decoration effect video, avoiding defects such as structural offset and texture discontinuity, but also achieves accurate matching between the decoration process and the real construction process, allowing users to intuitively and clearly understand the complete process from rough to finished. At the same time, it greatly improves the physical realism and visual credibility of the video. It can efficiently generate high-quality decoration effect videos without relying on complex 3D modeling, reducing the technical threshold and communication costs. It provides a practical and efficient tool for decoration design display, construction briefing and other scenarios, and has significant industrial application value and promotion prospects.

[0088] Please see Figure 4 , Figure 4 A flowchart illustrating the steps of a method for generating a decoration effect video according to an embodiment of the present invention.

[0089] This invention provides a method for generating decoration effect videos, comprising:

[0090] Step 401: Obtain the original space image and the target decoration rendering, and perform feature extraction and semantic segmentation on the original space image and the target decoration rendering to generate structured data;

[0091] Step 402: Integrate the structured data with the preset decoration process timing rules to generate a staged conditional control signal containing the mask of the area to be updated and the texture evolution path. Guided by the staged conditional control signal, and with the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, generate the initial image sequence frame by frame through reasoning.

[0092] Step 403: Based on the dynamically adjusted physical lighting model and material reflection properties at each stage of the process, perform lighting rendering on each frame of the initial image sequence to generate the target image sequence.

[0093] Step 404: Perform time-series encoding and video encapsulation on the target image sequence to generate a decoration effect video.

[0094] Please see Figure 5 , Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0095] An electronic device according to an embodiment of the present invention includes: a memory 501 and a processor 502. The memory 501 stores a computer program. When the computer program is executed by the processor 502, the processor 502 performs the decoration effect video generation method as described in any of the above embodiments.

[0096] Memory 501 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 501 has storage space 503 for program code 513 for performing any of the method steps described above. For example, storage space 503 for program code may include various program codes 513 for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When run by a computing processing device, this code causes the computing processing device to perform the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When this code is run by a computing device, it causes the computing device to perform the various steps in the decoration effect video generation method described above.

[0097] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the decoration effect video generation method as described in any of the above embodiments.

[0098] This invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the decoration effect video generation method as described in any of the above embodiments.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A system for generating videos showcasing interior decoration effects, characterized in that, include: The input module, timing processing module, physical rendering module, and video compositing module are connected in sequence. The input module is used to acquire the original spatial image and the target decoration rendering, and to perform feature extraction and semantic segmentation on the original spatial image and the target decoration rendering to generate structured data; The timing processing module is used to fuse the structured data with the preset timing rules of the decoration process, generate a staged conditional control signal containing the mask of the area to be updated and the texture evolution path, and use the staged conditional control signal as a guide, with the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, to generate an initial image sequence frame by frame. The physical rendering module is used to perform lighting rendering on each frame of the initial image sequence based on the physically adjusted lighting model and material reflection properties at each stage of the process, and generate the target image sequence. The video synthesis module is used to perform time-series encoding and video encapsulation on the target image sequence to generate a decoration effect video.

2. The decoration effect video generation system according to claim 1, characterized in that, The input module includes: a spatial structure segmentation unit, a decoration object identification unit, a material attribute extraction unit, and a structured data integration unit; The spatial structure segmentation unit is used to acquire the original spatial map and the target decoration rendering, and extract the fixed structures of walls, floors, ceilings, doors, windows, beams and columns from the original spatial map and the target decoration rendering to generate spatial structure data; The decoration object identification unit is used to identify the unfinished objects in the original space map and the completed objects in the target decoration effect map, and generate decoration object feature data. The material attribute extraction unit is used to extract the material parameters of various objects and spatial structures in the target decoration rendering, and associate them with the material base data of the corresponding area of ​​the original spatial rendering to generate material attribute association data; The structured data integration unit is used to integrate the spatial structure data, the decoration object feature data, and the material attribute association data to generate structured data.

3. The decoration effect video generation system according to claim 1, characterized in that, The timing processing module performs the following steps: The spatial structure features, decoration object features, and material features in the structured data are concatenated and weighted with the preset decoration process sequence rules to generate process guidance features; Based on the process guidance features, a staged condition control signal corresponding to each decoration process is constructed; the staged condition control signal includes the mask of the area to be updated and the texture evolution path corresponding to the current process; Guided by the staged conditional control signal, and constrained by the total loss obtained by weighting the fixed total structural loss value and the regional texture continuity loss, the initial image sequence is generated frame by frame iteratively.

4. The decoration effect video generation system according to claim 3, characterized in that, The step of iteratively generating an initial image sequence frame by frame, guided by the staged conditional control signal and constrained by the total loss obtained by weighting the fixed structural total loss value and the regional texture continuity loss, includes: The staged conditional control signal is input into the image generation network to generate the initial generated image of the current frame; Obtain the temporal context frame of the current frame; wherein the temporal context frame is the previous generated frame of the current frame; if the current frame is the first frame of the decoration process and there is no previous generated frame, then initialize and generate based on the spatial structure features in the structured data, and use the initialized and generated image as the temporal context frame. The initial generated image and the temporal context frame are input into the optical flow estimation network to calculate the forward optical flow field; The temporal context frame is distorted at the pixel level based on the forward optical flow field to generate the predicted image of the current frame; Calculate the optical flow consistency loss in the fixed structure region and the texture continuity loss in the non-fixed structure region, and then weight and fuse the optical flow consistency loss and the texture continuity loss to obtain the total loss value; When the total loss value is greater than or equal to the preset loss threshold and the current iteration number has not reached the preset maximum iteration number, the generation parameters corresponding to the current frame are iterated with the goal of minimizing the total loss, and the process jumps to the step of inputting the staged conditional control signal into the image generation network to generate the initial generated image of the current frame. When the total loss value is less than the preset loss threshold, or when the current iteration number reaches the preset maximum iteration number, it is determined that the current frame has been generated and the current frame is used as the temporal context frame of the subsequent frame. When all the image frames corresponding to the decoration process have been generated, the generated image frames are combined and output as the initial image sequence.

5. The decoration effect video generation system according to claim 1, characterized in that, The physical rendering module includes a physical lighting rendering unit and a material reflection property rendering unit; The physical lighting rendering unit is used to calculate the global lighting distribution and generate a lighting distribution map by using a ray tracing algorithm through a physical lighting model based on the indoor space geometric parameters and light source parameters in the structured data. The material reflection attribute rendering unit is used to dynamically extract the material reflection attribute parameters of the corresponding process stage according to the decoration process stage identifier of the frame, and combine the illumination distribution map to perform pixel-by-pixel illumination rendering on each frame of the initial image sequence through the physical illumination model to generate the target image sequence.

6. The decoration effect video generation system according to claim 5, characterized in that, The material reflection property rendering unit performs the following steps: Read the decoration process stage identifier corresponding to each frame of the initial image sequence; Based on the stage identifier of the decoration process, the material reflection attribute parameters corresponding to the current process stage are dynamically extracted from the material attribute association data in the structured data; The extracted material reflection attribute parameters and the illumination distribution map are fused pixel by pixel according to the physically based rendering equation corresponding to the physical lighting model. Then, the physical lighting model is used to sequentially render the illumination of each frame in the initial image sequence to generate the target image sequence.

7. A method for generating a decoration effect video, characterized in that, include: Obtain the original spatial image and the target decoration rendering, and perform feature extraction and semantic segmentation on the original spatial image and the target decoration rendering to generate structured data; The structured data is fused with the preset decoration process timing rules to generate a staged conditional control signal containing the mask of the area to be updated and the texture evolution path. Guided by the staged conditional control signal, and with the decoration fixed structure maintenance constraint and the area texture continuity constraint as dual constraints, the initial image sequence is generated frame by frame through reasoning. Based on the dynamically adjusted physical lighting model and material reflection properties at each stage of the process, lighting rendering is performed on each frame of the initial image sequence to generate the target image sequence. The target image sequence is time-series encoded and video-encapsulated to generate a decoration effect video.

8. An electronic device, characterized in that, The device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the decoration effect video generation method as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the decoration effect video generation method as described in claim 7.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein when the program instructions are executed by a computer, the computer performs the decoration effect video generation method as described in claim 7.

Images