Rendering optimization method and system for virtual and real scene fusion of cultural and creative products

Abstract

The invention provides a rendering optimization method and system for virtual-real scene fusion of a cultural and creative product, and the method comprises the steps: decomposing the material information of a virtual model of the cultural and creative product into basic color information and detail control information, and enabling low-frequency diffuse reflection, high-frequency texture and micro-surface details to be independently processed; secondly, constructing a visual weighting function which takes the fixation hotspot map and the virtual and real boundary distance as input, and realizing dynamic directional control of rendering parameters by calculating the attention weight of each pixel and generating a visual contribution adjustment coefficient which is positively correlated with the attention intensity and negatively correlated with the boundary distance; and finally, mapping the visual contribution adjustment coefficient to detail control information, performing pixel-level proportion adjustment on high-frequency textures and micro-surface details in a rendering process, and optimizing local detail representation and virtual-real boundary transition. By adopting the scheme of the invention, detail control pixel-level adaptive adjustment can be realized based on the fixation hotspot map and the virtual-real boundary information, so that the rendering precision of the virtual-real fusion scene is enhanced.

Description

Technical Field

[0001] This application relates to the field of scene fusion rendering technology, and more specifically, to a rendering optimization method and system for fusion of virtual and real scenes in cultural and creative products. Background Technology

[0002] In virtual and real scene rendering, the visual performance of cultural and creative products not only depends on the geometric accuracy of the model, but also on the realistic presentation of material details and lighting interaction effects. Traditional rendering optimization methods usually achieve visual optimization by unifying material parameters or adjusting global illumination. It is difficult to make fine control over high-frequency textures, micro-surface undulations and specular changes at the pixel level. As a result, there are problems such as insufficient local detail and uneven rendering realism in virtual and real scene integration. With the introduction of user interaction behavior and gaze patterns, higher requirements are placed on the adaptability of rendering algorithms in the area of ​​interest. It is urgent to realize a dynamic optimization mechanism based on visual perception to improve the overall rendering quality.

[0003] However, existing technologies typically employ fixed-ratio or globally uniform intensity processing in the rendering of detail control information, failing to effectively adjust pixel-level attention differences. For example, they cannot automatically adjust the contribution of high-frequency textures and micro-surface details based on the user's gaze hotspots and the spatial position of pixels relative to the virtual-real fusion boundary. This results in excessive or insufficient detail in the user's visual focus area, and abrupt rendering in the transition area of ​​the virtual-real boundary, thus affecting the overall virtual-real fusion effect. Existing technologies lack a mechanism to directly map user gaze behavior and spatial boundary sensitivity to detail rendering parameters, making it difficult to achieve personalized and pixel-level optimization of rendering results. Therefore, how to achieve pixel-level adaptive adjustment of detail control based on gaze hotspot maps and virtual-real boundary information, thereby enhancing the rendering accuracy of virtual-real fusion scenes, has become a challenge for the industry. Summary of the Invention

[0004] This application provides a rendering optimization method and system for the fusion of virtual and real scenes in cultural and creative products. It can achieve pixel-level adaptive adjustment of detail control based on gaze heatmap and virtual-real boundary information, thereby enhancing the rendering accuracy of the virtual-real fusion scene.

[0005] Firstly, this application provides a rendering optimization method for blending virtual and real scenes in cultural and creative products, comprising the following steps: The material information of the virtual model of cultural and creative products is separated into basic color information and detail control information; Construct a visual weighting function whose inputs include gaze heatmap and distance information between virtual and real boundaries; The visual weight function is configured to: obtain the attention weight of the pixel to be rendered in the virtual model of the cultural and creative product based on the gaze heat map, and output a visual contribution adjustment coefficient, which is positively correlated with the attention weight and negatively correlated with the distance from the pixel to be rendered to the virtual-real boundary. Based on the gaze heatmap and the distance between the virtual and real boundaries, the visual contribution adjustment coefficient corresponding to the detail control information is calculated through the visual weighting function. Based on the aforementioned visual contribution adjustment coefficient, the rendering intensity of visual details affected by the detail control information is adaptively adjusted during the rendering process.

[0006] Preferably, the material information of the virtual model of cultural and creative products is separated into basic color information and detail control information, specifically including: Obtain the original material texture data of the virtual model of cultural and creative products; The original material texture data is subjected to multi-scale decomposition processing; Extract low-frequency color components to characterize the overall color distribution as basic color information; High-frequency components used to characterize local texture changes are extracted as detail control information; The basic color information and detail control information are stored in separate material data channels.

[0007] Preferably, obtaining the attention weights of the pixels to be rendered in the virtual model of the cultural and creative product based on the gaze heatmap specifically includes: Obtain the gaze heatmap corresponding to the current rendering viewpoint; The gaze heatmap is mapped to the surface coordinate space of the virtual model of the cultural and creative product. Locate the corresponding position of the pixel to be rendered in the gaze heatmap; Read the hotspot intensity value at the corresponding location; The attention weight of the pixel to be rendered is determined based on the hotspot intensity value.

[0008] Preferably, the visual contribution adjustment coefficient corresponding to the detail control information is calculated using the visual weighting function based on the gaze heatmap and the distance between the real and virtual boundaries, specifically including: Based on the gaze heatmap, obtain the attention weights corresponding to the pixels to be rendered; Obtain the distance between the real and virtual boundaries corresponding to the pixel to be rendered; The attention weights and the distance between the virtual and real boundaries are input as input parameters into the visual weight function; The visual contribution adjustment coefficient corresponding to the pixel to be rendered is calculated; The visual contribution adjustment coefficient is associated with the corresponding detail control information.

[0009] Preferably, based on the visual contribution adjustment coefficient, adaptively adjusting the rendering intensity of visual details affected by the detail control information during the rendering process specifically includes: Identify the detail rendering parameters corresponding to the detail control information during the rendering phase; A visual contribution adjustment coefficient is introduced as an adjustment factor during the detail rendering calculation process; The detail rendering parameters are proportionally adjusted based on the visual contribution adjustment coefficient. Complete the detailed rendering output based on pixel-level adjustment results.

[0010] Preferably, the basic color information is a low-frequency color component extracted from the virtual model material, used to characterize the diffuse reflection color distribution characteristics of the model surface.

[0011] Preferably, the detail control information is a control parameter that characterizes the high-frequency visual components in the material of the virtual model of the cultural and creative product, and is used to describe texture details, surface microstructure undulations and specular response changes.

[0012] Preferably, the gaze heatmap is generated by performing spatial coordinate normalization processing and temporal smoothing on user gaze tracking data, and is used to describe the attention intensity distribution corresponding to each region on the surface of the virtual model.

[0013] Preferably, the virtual-real boundary distance is a spatial distance parameter from the position of the pixel to be rendered on the surface of the virtual model to the preset virtual-real fusion boundary, which is used to characterize the positional relationship of the pixel relative to the virtual-real boundary.

[0014] Secondly, this application provides a rendering optimization system for the fusion of virtual and real scenes in cultural and creative products, including: The material separation module is used to decompose the material information of the virtual model of cultural and creative products into basic color information and detail control information; The visual weight calculation module takes a gaze heatmap and virtual-real boundary distance information as input, and is used to obtain the attention weight of the pixel to be rendered based on the gaze heatmap, and outputs a visual contribution adjustment coefficient that is positively correlated with the attention weight and negatively correlated with the virtual-real boundary distance. The visual contribution mapping module is used to map the visual contribution adjustment coefficients calculated by the visual weight calculation module from the gaze heat map and the distance between the virtual and real boundaries to the adjustment parameters corresponding to the detail control information. The rendering adjustment module is used to perform pixel-level adaptive adjustment of the rendering intensity of visual details affected by detail control information based on the visual contribution adjustment coefficient during the rendering process.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: This technical solution achieves high-precision optimization of virtual-real scene fusion rendering by separating material information from virtual models of cultural and creative products, constructing a visual weight function, and adaptively adjusting pixel-level detail control information. First, material information is decomposed into basic color information and detail control information, allowing low-frequency diffuse reflection attributes and high-frequency textures, micro-surface undulations, and specular highlights to be processed independently during rendering, providing precise data support for pixel-level adjustment. Second, a visual weight function is constructed with gaze heatmaps and virtual-real boundary distance as inputs, quantifying user visual attention behavior and pixel spatial sensitivity in the virtual-real transition area into a visual contribution adjustment coefficient. This allows the detail rendering parameters of each pixel to dynamically respond to attention intensity and boundary sensitivity, thereby achieving directional control of the rendering process. Then, the visual contribution is adjusted... The adjustment coefficient is mapped to detail control information and acts as an adjustment factor in pixel-level rendering calculations to proportionally adjust high-frequency textures and micro-surface details. This enhances details in visually focused areas while moderately reducing details in non-focused areas, thus avoiding local over-rendering or loss of detail. Finally, the full pixel adjustment results are summarized to generate the final rendering output, enhancing the local detail of the virtual model while maintaining a smooth transition between virtual and real boundaries. This ensures that the rendering effect conforms to user gaze guidance while maintaining overall visual realism. In summary, this solution achieves closed-loop control from user perception to rendering output through multi-level collaboration of material separation, weight calculation, and pixel-level adaptive adjustment. This makes the detail presentation in virtual-real fusion scenes more refined and dynamically controllable, improving rendering accuracy and optimizing user visual experience and immersion. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating an application scenario of a rendering optimization method for blending virtual and real scenes in cultural and creative products, as shown in some embodiments of this application. Figure 2 This is an exemplary flowchart of a rendering optimization method for the fusion of virtual and real scenes for cultural and creative products, as shown in some embodiments of this application. Figure 3 This is a schematic flowchart illustrating the determination of attention weights according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a rendering optimization system for the fusion of virtual and real scenes for cultural and creative products, as shown in some embodiments of this application. Figure 5 This is a schematic diagram of the structure of a computer device that implements a rendering optimization method for blending virtual and real scenes for cultural and creative products, according to some embodiments of this application. Detailed Implementation

[0017] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] refer to Figure 1 The figure is a schematic diagram of the application scenario of the virtual and real scene fusion rendering optimization method for cultural and creative products shown in some embodiments of this application, including acquisition equipment, server, communication network and display terminal: the acquisition equipment and display terminal are connected to the server through the communication network.

[0019] The server first acquires the virtual model data, gaze heatmap, and virtual-real boundary distance provided by the acquisition device, and then breaks down the model material into basic color and detail control information. Next, it constructs a visual weight function. Then, it calculates the adjustment coefficients corresponding to the detail control information through this function, adaptively adjusts the detail rendering intensity, and generates an optimized virtual-real fusion rendering result. When it receives a request from the terminal to adjust the display effect, the server feeds back the rendering result, adjustment coefficient index, and detail rendering parameters to the terminal for selection by cultural and creative designers, operators, or users.

[0020] The acquisition equipment includes 3D modeling tools, gaze behavior acquisition systems, etc.; the display terminal can be augmented reality (AR) display equipment, virtual exhibition hall client, etc.; the server can be a local graphics workstation or a cultural and creative cloud distributed service platform.

[0021] In the fusion of virtual and real scenes for cultural and creative products, to enhance user immersion and visual realism, rendering effects not only rely on the geometric accuracy of the virtual model but also need to meticulously present details such as high-frequency textures, micro-surface undulations, and specular highlights. However, existing rendering technologies typically employ globally uniform material parameters or fixed-ratio detail rendering, failing to adjust pixel-level based on user gaze hotspots and virtual-real boundary positions. This results in insufficient or excessive detail representation in visually focused areas and unnatural transitions at the virtual-real fusion boundary. To address these issues, this application provides a rendering optimization method for the fusion of virtual and real scenes in cultural and creative products. By directly mapping user gaze behavior and virtual-real boundary sensitivity to pixel-level rendering adjustments of detail control information, adaptive optimization of detail contribution in virtual-real scenes is achieved, thereby enhancing the rendering accuracy and overall visual effect of virtual models in fused scenes.

[0022] refer to Figure 2 The figure is an exemplary flowchart of a rendering optimization method for blending virtual and real scenes of cultural and creative products, according to some embodiments of this application. The rendering optimization method for blending virtual and real scenes of cultural and creative products mainly includes the following steps: In step 101, the material information of the virtual model of the cultural and creative product is separated into basic color information and detail control information.

[0023] In some embodiments, separating the material information of a virtual model of a cultural and creative product from basic color information and detail control information can be achieved in the following ways: Obtain the original material texture data of the virtual model of cultural and creative products; The original material texture data is subjected to multi-scale decomposition processing; Extract low-frequency color components to characterize the overall color distribution as basic color information; High-frequency components used to characterize local texture changes are extracted as detail control information; The basic color information and detail control information are stored in separate material data channels.

[0024] It should be noted that the original material texture data refers to the basic data set that fully characterizes the surface color, texture, and lighting-related attributes of the virtual model of the cultural and creative product; the multi-scale decomposition processing refers to the decomposition processing method that splits the original material texture data into different frequency components according to different spatial scales; the low-frequency color component is a color information component used to characterize the overall color distribution and macroscopic diffuse reflection characteristics of the virtual model of the cultural and creative product; the basic color information is the low-frequency color component extracted from the virtual model material, used to characterize the diffuse reflection color distribution characteristics of the model surface; the high-frequency component is a material information component used to characterize the local texture changes and detail changes of the virtual model of the cultural and creative product; the detail control information is a control parameter characterizing the high-frequency visual components in the material of the virtual model of the cultural and creative product, used to describe texture details, surface microstructure undulations, and specular response changes.

[0025] In practical applications, firstly, by parsing the material description file or material binding information corresponding to the virtual model of the cultural and creative product, texture resource data associated with the model surface is read. This texture resource data includes two-dimensional texture data describing the color distribution, brightness variations, and detailed structure of the model surface. During the acquisition process, the texture resolution, color space, and coordinate mapping relationship are kept unchanged to ensure that subsequent processing is based on a complete and undistorted material representation. The acquired texture data is then used as the original material texture data. Secondly, multi-scale decomposition processing of the original material texture data can be achieved in the following way: the acquired original material texture data can be used as... The original material texture data is input as a two-dimensional texture, maintaining its original resolution and pixel arrangement. Then, based on a preset spatial scale hierarchy, a layer-by-layer decomposition operation is performed on the original material texture data. For example, by performing multiple smoothing and differencing processes on the texture data, the components that change slowly with spatial location are separated from those that change rapidly. In specific implementations, existing multi-scale analysis methods can be used to construct a coarse-to-fine decomposition hierarchy structure for the original material texture data. The low-scale level is obtained by weighted averaging of adjacent pixel regions to preserve overall color change information, while the high-scale level is obtained by calculating the differences between adjacent scale levels. The decomposition process is used to preserve local texture and detail variation information. The original material texture data is divided into multiple component sets corresponding to different spatial scales. Each component corresponds to texture features at a specific scale, thus completing the multi-scale decomposition of the original material texture data. Then, after completing the multi-scale decomposition, the color data corresponding to the low spatial frequency level is selected from the decomposition results. This part of the data mainly reflects the color change trend over a large area of ​​the material surface without containing obvious local texture fluctuations. By integrating this low-frequency level data, a color data set containing only the overall color distribution features is formed, thus obtaining the basic color information. Finally, in the same multi-scale decomposition results, data corresponding to the high spatial frequency level is selected. This part of the data contains detailed features such as local texture changes, fine structural undulations, and brightness abrupt changes on the material surface. By summarizing and organizing the high-frequency level data, detailed description data independent of the overall color distribution is formed, and detailed control information is obtained. Furthermore, the basic color information and the detailed control information are written into different material data channels respectively. Each material data channel is independent of each other in terms of storage structure and calling interface to ensure that the basic color information and detailed control information can be accessed separately during the rendering process, and to support individual adjustment operations of the detailed control information.

[0026] In step 102, a visual weighting function is constructed, whose inputs include the gaze heatmap and the distance information between the virtual and real boundaries.

[0027] In some embodiments, a visual weight function can be constructed using the following steps: The input parameters of the visual weighting function are determined, including attention weights and the distance between the virtual and real boundaries, wherein the attention weights are obtained from the gaze heatmap; The input parameters are processed to a uniform scale to fit the same computational space; Define the mapping relationship of the visual weight function so that its output is a single visual contribution adjustment coefficient; Configure the visual weight function as a calculation module for pixel-level rendering control.

[0028] It should be noted that the virtual-real boundary distance is the spatial distance parameter from the position of the pixel to be rendered on the surface of the virtual model to the preset virtual-real fusion boundary, which is used to characterize the positional relationship of the pixel relative to the virtual-real boundary; the visual weight function is a function used to map the pixel-level attention weight and the virtual-real boundary distance into a single visual contribution adjustment coefficient used to adjust the rendering intensity of detail control information.

[0029] In practical applications, firstly, during pixel-level rendering, the attention weight and virtual-real boundary distance are obtained for each pixel to be rendered. The attention weight is derived from the value of the gaze heatmap at that pixel location, and the virtual-real boundary distance is derived from a pre-calculated and stored virtual-real boundary distance field. Then, to facilitate joint calculation of parameters with different physical meanings within the same computational space, the attention weight and virtual-real boundary distance are linearly normalized to map them to a unified numerical range. Based on this, following a common weighted mapping modeling method, a function mapping relationship is pre-defined, so that the attention weight participates in the calculation as a positive weight and the virtual-real boundary distance participates in the calculation as a decay factor. This causes the function output to increase with the increase of the attention weight and decrease with the increase of the virtual-real boundary distance. The mapping relationship can be set during the rendering initialization stage through fixed parameter configuration. Finally, the function is encapsulated into a calculation module that can be called pixel by pixel during the rendering process, so that each pixel can call the visual weight function before performing detailed rendering calculations and output the corresponding visual contribution adjustment coefficient, thereby completing the construction and configuration of the visual weight function.

[0030] Preferably, in other embodiments, the visual weighting function can be expressed by the following formula:

[0031] Wherein, W is the visual contribution adjustment coefficient, an adjustment parameter used to control the intensity of detail control information's participation in pixel-level rendering; A is the attention weight, a weight parameter used to characterize the attention intensity of the pixel to be rendered at its corresponding position in the gaze heatmap, reflecting the user's perceptual attention to that pixel region; D is the virtual-real boundary distance, a distance parameter used to characterize the spatial distance from the pixel to be rendered on the virtual model surface to the virtual-real fusion boundary; and k is the weight scaling coefficient, a proportional parameter used to uniformly adjust the overall amplitude of the visual contribution adjustment coefficient, used to adapt to different rendering scenarios or different material detail intensity requirements. In specific implementation, the weight scaling coefficient k can be linearly calculated based on the global material brightness range, the upper limit of detail component intensity, and the maximum value of the pixel-level attention weight in the current rendering scene. The maximum visual contribution adjustment value required for rendering is divided by the theoretical maximum value of the attention weight to obtain a uniform scaling coefficient, so that in pixel-level calculations, regardless of the original attention weight value... The product of these parameters all falls within the preset usable range. λ is the distance attenuation coefficient, an adjustment parameter used to control the influence of the distance between the virtual and real boundaries on the visual contribution adjustment coefficient. Its value determines the attenuation rate of the visual contribution adjustment coefficient as the distance increases. In practice, the distance attenuation coefficient λ can be calculated by linearly or exponentially fitting the minimum and maximum pixel distance range of the virtual-real fusion boundary and the desired attenuation rate in rendering. This ensures that the attenuation effect is minimal when the pixel is close to the virtual-real boundary, and significant when the pixel is far from the boundary, thus achieving boundary-sensitive detail control. It should be further explained that, in principle, the calculation of the weight scaling coefficient ensures that the linear mapping relationship between the visual contribution adjustment coefficient and the attention weight is controllable, while the calculation of the distance attenuation coefficient achieves boundary-sensitive adjustment through the coupling of spatial distance and attenuation function. This allows the visual weight function to reflect both user attention and virtual-real fusion position in rendering, applying both to pixel-level rendering calculations as visual contribution adjustment factors.

[0032] Preferably, in some embodiments, the gaze heatmap can be generated by performing spatial coordinate normalization and temporal smoothing on user gaze tracking data. It is used to describe the attention intensity distribution corresponding to different regions of the virtual model surface. Specifically, the generation of the gaze heatmap can be accomplished according to the following technical steps: First, collect the user's gaze coordinate data in the virtual scene using existing gaze tracking devices, and map and record the texture coordinates or 3D surface coordinates corresponding to each sampling point on the virtual model surface; second, normalize the mapped coordinate data in the spatial dimension, uniformly mapping all gaze positions to the virtual model surface. Within the texture space or discrete pixel grid, the influence of different resolutions or model sizes on the coordinate distribution is eliminated. Then, the continuously sampled gaze data is smoothed in the time dimension, using moving average or Gaussian time-weighted filtering to suppress short-term jitter and noise while preserving attention intensity information in continuous gaze regions. Finally, the normalized and smoothed gaze point information is accumulated at the corresponding positions on the virtual model surface to form a gaze intensity distribution map, and normalized or standardized according to the pixel-level distribution to generate attention weights corresponding to each pixel. The final processing result is used as a gaze heatmap for visual contribution adjustment calculation in pixel-level rendering.

[0033] In step 103, the visual weight function is configured to: obtain the attention weight of the pixel to be rendered in the virtual model of the cultural and creative product based on the gaze heat map, and output a visual contribution adjustment coefficient, which is positively correlated with the attention weight and negatively correlated with the distance from the pixel to be rendered to the virtual-real boundary.

[0034] In some embodiments, reference Figure 3 As shown in the figure, this is a flowchart illustrating the process of determining attention weights in some embodiments of this application. In this embodiment, obtaining the attention weights of pixels to be rendered in the virtual model of cultural and creative products based on the gaze heatmap can be achieved through the following steps: In step 1031, the gaze heatmap corresponding to the current rendering viewpoint is obtained; In step 1032, the gaze heatmap is mapped to the surface coordinate space of the virtual model of the cultural and creative product; In step 1033, the corresponding position of the pixel to be rendered in the gaze heat map is located; In step 1034, the hotspot intensity value at the corresponding location is read; In step 1035, the attention weight of the pixel to be rendered is determined based on the hotspot intensity value.

[0035] It should be noted that the surface coordinate space is used to map the three-dimensional surface points of the virtual model to a unified rendering coordinate or texture coordinate system; the hotspot intensity value is a numerical parameter used to quantify the degree of user visual attention reflected by the gaze heatmap at a specific location; and the attention weight is a weight parameter used to adjust the visual contribution of detail control information during pixel-level rendering, reflecting the degree of user attention to a single pixel.

[0036] In specific implementation, firstly, mapping the gaze heatmap to the surface coordinate space of the virtual model of the cultural and creative product can be achieved in the following way: firstly, obtain the surface coordinate information of the virtual model, including vertex positions and texture coordinates; then, convert the coordinate values ​​of each pixel or mesh unit in the gaze heatmap into the corresponding three-dimensional position or texture coordinates on the model surface through a mapping relationship, so as to achieve a one-to-one correspondence between the heatmap and the model surface. Secondly, locating the corresponding position of the pixel to be rendered in the gaze heatmap can be achieved in the following way: for each pixel to be rendered, use its coordinates in the virtual model surface or texture space to find and determine the corresponding pixel or mesh unit in the mapped gaze heatmap, and use spatial indexing. Each pixel to be rendered is uniquely associated with its corresponding position in the gaze heatmap. The heatmap intensity value at the corresponding position can then be read by accessing the located pixel or grid cell in the gaze heatmap to obtain its stored heatmap intensity value. This value represents the user's visual attention at that position, and the reading operation can be performed through pixel value indexing or grid cell lookup. Finally, the attention weight of the pixel to be rendered can be determined based on the heatmap intensity value by normalizing the heatmap intensity value to fit the input range of the visual weight function, ensuring that attention differences between different pixels are preserved, and then using the processing result as the pixel-level attention weight parameter.

[0037] In step 104, based on the gaze heatmap and the distance between the virtual and real boundaries, the visual contribution adjustment coefficient corresponding to the detail control information is calculated using the visual weight function.

[0038] In some embodiments, calculating the visual contribution adjustment coefficient corresponding to the detail control information based on the gaze heatmap and the distance between the virtual and real boundaries using the visual weighting function specifically includes: Based on the gaze heatmap, obtain the attention weights corresponding to the pixels to be rendered; Obtain the distance between the real and virtual boundaries corresponding to the pixel to be rendered; The attention weights and the distance between the virtual and real boundaries are input as input parameters into the visual weight function; The visual contribution adjustment coefficient corresponding to the pixel to be rendered is calculated; The visual contribution adjustment coefficient is associated with the corresponding detail control information.

[0039] It should be noted that the visual contribution adjustment coefficient is a numerical parameter used to adjust the intensity of the contribution of detail control information to the pixel visual effect during the rendering process; the detail control information is a control parameter that characterizes the high-frequency visual components in the material of the virtual model of cultural and creative products.

[0040] In practical applications, firstly, based on the gaze heatmap, the attention weights corresponding to the pixels to be rendered can be obtained in the following way: The position of the pixel to be rendered on the model surface or in texture space is matched with the gaze heatmap using the previously generated gaze heatmap. The numerical information of the corresponding position in the gaze heatmap is accessed, and this value is normalized as necessary to adapt to subsequent function inputs. The obtained value is then used as the attention weight. Secondly, the distance between the virtual and real boundaries corresponding to the pixels to be rendered can be obtained in the following way: The position of the virtual-real fusion boundary on the model surface or in texture space is determined, and then the shortest spatial distance from the pixel to be rendered to this boundary is calculated. This distance is obtained through the difference in spatial coordinates between pixels or the difference in mesh indexes. Then, the shortest spatial distance is used as the virtual-real boundary distance corresponding to the pixel to be rendered; further, the attention weight and the virtual-real boundary distance are input as input parameters to the visual weight function; then, the pre-set visual weight function is called, and the attention weight and the virtual-real boundary distance are input into the visual weight function to calculate the visual contribution adjustment coefficient corresponding to the pixel to be rendered; finally, associating the visual contribution adjustment coefficient with the corresponding detail control information means mapping the visual contribution adjustment coefficient calculated for each pixel to its corresponding detail control information, and applying the coefficient as an adjustment factor in the rendering pipeline to the rendering calculation of high-frequency textures, micro-surface undulations or specular changes of materials, to achieve pixel-level detail visual control.

[0041] In step 105, based on the visual contribution adjustment coefficient, the rendering intensity of the visual details affected by the detail control information is adaptively adjusted during the rendering process.

[0042] In some embodiments, adaptively adjusting the rendering intensity of visual details affected by the detail control information during the rendering process, based on the visual contribution adjustment coefficient, specifically includes: Identify the detail rendering parameters corresponding to the detail control information during the rendering phase; A visual contribution adjustment coefficient is introduced as an adjustment factor during the detail rendering calculation process; The detail rendering parameters are proportionally adjusted based on the visual contribution adjustment coefficient. Complete the detailed rendering output based on pixel-level adjustment results.

[0043] It should be noted that the detailed rendering parameters refer to adjustable material parameters used to control the intensity of pixel-level high-frequency textures, micro-surface undulations, and specular changes during the rendering process.

[0044] In practical applications, adaptive adjustment of visual details influenced by detail control information based on the visual contribution adjustment coefficient can be achieved step-by-step on the basis of the existing pixel-level rendering pipeline through the following steps: First, in the rendering preparation stage, the materials of the virtual model of the cultural and creative product are initialized, and the detail control information corresponding to each pixel is mapped to adjustable detail rendering parameters, including high-frequency texture coefficients, micro-surface undulation amplitude, and specular reflection intensity, etc., and a parameter index is established for each pixel so that these parameters can be accessed and modified pixel by pixel during the rendering calculation stage; Second, in the actual rendering calculation process, for each pixel to be rendered, its corresponding visual contribution adjustment coefficient is first obtained from the visual weight function, and then... The coefficients, introduced as adjustment factors in the calculation of detail rendering parameters, can be achieved through linear multiplication or weighted superposition in existing graphics rendering techniques. This allows parameters such as high-frequency texture, micro-surface undulations, and specular reflection of pixels to be scaled proportionally according to their visual contribution adjustment coefficients. Then, the adjusted detail rendering parameters are input into the rendering equation for pixel-level calculations, including lighting calculations, shadow processing, and high-frequency detail superposition, thereby generating the final color value and high-frequency detail contribution for each pixel. Finally, after the full pixel calculation is completed, the adjustment results of all pixels are summarized to form a complete detail rendering output. This output visually reflects both the influence of user gaze hotspots and the sensitivity of the virtual-real boundary, achieving pixel-level adaptive detail rendering.

[0045] It should be noted that the proposed solution introduces a visual contribution adjustment coefficient into the pixel-level detail rendering process, thereby achieving adaptive adjustment of detail control information such as high-frequency textures, micro-surface undulations, and specular changes. Compared with the existing technology that typically uses fixed or globally uniform intensity processing for detail rendering parameters, this solution can solve the problem that the impact of the user's gaze focus and the position of the virtual-real boundary on visual perception is not quantified. By introducing adjustment factors at the pixel level and adjusting detail rendering parameters proportionally, this solution can achieve dynamic changes in detail intensity with user attention and sensitivity to virtual-real boundaries, significantly improving the visual realism and attention guidance effect of the rendering, thereby achieving the technical effect of enhancing the perception of virtual-real fusion and optimizing the performance of local visual details.

[0046] On the other hand, in some embodiments, this application provides a rendering optimization system for the fusion of virtual and real scenes in cultural and creative products, with reference to... Figure 4The figure is a schematic diagram of the structure of a rendering optimization system for the fusion of virtual and real scenes for cultural and creative products, according to some embodiments of this application. The rendering optimization system 400 for the fusion of virtual and real scenes for cultural and creative products includes: a material separation module 401, a visual weight calculation module 402, a visual contribution mapping module 403, and a rendering adjustment module 404, which are described below: Material separation module 401, in this application, is mainly used to decompose the material information of the virtual model of cultural and creative products into basic color information and detail control information; The visual weight calculation module 402 in this application takes a gaze heat map and virtual-real boundary distance information as input, and is used to obtain the attention weight of the pixel to be rendered based on the gaze heat map, and output a visual contribution adjustment coefficient that is positively correlated with the attention weight and negatively correlated with the virtual-real boundary distance. The visual contribution mapping module 403 in this application is mainly used to map the visual contribution adjustment coefficient calculated by the visual weight calculation module from the gaze heat map and the distance between the virtual and real boundaries to the adjustment parameters corresponding to the detail control information. The rendering adjustment module 404 in this application is mainly used to perform pixel-level adaptive adjustment of the rendering intensity of visual details affected by the detail control information according to the visual contribution adjustment coefficient during the rendering process.

[0047] In addition, this application also provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to acquire the code and execute the above-described rendering optimization method for the fusion of virtual and real scenes for cultural and creative products.

[0048] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a rendering optimization method for blending virtual and real scenes of cultural and creative products, according to some embodiments of this application. The rendering optimization method for blending virtual and real scenes of cultural and creative products in the above embodiments can be achieved through... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0049] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0050] The communication bus 502 can be used to transmit information between the aforementioned components.

[0051] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0052] The memory 503 stores program code for executing the solution of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The rendering optimization method for virtual-real scene fusion of cultural and creative products in the above embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0053] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0054] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0055] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0056] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described rendering optimization method for the fusion of virtual and real scenes in cultural and creative products.

[0057] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0058] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A rendering optimization method for virtual-real scene fusion of a creative product, characterized in that, include: The material information of the virtual model of cultural and creative products is separated into basic color information and detail control information; Construct a visual weighting function whose inputs include gaze heatmap and distance information between virtual and real boundaries; The visual weight function is configured to: obtain the attention weight of the pixel to be rendered in the virtual model of the cultural and creative product based on the gaze heat map, and output a visual contribution adjustment coefficient, which is positively correlated with the attention weight and negatively correlated with the distance from the pixel to be rendered to the virtual-real boundary. Based on the gaze heatmap and the distance between the virtual and real boundaries, the visual contribution adjustment coefficient corresponding to the detail control information is calculated through the visual weighting function. Based on the aforementioned visual contribution adjustment coefficient, the rendering intensity of visual details affected by the detail control information is adaptively adjusted during the rendering process.

2. The method of claim 1, wherein, The material information of the virtual model of cultural and creative products is separated into basic color information and detail control information, specifically including: Obtain the original material texture data of the virtual model of cultural and creative products; The original material texture data is subjected to multi-scale decomposition processing; Extract low-frequency color components to characterize the overall color distribution as basic color information; High-frequency components used to characterize local texture changes are extracted as detail control information; The basic color information and detail control information are stored in separate material data channels.

3. The method of claim 1, wherein, Obtaining the attention weights of pixels to be rendered in the virtual model of cultural and creative products based on the gaze heatmap specifically includes: Obtain the gaze heatmap corresponding to the current rendering viewpoint; The gaze heatmap is mapped to the surface coordinate space of the virtual model of the cultural and creative product. Locate the corresponding position of the pixel to be rendered in the gaze heatmap; Read the hotspot intensity value at the corresponding location; The attention weight of the pixel to be rendered is determined based on the hotspot intensity value.

4. The method of claim 1, wherein, Based on the gaze heatmap and the distance between the real and virtual boundaries, the visual contribution adjustment coefficient corresponding to the detail control information is calculated through the visual weighting function, specifically including: Based on the gaze heatmap, obtain the attention weights corresponding to the pixels to be rendered; Obtain the distance between the real and virtual boundaries corresponding to the pixel to be rendered; The attention weights and the distance between the virtual and real boundaries are input as input parameters to the visual weight function; The visual contribution adjustment coefficient corresponding to the pixel to be rendered is calculated; The visual contribution adjustment coefficient is associated with the corresponding detail control information.

5. The method of claim 1, wherein, Based on the aforementioned visual contribution adjustment coefficient, the adaptive adjustment of the rendering intensity of visual details affected by the detail control information during the rendering process specifically includes: Identify the detail rendering parameters corresponding to the detail control information during the rendering phase; A visual contribution adjustment coefficient is introduced as an adjustment factor during the detail rendering calculation process; The detail rendering parameters are proportionally adjusted based on the visual contribution adjustment coefficient. Complete the detailed rendering output based on pixel-level adjustment results.

6. The method of claim 1, wherein, The basic color information consists of low-frequency color components extracted from the virtual model material, used to characterize the diffuse reflection color distribution features of the model surface.

7. The method of claim 1, wherein, The detailed control information refers to the control parameters that characterize the high-frequency visual components in the material of the virtual model of cultural and creative products, and are used to describe texture details, surface microstructure undulations, and specular response changes.

8. The method as described in claim 1, characterized in that, The gaze heatmap is generated by performing spatial coordinate normalization and temporal smoothing on user gaze tracking data, and is used to describe the distribution of attention intensity in different regions of the virtual model surface.

9. The method as described in claim 1, characterized in that, The virtual-real boundary distance is a spatial distance parameter from the position of the pixel to be rendered on the surface of the virtual model to the preset virtual-real fusion boundary, which is used to characterize the positional relationship of the pixel relative to the virtual-real boundary.

10. A rendering optimization system for blending virtual and real scenes in cultural and creative products, characterized in that, include: The material separation module is used to decompose the material information of the virtual model of cultural and creative products into basic color information and detail control information; The visual weight calculation module takes a gaze heatmap and virtual-real boundary distance information as input, and is used to obtain the attention weight of the pixel to be rendered based on the gaze heatmap, and outputs a visual contribution adjustment coefficient that is positively correlated with the attention weight and negatively correlated with the virtual-real boundary distance. The visual contribution mapping module is used to map the visual contribution adjustment coefficients calculated by the visual weight calculation module from the gaze heat map and the distance between the virtual and real boundaries to the adjustment parameters corresponding to the detail control information. The rendering adjustment module is used to perform pixel-level adaptive adjustment of the rendering intensity of visual details affected by detail control information based on the visual contribution adjustment coefficient during the rendering process.

Images