A method and system for video view point adaptive rendering and detail enhancement

Abstract

The application provides a video viewpoint adaptive rendering and detail enhancement method and system, relates to the technical field of digital twinning, and obtains rendering related data of an interactive video of a digital twinning scene in a rendering process; based on texture resource data in the rendering related data, an optimization scheduling scheme of rendering resources is determined, based on topology data, an association reference of the digital twinning scene is determined; based on the optimization scheduling scheme and the association reference, the digital twinning scene is rendered by using a ray tracing technology, scene image data is generated; according to the scene image data, a target image sequence is generated, the target image sequence is compressed and associated by using a video coding technology, and a rendered video is generated, so that video rendering and detail enhancement of the digital twinning scene are realized, and high-fidelity detail restoration and visually coherent video rendering in the interactive digital twinning scene are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital twinning, in particular to a video viewpoint adaptive rendering and detail enhancement method and system. BACKGROUND

[0002] In the application scenarios of digital twinning such as smart city, industrial simulation and remote operation, users have higher requirements for the real-time performance, realism and detail fidelity of video presentation. The system not only needs to efficiently restore the spatial structure and surface material of the physical entity, but also needs to maintain high frame rate and low delay in the dynamic interaction process, while ensuring that the texture details of the key area are not lost due to compression or rendering simplification, which constitutes a comprehensive technical challenge to the resource scheduling, geometric correlation modeling and image enhancement capabilities in the video rendering process.

[0003] The current scheme adopts an offline rendering strategy based on pre-baked lighting and static resource binding. When building a digital twinning scene, a light map is pre-generated and the texture allocation is fixed, and then the video frame synthesis is combined with traditional rasterization methods. However, this scheme has some significant defects, for example, it is difficult to adapt to the rendering requirements brought by sudden changes in viewing angle or dynamic lighting in the interaction process, and the preset resources cannot flexibly respond to real-time texture calls, resulting in missing details or redundant loading; at the same time, the static topological correlation ignores the dynamic semantic relationship between scene elements, causing structural misalignment or material misplacement in the picture under complex operations, affecting the overall visual coherence and information accuracy. SUMMARY

[0004] The purpose of the present application is to provide a video viewpoint adaptive rendering and detail enhancement method and system to solve the problem of missing details and structural misalignment caused by the inability of preset resources and static topology to adapt to dynamic interaction in the prior art.

[0005] To solve the above technical problems, in a first aspect, the present application provides a video viewpoint adaptive rendering and detail enhancement method, comprising:

[0006] Obtaining rendering-related data in the rendering process of an interactive video of a digital twinning scene, the rendering-related data including texture resource data and topological data;

[0007] Based on the texture resource data in the rendering-related data, determining an optimization scheduling scheme of rendering resources, and based on the topological data, determining an association reference of the digital twinning scene;

[0008] Based on the optimization scheduling scheme and the association reference, performing ray tracing rendering on the digital twinning scene using a ray tracing technology to generate scene image data;

[0009] According to the scene image data, a target image sequence is generated, the target image sequence is compressed and associated by using a video coding technology, and a rendered video is generated to realize video rendering and detail enhancement of the digital twin scene.

[0010] Optionally, based on the optimized scheduling scheme and the association benchmark, a digital twin scene is rendered by using a ray tracing technology to generate scene image data, including:

[0011] Based on the association benchmark, a set of scene entities that need to be associated in the rendering process is selected, and an association trigger relationship between the scene entities in the set is determined.

[0012] According to the texture-related information of each scene entity in the texture resource data and the spatial distribution rule in the topology data, the propagation and interaction of light on the surface of each scene entity are rendered by using a ray tracing technology to generate basic rendering data of each scene entity.

[0013] According to the association trigger relationship, mutual influence data between the scene entities in the set is generated.

[0014] Based on the basic rendering data of each scene entity and the corresponding mutual influence data, scene image data corresponding to each image frame is generated.

[0015] Optionally, according to the texture-related information of each scene entity in the texture resource data and the spatial distribution rule in the topology data, the propagation and interaction of light on the surface of each scene entity are rendered by using a ray tracing technology to generate basic rendering data of each scene entity, including:

[0016] The spatial position coordinates, shape structure information, and relative distance from surrounding entities of each scene entity are extracted from the topology data.

[0017] According to the texture-related information of each scene entity in the texture resource data, the ray attribute information of each texture is determined.

[0018] According to the spatial position coordinates, shape structure information, and relative distance from surrounding entities of each scene entity, the ray attribute information of each texture is combined to determine the ray coverage range of each scene entity.

[0019] Taking a preset starting position as a starting point, the propagation path of the propagation of light in the ray coverage range is rendered by using a ray tracing technology, and propagation path data is recorded.

[0020] According to the ray attribute information, the interaction of light on the surface of each scene entity is rendered by using a ray tracing technology, and interaction data of light and the surface of each scene entity is recorded.

[0021] The propagation path data and the interaction data of each scene entity are integrated to form basic rendering data.

[0022] Optionally, according to the associated trigger relationship, mutual influence data between each scene entity in the set of scene entities is generated, including:

[0023] According to the associated trigger relationship, a target scene entity pair in the set of scene entities is determined, and an influence transmission direction of each target scene entity pair is determined;

[0024] According to the corresponding light attribute information of each target scene entity pair, a ray tracing technology is used to render the propagation of light from the surface of the scene entity of the emission end to the surface of the scene entity of the receiving end, and energy change information of the light is recorded;

[0025] According to the shape structure information, the spatial position coordinates of the scene entity of the receiving end, and the energy change information, a first influence detail generated by the light on the surface of the scene entity of the receiving end is obtained;

[0026] If there is an indirect influence path between each target scene entity pair, a second influence detail corresponding to the indirect propagation path in the preset influence transmission direction is obtained;

[0027] At least one of the first influence detail and the second influence detail is integrated to generate mutual influence data between each scene entity in the set of scene entities.

[0028] Optionally, based on the texture resource data in the rendering related data, an optimization scheduling scheme of rendering resources is determined, including:

[0029] According to the texture resource data in the rendering related data, in combination with the number of scene entities and the number of texture applications corresponding to each image frame in the interactive video, the rendering complexity of each image frame is determined;

[0030] The repetition frequency and matching degree of different texture related information in the texture resource data in adjacent image frames are compared to calculate a target matching degree of each texture related information;

[0031] According to the generation order of each image frame in the rendering process of the interactive video, the calling frequency of different texture related information in the rendering process is counted;

[0032] Based on the texture resource data, the rendering complexity, the target matching degree, and the calling frequency, an optimization scheduling scheme of rendering resources is determined.

[0033] Optionally, based on the topology data, an associated reference of the digital twin scene is determined, including:

[0034] extracting spatial distribution rules of each scene entity and connection tightness between scene entities from the topological data, combining state change trends of each scene entity in a rendering process of the interactive video, and calculating a correlation coefficient of each scene entity;

[0035] determining a rendering weight value of each scene entity according to functional importance and rendering priority requirements of each scene entity in the digital twin scene;

[0036] correlating and matching the spatial distribution rules, the connection tightness, the correlation coefficient, and the rendering weight value of each scene entity according to structure requirements of the digital twin scene, to determine an associated priority of each scene entity in the rendering process;

[0037] determining an interaction range between each scene entity based on the associated priority and the spatial distribution rules of the scene entity, and combining the rendering weight value to determine an associated threshold between each scene entity;

[0038] integrating the associated priority of each scene entity, the interaction range between each scene entity, and the associated threshold to form an associated benchmark of the digital twin scene.

[0039] Optionally, according to the scene image data, a target image sequence is generated, and a video encoding technology is used to compress and correlate the target image sequence to generate a rendered video, including:

[0040] adjusting the scene image data according to the image frame time sequence of the interactive video and a preset playing requirement to generate a target image sequence;

[0041] determining a playing adaptation parameter corresponding to the target image sequence, and using a video encoding technology to compress and correlate the target image sequence according to the playing adaptation parameter to generate an encoded video stream;

[0042] if there is audio data matched with the interactive video, integrating and packaging the encoded video stream and the audio data to form a rendered video, and if there is no audio data matched with the interactive video, packaging the encoded video stream to form a rendered video.

[0043] In a second aspect, the present application provides a video viewpoint adaptive rendering and detail enhancement system, including:

[0044] an acquisition module configured to acquire rendering related data of an interactive video of a digital twin scene in a rendering process, the rendering related data including texture resource data and topological data;

[0045] determining a resource optimization scheduling scheme of rendering resources based on the texture resource data in the rendering-related data, and determining a correlation benchmark of the digital twin scene based on the topology data;

[0046] rendering the digital twin scene by using a ray tracing technology based on the resource optimization scheduling scheme and the correlation benchmark, to generate scene image data;

[0047] generating a target image sequence according to the scene image data, compressing and correlating the target image sequence by using a video encoding technology, and generating a rendered video to realize video rendering and detail enhancement of the digital twin scene.

[0048] In a third aspect, the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to realize the steps of the video viewpoint adaptive rendering and detail enhancement method according to the first aspect.

[0049] In a fourth aspect, the present application provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is executed by a computer to realize the steps of the video viewpoint adaptive rendering and detail enhancement method according to the first aspect.

[0050] The video viewpoint adaptive rendering and detail enhancement method provided by the present application can dynamically formulate a resource scheduling strategy that fits the current interactive state and establish a precise scene structure correlation benchmark by obtaining texture resource data and topology data in the interactive video rendering process of the digital twin scene; on this basis, the ray tracing technology is introduced for rendering, which improves the realism of material performance and the accuracy of light response; then the generated image sequence is subjected to targeted video encoding and correlation processing, which guarantees visual coherence while taking into account transmission efficiency, thereby realizing high-fidelity, low-redundancy and rapid response video rendering and detail enhancement effect under complex interactive conditions.

[0051] Further, by identifying the entity set that needs to be cooperatively updated and its trigger relationship according to the correlation benchmark of the digital twin scene, and combining the texture information and the real propagation behavior of light on the surface of the entity, the basic rendering data is generated, and the mutual influence information between entities is fused to form the final image frame; the material drift or geometric misplacement problem caused by static binding is overcome, and the stability of the picture structure and the integrity of the detail expression under complex interaction are enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings described below are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative labor on the basis of these drawings also belong to the protection scope of the present application.

[0053] Figure 1 A flowchart of a video viewpoint adaptive rendering and detail enhancement method provided by an embodiment of the present application is shown in the figure.

[0054] Figure 2 A specific implementation diagram of a video viewpoint adaptive rendering and detail enhancement method provided by an embodiment of the present application is shown in the figure.

[0055] Figure 3 A structure diagram of a video viewpoint adaptive rendering and detail enhancement system provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0056] In view of the problem that the existing digital twin video rendering scheme is difficult to cope with the detail loss and structure misalignment caused by sudden changes in viewing angle, dynamic changes in light and complex correlations between scene elements in the interactive process due to the dependence on preset light and static resource binding, the present application provides a video viewpoint adaptive rendering and detail enhancement method. The core idea of the method is to: synchronously collect texture resources and topological structure information, respectively construct a dynamic optimization strategy for resource scheduling and a correlation benchmark for scene understanding, introduce a ray tracing technology to realize a more realistic restoration of materials and light, and combine video coding to compress and semantically associate the image sequence, so as to improve the picture detail fidelity and overall consistency while ensuring the rendering efficiency, and meet the high-fidelity and strong-interactive video presentation requirements in the digital twin scene.

[0057] In order to enable the person skilled in the art to better understand the present application, the present application will be further described in detail below in combination with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor also belong to the protection scope of the present application.

[0058] The core of the present application is to provide a video viewpoint adaptive rendering and detail enhancement method, and a flowchart of a specific embodiment of the method is shown in the figure. Figure 1 The method comprises the following steps.

[0059] Step 101: acquiring rendering related data of an interactive video of a digital twin scene in a rendering process, wherein the rendering related data comprises texture resource data and topological data.

[0060] In this step, the digital twin scene refers to a virtual simulation scene constructed based on the physical characteristics, spatial relationships and running states of the real scene.

[0061] The rendering-related data refers to a core data set required in the interactive video rendering process to support the rendering effect and efficiency.

[0062] The texture resource data refers to related information extracted from the surface of the scene entity for presenting the texture, color characteristics and detail identification of the entity surface, which includes the material texture and color parameters of the entity surface.

[0063] The topology data refers to a data set recording the spatial position information, shape structure information and connection relationship and distribution characteristics of the scene entity, which includes the spatial coordinates, relative distance and connection tightness of the entity.

[0064] In the embodiments of the present application, first, the preset range of the digital twin scene is determined to determine the scene entities that need to participate in the interactive video rendering, and the texture-related information of the surface of each scene entity is collected, and the spatial position information, shape structure information and connection relationship and distribution characteristics of each scene entity are collected. Finally, the texture-related information is integrated to form the texture resource data, and the collected spatial position information, shape structure information and other information are integrated to form the topology data, and finally the rendering-related data is obtained.

[0065] Step 102: determining an optimization scheduling scheme of rendering resources based on the texture resource data in the rendering-related data, and determining an association reference of the digital twin scene based on the topology data.

[0066] In this step, the optimization scheduling scheme of the rendering resources refers to a scheme of reasonably allocating the calculation resources, storage resources and calling sequence according to the characteristics of the texture resource data and the rendering requirements.

[0067] The association reference refers to a standard set constructed based on the topology data for defining the linkage rendering rules between the scene entities.

[0068] In the embodiments of the present application, the optimization scheduling scheme of the rendering resources is determined based on the texture resource data in the rendering-related data, which can specifically include the following steps:

[0069] Step 201: determining the rendering complexity of each image frame according to the texture resource data in the rendering-related data, in combination with the number of scene entities and the number of texture applications corresponding to each image frame in the interactive video.

[0070] In this step, the scene entity quantity refers to the number of scene entities participating in interactive video rendering in each image frame, used to reflect the entity scale of single-frame rendering; the texture application quantity refers to the total number of texture-related information applied by each scene entity in each image frame, including the number of repeatedly applied textures; the rendering complexity can be understood as an index for measuring the amount of calculation and processing difficulty required for single-frame image rendering, which is determined comprehensively based on the scene entity quantity and the texture application quantity.

[0071] In the embodiments of the present application, the types and characteristics of each texture-related information are extracted from the texture resource data; then each image frame of the interactive video is traversed, the scene entity quantity corresponding to each image frame is counted, and the number of texture-related information applied by each scene entity in each image frame is counted, to obtain the texture application quantity of each image frame; finally, the scene entity quantity and the texture application quantity are comprehensively analyzed according to the complexity of the textures in the texture resource data, to determine the rendering complexity of each image frame.

[0072] Step 202: Comparing the repetition frequency and matching degree of different texture-related information in adjacent image frames in the texture resource data to calculate the target matching degree of each texture-related information.

[0073] In this step, the texture-related information refers to information extracted from the surface of a scene entity for presenting the texture, color characteristics and detail identification of the entity surface, including material texture, color parameters, etc.

[0074] The repetition frequency refers to the number of times that the same texture-related information appears in two adjacent image frames.

[0075] The matching degree refers to the feature coincidence degree of the same texture-related information in adjacent image frames, which includes the consistency degree of texture size, color parameters, etc.

[0076] The target matching degree refers to a quantitative index for measuring the adaptability of texture-related information in adjacent image frames.

[0077] In the embodiments of the present application, first, adjacent two image frames are compared in sequence according to the generation order of each image frame in the rendering process of the interactive video, and the repetition frequency of each texture-related information in the texture resource data in adjacent image frames is counted; then the matching degree of the same texture-related information in adjacent image frames is analyzed through feature comparison; finally, the repetition frequency and the matching degree are weighted and summed with equal weight to obtain the target matching degree of each texture-related information.

[0078] Step 203: According to the generation order of each image frame in the rendering process of the interactive video, the calling frequency of different texture-related information in the rendering process is counted.

[0079] In this step, the call frequency refers to the total number of times that the same texture-related information is called by each image frame in the entire interactive video rendering process.

[0080] In the embodiments of the present application, first, all image frames are traversed in the generation order of the interactive video, and the texture-related information applied in each image frame is recorded, and the total number of calls of each texture-related information in all image frames is counted, that is, the call frequency of each texture-related information is obtained.

[0081] Step 204: Based on the texture resource data, rendering complexity, target matching degree, and call frequency, an optimized scheduling scheme of rendering resources is determined.

[0082] In the embodiments of the present application, first, according to the texture resource data and the rendering complexity of each image frame, the rendering complexity is divided into levels, and the corresponding computing resource allocation mode is matched according to different levels. A specific allocation mode can be: for image frames with high rendering complexity, more computing resources are allocated in combination with the number of textures applied thereto and the complexity of the textures in the texture resource data.

[0083] On this basis, according to the importance of the textures in the texture resource data, a weight is determined, the target matching degree and the call frequency of each texture-related information are weighted and summed according to the weight, a comprehensive score is obtained, and textures with a comprehensive score higher than a preset score are screened out. A cache mechanism is established for such textures to reduce repeated loading loss. Then, taking the comprehensive score as the core basis, the texture loading sequence is formulated in combination with the loading characteristics of the textures in the texture resource data. Finally, the above computing resource allocation mode, cache mechanism, and texture loading sequence are integrated to obtain an optimized scheduling scheme of rendering resources.

[0084] Based on the topology data, the correlation reference of the digital twin scene is determined. This step can specifically include the following steps:

[0085] Step 211: The spatial distribution rule of each scene entity and the connection tightness between scene entities are extracted from the topology data, and the correlation coefficient of each scene entity is calculated in combination with the state change trend of each scene entity in the rendering process of the interactive video.

[0086] In this step, the spatial distribution rule refers to the spatial position arrangement characteristics and distribution mode of the scene entity in the digital twin scene. The spatial distribution rule includes the coordinate distribution, arrangement order, etc. of the entity.

[0087] The connection tightness refers to the tightness of the physical connection or logical association between the scene entities.

[0088] The state change trend refers to the change direction and law of the running state of the scene entity over time in the interactive video rendering process, such as starting, stopping, and change of motion trajectory.

[0089] The association coefficient refers to a quantitative index for measuring the degree of association between the scene entity and other entities.

[0090] In the embodiment of the application, first, the spatial position coordinates, shape size, and preset function attribute information of each scene entity are extracted from the topology data, and the spatial distribution law is obtained through coordinate clustering analysis and function area division. Specifically, entities with close distances and consistent function attributes can be classified into the same clustering unit, and then the average distance of entities in the same clustering unit and the minimum distance between different clustering units are calculated to obtain the spatial distance parameter corresponding to the spatial distribution law, and the smaller the distance, the greater the value of the spatial distance parameter.

[0091] Secondly, the physical connection or logical association relationship between entities recorded in the topology data is extracted, the connection stability is judged by analyzing the connection duration and connection fault frequency, the interaction frequency between entities within a unit rendering duration is determined by counting the number of interactions, and the connection tightness between entities is obtained by combining the stability and interaction frequency and dividing them into three levels of high, medium, and low. Then, the connection tightness of the three levels of high, medium, and low is quantified into corresponding numerical values respectively to obtain the association strength parameter value, and the higher the level, the greater the association strength parameter value.

[0092] Then, according to the core simulation target of the interactive video rendering, the existing entity state change records are extracted from the historical rendering data, and the change direction and amplitude of the state change that has not occurred are predicted based on the scene running logic, so as to determine the state change trend of each scene entity. Based on the state change trend, the degree of synchronization of the entity state change is analyzed, and the collaborative change parameter corresponding to the state change trend is determined, and the higher the change synchronization, the greater the value of the collaborative change parameter.

[0093] Finally, according to the simulation core target of the digital twin scene, the weights of each parameter are determined, the spatial distance parameter corresponding to the spatial distribution law, the association strength parameter corresponding to the connection tightness, and the collaborative change parameter corresponding to the state change trend are weighted and summed according to the weights, and then the weighted sum result is calibrated through a preset coefficient calibration range to obtain the association coefficient of each scene entity.

[0094] Step 212: Determine the rendering weight value of each scene entity according to the functional importance and rendering priority requirement of each scene entity in the digital twin scene.

[0095] In this step, the functional importance refers to the influence degree of the function assumed by the scene entity in the digital twin scene on the whole scene simulation target, and the importance of the core function entity is higher than that of the auxiliary function entity.

[0096] The rendering priority requirement refers to the requirement for the order and precision of rendering of each scene entity according to the core target of scene simulation.

[0097] The rendering weight value refers to a quantitative index for measuring the proportion of resources occupied and the degree of priority processing of a scene entity in the rendering process.

[0098] In the embodiments of the present application, first, the core simulation target of the digital twin scene is determined, the functions of each scene entity under the target are analyzed, the importance of the functions of each entity is judged, and the importance of the functions is divided into three importance levels of high, medium and low, and a corresponding quantitative value is set for each importance level. Then, according to the rendering requirement of the interactive video and the core simulation target of the digital twin scene, the rendering priority requirement of each scene entity is determined, which can be specifically expanded from three core dimensions: first, the picture focus requirement, according to the key display paragraphs marked by the video script and the user interaction hot area preset region, the picture area which the user is most likely to focus on in the interactive video is determined, and the entity in the focus area needs to be rendered first; second, the interaction response requirement, the entity which can be operated by the user in the interactive video is determined, such entity needs to ensure the rendering real-time, and the priority is higher than that of the non-interactive entity; third, the detail presentation requirement, according to the video definition requirement, it is judged which entity needs to present high-precision texture and dynamic effect. Based on the three dimensions, three rendering priority levels of high, medium and low are divided, and a corresponding quantitative value is set for each rendering priority level, wherein the core function entity corresponds to high priority, among the auxiliary function entities, the entity in the focus area or the entity needing interaction can be promoted to medium priority, and the remaining auxiliary entities are low priority.

[0099] Finally, according to the core simulation target of the digital twin scene, the weight coefficients of the function importance and the rendering priority requirement are set; according to the weight coefficients set above, the quantitative values of the importance levels of each scene entity and the quantitative values of the rendering priority levels are weighted and summed to calculate the comprehensive priority score of each scene entity, and then the corresponding rendering weight value is allocated according to the score, so that the entity with high function importance and high rendering priority can obtain greater rendering weight value.

[0100] Step 213: According to the structure requirement of the digital twin scene, the spatial distribution law, the connection tightness, the correlation coefficient and the rendering weight value of each scene entity are associated and matched to determine the correlation priority of each scene entity in the rendering process.

[0101] In this step, the structure requirement of the digital twin scene refers to the constraint requirement for the correlation relationship and hierarchical structure between scene entities to realize the reality and logic of scene simulation.

[0102] The correlation priority refers to the priority response order of the scene entity in the linkage rendering process.

[0103] In the embodiments of the present application, first, the structural requirements of the digital twin scene are determined, the core constraint conditions of the association relationship between entities are determined in combination with the core simulation target of the digital twin scene, and the core constraint conditions can be entity function dependency relationship, spatial level subordinate relationship, etc. Second, the spatial distribution law, connection tightness, association coefficient and rendering weight value of each scene entity are taken as the core dimensions of association matching, which are corresponded to each scene entity one by one to form the parameter combination of each scene entity, and then the corresponding rules of parameter combination and association priority level are formulated in combination with the core constraint conditions, for example, the scene entities in the same spatial clustering unit, high connection tightness, high association coefficient and large rendering weight value correspond to high association priority, and the scene entities with low connection tightness, low association coefficient and small rendering weight value correspond to low association priority. Finally, according to the corresponding rules, each scene entity is classified to obtain the association priority of each scene entity in the rendering process.

[0104] Step 214: Based on the association priority and the spatial distribution law of the scene entity, the interaction range between each scene entity is determined, and the association threshold between each scene entity is determined in combination with the rendering weight value.

[0105] In this step, the interaction range refers to the spatial range in which the scene entity can have a linkage rendering influence on other entities in the rendering process, which is determined by the spatial position, shape size and association attribute of the entity. The entities beyond the range do not have a linkage rendering influence on each other.

[0106] The association threshold refers to a critical quantitative value for judging whether the linkage rendering between scene entities needs to be triggered. When the association degree parameter between entities reaches or exceeds the threshold, the linkage rendering process of the two is started.

[0107] In the embodiments of the present application, first, based on the association priority and the spatial distribution law of the scene entity, in combination with the shape structure size of each scene entity, the interaction range of each scene entity is determined by spatial range measurement, for example, the interaction range of the core entity with high association priority can be appropriately enlarged. Then, only the entities in the range are calculated for the association threshold, so as to avoid invalid threshold setting for the entities outside the range without association significance. The association threshold between each scene entity is obtained by calculating the product of the rendering weight value and the association strength parameter between entities, wherein the association threshold corresponding to the scene entity with high association priority and large rendering weight value can be appropriately reduced to ensure the timeliness of the linkage of the core entity.

[0108] Step 215: The association priority of each scene entity, the interaction range and the association threshold between each scene entity are integrated to form the association benchmark of the digital twin scene.

[0109] In this step, the association reference refers to the standardized data set formed after integrating the association priority of the scene entity, the interaction range between the scene entities, and the association threshold.

[0110] In the embodiments of the present application, first, the association priority of each scene entity, the interaction range between each scene entity, and the association threshold are summarized, and these data are structured and organized in combination with the association space distribution rule and the association strength parameter, the data format and the quantitative unit are unified, and the standardized data set is formed; second, the information in the data set is checked for consistency, and it is checked whether the association priority of the same scene entity matches the corresponding interaction range and association threshold, such as checking whether the interaction range of the scene entity with high association priority covers the associated entities in its clustering unit and whether the association threshold matches the association strength parameter, and eliminating or correcting the unmatched data; finally, the standardized data set after verification is integrated into the association reference of the digital twin scene.

[0111] The embodiments of the present application realize accurate allocation of rendering resources by formulating an optimized scheduling scheme through multi-dimensional analysis of texture resource data related parameters, reduce resource waste, and improve rendering efficiency; the association reference constructed based on multiple parameters of topological data avoids confusion in linked rendering logic; provides efficient resource support and clear linkage basis for subsequent ray tracing rendering, and solves the problem of inefficient rendering caused by unreasonable resource allocation and unclear entity association in traditional rendering.

[0112] Step 103: Based on the optimized scheduling scheme and the association reference, the digital twin scene is rendered by ray tracing technology to generate scene image data.

[0113] In this step, the scene image data refers to the digital twin scene visualization data of each image frame of the corresponding interactive video generated by fusing the basic rendering data and the mutual influence data of each scene entity, which includes entity appearance details, entity interaction effects, and other information.

[0114] As shown in the embodiments of the present application, Figure 2 Step 103 can specifically include the following steps:

[0115] Step 301: Based on the association reference, a set of scene entities that need to be handled in the rendering process of each image frame is selected, and the association trigger relationship between each scene entity in the set of scene entities is determined.

[0116] In this step, the set of scene entities refers to the combination of multiple scene entities that need to be handled in the rendering process of the same image frame based on the association reference.

[0117] The association trigger relationship refers to a condition and a logical relationship of triggering linkage rendering between entities in a scene entity set, and the association trigger relationship includes a trigger condition, a response mode, and the like.

[0118] In the embodiments of the present application, scene entities in the same interaction range and with an association degree reaching an association threshold are filtered out from each image frame to form a scene entity set of each image frame; and the association trigger relationship of the scene entities in the set is determined according to the association priority in the association reference and the functional dependency relationship between the scene entities, for example, when the state of a scene entity with a high association priority changes, other scene entities in the set with an association degree reaching the threshold are triggered to perform synchronous rendering adjustment.

[0119] Step 302: According to the texture-related information of each scene entity in the texture resource data and the spatial distribution law in the topology data, the propagation and interaction of light on the surface of each scene entity are rendered by using a ray tracing technology to generate basic rendering data of each scene entity.

[0120] In this step, the basic rendering data refers to entity appearance detail data of a single scene entity, which is obtained by ray tracing technology based on its texture-related information and spatial distribution law.

[0121] In the embodiments of the present application, step 302 specifically includes the following steps:

[0122] Step 311: Spatial position coordinates, shape structure information, and relative distances to surrounding entities of each scene entity are extracted from the topology data.

[0123] In the embodiments of the present application, the spatial position coordinates, shape structure information, and relative distances to surrounding entities of each scene entity are extracted from the topology data, and these information is classified and sorted according to entities to form a spatial feature data set of each entity, wherein the spatial position coordinates of each scene entity are three-dimensional coordinate positions of each scene entity in the digital twin scene, the shape structure information includes features such as shape, size, and surface contour of the entity, and the relative distance is a straight-line distance between each scene entity and adjacent scene entities.

[0124] Step 312: According to the texture-related information of each scene entity in the texture resource data, light attribute information of each texture is determined.

[0125] In this step, the light attribute information refers to parameters such as reflection characteristics, refraction characteristics, and absorption characteristics of each texture to light, which are determined based on features such as material type and surface roughness of the texture-related information.

[0126] In the embodiments of the present application, the light attribute information corresponding to each texture is determined according to the physical and optical properties of different materials in the texture-related information of each scene entity, for example, the high reflection and low absorption properties of a metal texture, and the medium reflection and medium refraction properties of a plastic texture.

[0127] Step 313: According to the spatial position coordinates, shape structure information and relative distance from the surrounding entities of each scene entity, and in combination with the light attribute information of each texture, the light coverage range of each scene entity is determined.

[0128] In this step, the light coverage range refers to the spatial range in which light can propagate and interact with the surface of a scene entity, which is determined by the spatial characteristics of the scene entity and the light attribute of the texture. Light beyond this range will no longer be rendered for the scene entity.

[0129] In the embodiments of the present application, first, the propagation ability of light when emitted or received from the surface of an entity is analyzed according to the spatial position coordinates, shape structure information and relative distance from the surrounding entities of each scene entity, and in combination with the light attribute information of each texture. Then, according to the shape size and spatial position of the scene entity, and in combination with the propagation ability of light, the three-dimensional spatial range that can be covered by light is determined based on the surface of the scene entity, to ensure that the light coverage range not only includes the entity itself, but also matches the relative position relationship with the surrounding entities.

[0130] Step 314: The propagation path of light in the light coverage range is rendered using ray tracing technology with a preset starting position as the starting point, and the propagation path data is recorded.

[0131] In this step, the preset starting position refers to the initial three-dimensional coordinate point at which light begins to propagate. The preset starting position includes the position of a fixed light source in the scene or a light emission point on the surface of an entity.

[0132] The propagation path data refers to the trajectory information of light when propagating in the light coverage range. This data includes the propagation direction of light, the coordinate points passed through, the propagation distance and other parameters.

[0133] In the embodiments of the present application, first, a preset starting position is set according to the lighting requirements of the digital twin scene. Then, the propagation process of light in the light coverage range is simulated using ray tracing technology with the preset starting position as the starting point, and the changes in the propagation direction of light, the three-dimensional coordinate points at each pass and the total propagation distance are recorded in real time to form the propagation path data.

[0134] Step 315: According to the light attribute information, the interaction of light with the surface of each scene entity is rendered using ray tracing technology, and the interaction data of light with the surface of each scene entity is recorded.

[0135] In this step, the interaction data refers to the record data of the reflection, refraction, absorption and other interaction behaviors of the light ray when it contacts the surface of the scene entity, which includes the interaction type, the direction of the light ray after the interaction, the energy change and other information.

[0136] In the embodiments of the present application, when the light ray propagates to the surface of the scene entity, the light ray interaction process with the surface of the scene entity is simulated by using the light ray tracing technology. The specific process is as follows: the direction of the reflected light ray is determined according to the reflection characteristics of the texture, the path of the refracted light ray is determined according to the refraction characteristics, the degree of loss of the light ray energy is determined according to the absorption characteristics, and the simulation is performed according to the above determined information, while the type of each interaction, the coordinate of the direction of the light ray after the interaction, the energy change value and other information are recorded in real time to form the interaction data.

[0137] Step 316: The propagation path data and the interaction data of each scene entity are integrated to form the basic rendering data.

[0138] In the embodiments of the present application, the propagation path data and the interaction data are first classified according to the scene entities to ensure the corresponding association of the two types of data of the same scene entity; then the propagation path data and the interaction data of the same scene entity are structured and integrated, and the spatial position coordinates and the texture related information of the scene entity are supplemented to form the basic rendering data containing the entity space trajectory and the surface optical interaction details.

[0139] Step 303: According to the associated trigger relationship, the mutual influence data between each scene entity in the scene entity set is generated.

[0140] In this step, the mutual influence data refers to the interaction effect data between each entity in the scene entity set generated by the light ray propagation, which includes the details such as the light ray irradiation, the shadow projection, the brightness influence and other details of one entity to another entity.

[0141] In the embodiments of the present application, step 303 specifically includes the following steps:

[0142] Step 321: According to the associated trigger relationship, the target scene entity pair in the scene entity set is determined, and the influence transmission direction of each target scene entity pair is determined.

[0143] In this step, the target scene entity pair refers to the combination of two scene entities in the scene entity set which have the associated trigger relationship and need to calculate the mutual influence.

[0144] The influence transmission direction refers to the direction of the light ray propagating from one entity to another entity, i.e. the direction of the influence transmitting from the transmitting end entity to the receiving end entity.

[0145] In the embodiment of the present application, firstly, according to the association trigger relationship, the entity combination with linkage demand is filtered from the scene entity set to form a target scene entity pair; then, according to the spatial position coordinates of each scene entity, it is determined that the light propagates from which scene entity to another scene entity to be more in line with the association trigger logic, and then the influence transmission direction of each target scene entity pair is determined.

[0146] Step 322: According to the light attribute information corresponding to each target scene entity pair, the light propagation from the surface of the scene entity of the emission end to the surface of the scene entity of the receiving end is rendered by using the ray tracing technology, and the energy change information of the light is recorded.

[0147] In this step, the surface of the scene entity of the emission end refers to the surface of the entity in the target scene entity pair as the light emission source, which is the starting contact surface of the light propagation.

[0148] The surface of the scene entity of the receiving end refers to the surface of the entity in the target scene entity pair that receives the light propagation, which is the termination contact surface of the light interaction.

[0149] The energy change information refers to the record of the energy increase or decrease caused by reflection, refraction, absorption and other behaviors during the process of the light propagation from the surface of the emission end entity to the surface of the receiving end entity.

[0150] In the embodiment of the present application, firstly, according to the light attribute information corresponding to each target scene entity pair, the light emission characteristics of the surface of the scene entity of the emission end and the light interaction characteristics of the surface of the scene entity of the receiving end are determined, and then the initial light parameters are determined based on the light emission characteristics of the surface of the scene entity of the emission end; the complete process of the light propagation from the surface of the emission end entity along the influence transmission direction to the surface of the receiving end entity is simulated by using the ray tracing technology according to the initial light parameters; during the simulation process, the energy change logic after the light reaches the receiving end is determined according to the light interaction characteristics of the surface of the scene entity of the receiving end, and the energy loss or enhancement of the light during the propagation process is recorded in real time, such as the energy decrease caused by air absorption and the energy increase caused by reflection superposition, to form the energy change information.

[0151] Step 323: According to the shape structure information, the spatial position coordinates of the scene entity of the receiving end and the energy change information, the first influence details of the light on the surface of the scene entity of the receiving end are obtained.

[0152] In this step, the first influence details refer to the visual effect details generated on the surface of the receiving end entity after the light directly propagates from the surface of the emission end entity to the surface of the receiving end entity, and the first influence details include brightness change, color superposition, shadow contour and other information.

[0153] In the embodiment of the present application, based on the shape structure information and spatial position coordinates of the scene entity of the receiving end, combined with the light propagation direction and energy distribution, the irradiation area of the light on the surface of the scene entity of the receiving end is determined; according to the light attribute information and energy change information, the visual effect details are further generated, specifically, the brightness numerical change is calculated through the light energy change and the light reflection characteristics of the texture, the color mixing effect is formed by the color transfer characteristics in the light attribute and the texture color of the receiving end entity itself superposition, the light missing area formed in the irradiation area of the receiving end after the light is blocked by the emitting end or the surrounding entity constitutes the shadow contour, and finally the first influence detail is formed.

[0154] Step 324: If there is an indirect influence path for each target scene entity pair, a second influence detail corresponding to the indirect propagation path in the preset influence transmission direction is obtained.

[0155] In this step, the preset influence transmission direction refers to the extension direction of the influence transmission direction of the target scene entity pair, which is used to judge whether the indirect influence propagation path meets the linkage logic.

[0156] The indirect propagation path refers to the propagation path of the light from the surface of the scene entity of the emitting end to the surface of the scene entity of the receiving end without directly reaching the surface of the receiving end entity, but reaching the surface of the receiving end entity through the reflection, refraction and other interactions of other entities in the scene entity set.

[0157] The second influence detail refers to the visual effect detail generated on the surface of the receiving end entity after the light reaches the surface of the receiving end entity through the indirect propagation path, and the second influence detail includes secondary brightness change, indirect shadow, color shift and other information.

[0158] In the embodiment of the present application, firstly, it is judged whether there is other scene entity in the influence transmission direction of the target scene entity pair, if there is, it is determined that there is an indirect influence path; then the complete indirect propagation path of the light from the surface of the emitting end entity to the surface of the scene entity of the receiving end through the reflection or refraction of the intermediate entity is tracked, and the light energy change and the interaction type on the path are recorded; combined with the shape structure information and spatial position coordinates of the receiving end entity, while referring to the light energy change and the interaction type on the indirect propagation path, the visual effect of the indirect propagation light on the surface of the scene entity of the receiving end is analyzed, such as secondary brightness superposition, indirect formed fuzzy shadow, color shift caused by multiple reflections, etc., and the second influence detail is formed by extracting these details.

[0159] Step 325: The first influence detail and the second influence detail are integrated to generate the mutual influence data between each scene entity in the scene entity set.

[0160] In the embodiments of the present application, it is first judged whether there is an indirect influence path for each target scene entity pair: if there is only a direct propagation path, only the first influence detail is extracted; if there are both direct and indirect propagation paths, the first influence detail and the second influence detail are extracted; the extracted influence details are sorted and integrated according to the influence degree, and the identification of the target scene entity pair and the influence transmission direction information are supplemented, to generate the mutual influence data between each scene entity in the scene entity set.

[0161] Step 304: Based on the basic rendering data of each scene entity and the corresponding mutual influence data, scene image data corresponding to each image frame is generated.

[0162] In the embodiments of the present application, the basic rendering data and the mutual influence data of each scene entity are correspondingly associated according to the corresponding relationship between the scene entity set and the image frame, and then the mutual influence data of each scene entity received from other scene entities is superimposed into the basic rendering data, for example, the brightness change and the shadow effect generated by other entities are superimposed on the appearance details of the entity itself; the superimposed data set is subjected to consistency verification, and the verification content includes whether the entity's own appearance details, light interaction effects and the like match the linkage influence details, so as to ensure that the entity's own details and the linkage influence details do not conflict, and finally complete scene image data corresponding to each image frame is generated.

[0163] The embodiments of the present application solve the problem of inaccurate entity linkage and insufficient detail presentation in traditional rendering, and through accurate light propagation and interaction simulation, both the appearance details of a single entity and the linkage influence between entities are restored.

[0164] Step 104: According to the scene image data, a target image sequence is generated, the target image sequence is compressed and associated by using video encoding technology, and a rendered video is generated, so as to realize video rendering and detail enhancement of the digital twin scene.

[0165] In this step, the target image sequence refers to an ordered image set formed by adjusting the scene image data according to the image frame time sequence of the interactive video and the preset playback requirement, and the sequence includes complete scene details of each image frame.

[0166] The rendered video refers to a digital twin scene video file which can be directly played after encoding processing of the target image sequence, and if there is a matching audio, the video file contains audio and video synchronization content.

[0167] In the embodiments of the present application, step 104 specifically includes the following steps:

[0168] Step 401: According to the image frame time sequence of the interactive video and the preset playback requirement, the scene image data is adjusted to generate a target image sequence.

[0169] In this step, the image frame time sequence refers to the playing order of each image frame of the interactive video preset, which is set based on the time line of the scene simulation.

[0170] The preset playing requirement refers to the video parameter requirement determined according to the playing device, the propagation scene, etc., which includes image resolution, frame rate, picture ratio, etc.

[0171] In the embodiment of the present application, first, the image frame time sequence of the interactive video is acquired, and the scene image data is sorted according to the sequence, so as to ensure that the playing logic of each frame of scene image is consistent with the scene simulation process; then, according to the preset playing requirement, the parameter of the sorted scene image data is adjusted, for example, the image resolution is uniformly adjusted to a preset value, redundant frames are supplemented or deleted according to the frame rate requirement, and the picture ratio is corrected to an adaptive range, and after the adjustment, a target image sequence is formed.

[0172] Step 402: determine the playing adaptation parameter corresponding to the target image sequence, and use video coding technology to compress and associate the target image sequence according to the playing adaptation parameter, to generate an encoded video stream.

[0173] In this step, the playing adaptation parameter refers to the video coding related parameter adapted to different playing devices or propagation networks, which includes the encoding format, the code rate, the frame rate adaptation value, etc., and is determined based on the playing scene and the device performance.

[0174] The encoded video stream refers to the ordered video data string formed after the target image sequence is compressed and associated by using the video coding technology.

[0175] In the embodiment of the present application, first, the playing adaptation parameter corresponding to the target image sequence is determined according to the performance of the playing device and the bandwidth condition of the propagation network, for example, a lightweight encoding format is selected for a low-power device, and a low code rate is set for a low-bandwidth network; then, the target image sequence is processed according to the playing adaptation parameter by using the video coding technology, and the specific processing process includes: on the one hand, the redundant information in the image data is compressed to reduce the data amount; on the other hand, the inter-frame association relationship is established to ensure the continuity of the frame sequence during playing, and after the processing, an encoded video stream is generated.

[0176] Step 403: if there is audio data matched with the interactive video, the encoded video stream and the audio data are integrated and packaged, to form a rendered video; if there is no audio data matched with the interactive video, the encoded video stream is packaged, to form a rendered video.

[0177] In the embodiment of the present application, it is first judged whether there is audio data matched with the interactive video, such as device operation sound effect, voice explanation and the like in the digital twin scene; if there is matched audio data, the encoded video stream is first synchronized and calibrated with the audio data to ensure that the timestamps of the video frames and the corresponding audio segments are consistent, and then the synchronized audio and video data is integrated and packaged according to the preset video packaging format to form a rendered video containing audio and video; if there is no matched audio data, the encoded video stream is directly packaged according to the preset packaging format to form a rendered video containing only video.

[0178] The embodiment of the present application solves the problems of sequence confusion, low coding efficiency, and audio and video synchronization in traditional video rendering; the rendered video is adapted to various playing scenes, which not only retains the detail enhancement effect of the digital twin scene, but also guarantees the smoothness of playing, and realizes the complete conversion of the digital twin scene from image data to presentable video.

[0179] Figure 3 A structural schematic diagram of a specific embodiment of a video viewpoint adaptive rendering and detail enhancement system provided by the embodiment of the present application is shown in Figure 3 The system can include:

[0180] The acquisition module 31 is configured to acquire rendering related data of the interactive video of the digital twin scene in a rendering process, wherein the rendering related data includes texture resource data and topology data.

[0181] The determination module 32 is configured to determine an optimized scheduling scheme of rendering resources based on the texture resource data in the rendering related data, and determine an association reference of the digital twin scene based on the topology data.

[0182] The rendering module 33 is configured to perform ray tracing rendering on the digital twin scene by using a ray tracing technology based on the optimized scheduling scheme and the association reference, to generate scene image data.

[0183] The generation module 34 is configured to generate a target image sequence according to the scene image data, compress and associate the target image sequence by using a video encoding technology, and generate a rendered video, to realize video rendering and detail enhancement of the digital twin scene.

[0184] The video viewpoint adaptive rendering and detail enhancement system of the embodiment of the present application is used to realize the aforementioned video viewpoint adaptive rendering and detail enhancement method, and thus the specific embodiments of the video viewpoint adaptive rendering and detail enhancement system can be seen from the embodiment part of the video viewpoint adaptive rendering and detail enhancement method in the foregoing, and the specific embodiments can be referred to the description of the corresponding embodiment part, which will not be described herein again.

[0185] The application further provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component, and the steps of the video viewpoint adaptive rendering and detail enhancement method described above are implemented.

[0186] The application further provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is executed by a computer to implement the steps of the video viewpoint adaptive rendering and detail enhancement method described above.

[0187] In an exemplary embodiment, the computer storage medium described above can include but is not limited to a U disk, a read-only memory, a random access memory, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0188] The embodiments of the application further provide a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps in the video viewpoint adaptive rendering and detail enhancement method embodiments described above.

[0189] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0190] The above describes in detail the video viewpoint adaptive rendering and detail enhancement method and system provided by the application. The principles and implementation manners of the application are described by applying specific examples in this paper, and the above example descriptions are only used to help understand the method of the application and its core idea. It should be pointed out that for ordinary skilled person in the technical field, some improvements and modifications can be made to the application without departing from the principles of the application, and these improvements and modifications also fall within the protection scope of the application.

Claims

1. A video viewpoint adaptive rendering and detail enhancement method, characterized in that, include: The rendering-related data of the interactive video of the digital twin scene during the rendering process is obtained, including texture resource data and topology data; Based on the texture resource data in the rendering-related data, an optimized scheduling scheme for rendering resources is determined. Based on the topology data, an association benchmark for the digital twin scene is determined. The optimized scheduling scheme for rendering resources refers to a scheme that reasonably allocates computing resources, storage resources, and calling order according to the characteristics of texture resource data and rendering requirements. The association benchmark refers to a set of standards constructed based on topology data to define the rules for interactive rendering between scene entities. Based on the optimized scheduling scheme and the associated benchmark, ray tracing technology is used to perform ray tracing rendering on the digital twin scene to generate scene image data; Based on the scene image data, a target image sequence is generated. The target image sequence is compressed and associated using video encoding technology to generate a rendered video, thereby realizing video rendering and detail enhancement of the digital twin scene. Based on the optimized scheduling scheme and the associated benchmark, ray tracing technology is used to perform ray tracing rendering on the digital twin scene to generate scene image data, including: Based on the aforementioned correlation benchmark, a set of scene entities that need to be linked during the rendering process of each image frame is selected, and the correlation triggering relationship between each scene entity in the set of scene entities is determined. Based on the texture-related information of each scene entity in the texture resource data and the spatial distribution pattern in the topology data, ray tracing technology is used to render the propagation and interaction of light on the surface of each scene entity to generate the basic rendering data of each scene entity. Based on the aforementioned association triggering relationship, generate mutual influence data between scene entities within the scene entity set; Based on the basic rendering data of each scene entity and the data on the mutual influence between each scene entity, scene image data corresponding to each image frame is generated. Based on the aforementioned association triggering relationship, mutual influence data between scene entities within the scene entity set is generated, including: Based on the associated triggering relationship, the target scene entity pairs within the scene entity set are determined, and the influence transmission direction of each target scene entity pair is determined. Based on the corresponding light attribute information of each target scene entity, ray tracing technology is used to render the light rays from the surface of the scene entity at the emitting end to the surface of the scene entity at the receiving end, and the energy change information of the light rays is recorded. Based on the shape and structure information, spatial coordinates, and energy change information of the scene entity at the receiving end, the first impact details of the light on the surface of the scene entity at the receiving end are obtained. If there is an indirect influence path for each target scene entity pair, then obtain the second influence details corresponding to the indirect propagation path in the preset influence transmission direction; Integrate at least one of the first influence details and the second influence details to generate mutual influence data between scene entities within the scene entity set; Based on the aforementioned topological data, the association benchmarks for the digital twin scenario are determined, including: The spatial distribution patterns of each scene entity and the degree of connection between scene entities are extracted from the topology data. Combined with the state change trends of each scene entity during the rendering process of the interactive video, the correlation coefficient of each scene entity is calculated. Based on the functional importance and rendering priority requirements of each scene entity in the digital twin scene, determine the rendering weight value of each scene entity. Based on the structural requirements of the digital twin scenario, the spatial distribution patterns, connection tightness, correlation coefficients, and rendering weight values ​​of the entities in each scenario are correlated and matched to determine the correlation priority of each entity in the rendering process. Based on the association priority and the spatial distribution pattern of scene entities, the interaction range between scene entities is determined, and the association threshold between scene entities is determined in combination with the rendering weight value. The association priority of each scene entity, the scope of interaction between each scene entity, and the association threshold are integrated to form the association benchmark of the digital twin scene.

2. The method according to claim 1, characterized in that, Based on the texture-related information of each scene entity in the texture resource data and the spatial distribution patterns in the topology data, ray tracing technology is used to render the propagation and interaction of light on the surfaces of each scene entity to generate basic rendering data for each scene entity, including: The spatial coordinates, shape and structure information, and relative distances to surrounding entities of each scene entity are extracted from the topology data. Based on the texture-related information of each scene entity in the texture resource data, determine the lighting attribute information of each texture; Based on the spatial coordinates, shape and structure information, and relative distance to surrounding entities of each scene entity, and combined with the light attribute information of each texture, the light coverage range of each scene entity is defined. Starting from a preset starting position, the propagation path of light within the light coverage area is rendered using ray tracing technology, and the propagation path data is recorded. Based on the light attribute information, ray tracing technology is used to render the interaction of light on the surfaces of various scene entities, and the interaction data between light and the surfaces of various scene entities is recorded. The propagation path data and interaction data of entities in each scene are integrated to form basic rendering data.

3. The method according to claim 1, characterized in that, Based on the texture resource data in the rendering-related data, an optimized scheduling scheme for rendering resources is determined, including: Based on the texture resource data in the rendering-related data, combined with the number of scene entities and the number of texture applications corresponding to each image frame in the interactive video, the rendering complexity of each image frame is determined. The frequency of recurrence and matching degree of different texture-related information in adjacent image frames are compared to calculate the target matching degree of each texture-related information. Based on the generation order of each image frame during the rendering process of the interactive video, the frequency of calls to different texture-related information in the rendering process is statistically analyzed. Based on the texture resource data, rendering complexity, target matching degree, and call frequency, an optimized scheduling scheme for rendering resources is determined.

4. The method according to claim 1, characterized in that, Based on the scene image data, a target image sequence is generated. The target image sequence is then compressed and correlated using video encoding technology to generate a rendered video, including: Based on the time sequence of the image frames in the interactive video and the preset playback requirements, the scene image data is adjusted to generate a target image sequence; Determine the playback adaptation parameters corresponding to the target image sequence, and compress and associate the target image sequence using video encoding technology based on the playback adaptation parameters to generate an encoded video stream; If audio data exists that accompanies the interactive video, the encoded video stream and the audio data are integrated and encapsulated to form a rendered video. If no audio data exists that accompanies the interactive video, the encoded video stream is encapsulated to form a rendered video.

5. A video viewpoint adaptive rendering and detail enhancement system, used to execute a video viewpoint adaptive rendering and detail enhancement method as described in claim 1, characterized in that, include: The acquisition module is used to acquire rendering-related data of the interactive video of the digital twin scene during the rendering process. The rendering-related data includes texture resource data and topology data. The determination module is used to determine an optimized scheduling scheme for rendering resources based on the texture resource data in the rendering-related data, and to determine the association benchmark of the digital twin scene based on the topology data. The rendering module is used to perform ray tracing rendering on the digital twin scene based on the optimized scheduling scheme and the associated benchmark, and generate scene image data. The generation module is used to generate a target image sequence based on the scene image data, compress and associate the target image sequence using video encoding technology, and generate a rendered video to achieve video rendering and detail enhancement of the digital twin scene.

6. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a video viewpoint adaptive rendering and detail enhancement method as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a video viewpoint adaptive rendering and detail enhancement method as described in any one of claims 1 to 4.

Images