A real-time rendering method and system for virtual scenes with enhanced immersion

Abstract

The present invention relates to the technical field of real-time rendering of virtual scenes, and particularly relates to a method and system for real-time rendering of virtual scenes that enhance the sense of immersion. The method obtains the rendering components of the virtual scene image and their display LOD levels; takes the rendering components in the current frame image as target components, and obtains their target LOD levels in the specified image according to the distribution of the target components in the specified image; obtains the rendering accuracy according to the difference between the target LOD level of the target components in the specified image and the display LOD level; and obtains the predicted LOD level of the target components in the next frame image of the current frame according to the rendering accuracy, the target LOD level of the target components, and the display LOD level. By accurately obtaining the predicted LOD level, the present invention is beneficial to reasonably allocate computing power in real-time rendering, and at the same time makes the rendering degree of the rendering components more in line with the user's feelings, improves the efficiency and accuracy of real-time rendering, and enhances the user's experience and sense of immersion.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time rendering of virtual scenes, and particularly to a method and system for real-time rendering of virtual scenes with enhanced immersion. Background Art

[0002] In order to allow users to more intuitively and conveniently understand components such as cultural relics and artworks in the Bowen Museum, users can wear display devices for the virtual museum real scene to generate the museum scenes observed by the users in real time. In order to improve the user experience in the virtual scene, it is necessary to perform real-time rendering on the museum components in the virtual scene. Real-time rendering allows users to interact with virtual scene images, which is beneficial for users to better observe the components in the Bowen Museum. At the same time, real-time rendering has many advantages such as fast speed, strong interactivity, high efficiency, good flexibility, strong authenticity, and good immersion. Therefore, real-time rendering is widely used in smart museums. By creating a virtual museum display, users can appreciate components such as cultural relics and artworks in the Bowen Museum in a virtual environment.

[0003] In the existing methods, real-time rendering is performed through the LOD model. According to the virtual distance between the components in the virtual scene displayed on the display device and the user, or the position of the components in the virtual scene in the image displayed on the display device, different degrees of rendering are given to the components in the virtual scene. Among them, the smaller the virtual distance corresponding to the rendering component or the more centered the position in the display image, the more detailed information needs to be rendered for the rendering component, and the lower the LOD level of the rendering component. However, in actual situations, the virtual distance information or position information of the rendering component does not reflect the interaction between the rendering component and the user, and does not consider the components that the user actually focuses on in the virtual scene. Therefore, the real-time rendering of the existing methods cannot meet the interests of the user in the virtual scene, affecting the user experience and immersion. At the same time, the update of the LOD model may continuously re-render the same rendering component, resulting in a large amount of computing power generated during the rendering process, and thus there are obvious delays or lags in the update of the digital museum scene experienced by the user, affecting the user experience and immersion. Summary of the Invention

[0004] In order to solve the technical problems that during the process of real-time rendering of a virtual scene through the LOD model, there may be inaccurate rendering degrees of rendering components and continuous re-rendering of the same rendering component, resulting in a large amount of computing power and causing lags, and thus the user experience and immersion are relatively poor, the purpose of the present invention is to provide a method and system for real-time rendering of virtual scenes with enhanced immersion. The specific technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for real-time rendering of virtual scenes with enhanced immersion. The method includes the following steps:

[0006] Obtain each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component;

[0007] Take each rendering component in the current frame image as the target component, and according to the distribution of each target component in each frame of the specified image, obtain the target LOD level of each target component in each frame of the specified image where it appears;

[0008] According to the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image where it appears, obtain the rendering accuracy of each target component;

[0009] According to the rendering accuracy, the display LOD level of each target component in the specified image where it appears, and the target LOD level and display LOD level of each target component in the current frame image, obtain the predicted LOD level of each target component in the next frame image of the current frame.

[0010] Furthermore, the method for obtaining the target LOD level is as follows:

[0011] Take the target components that appear in each frame of the specified image as the high - attention components;

[0012] According to the rendering color, position distribution and area of each high - attention component in each frame of the specified image, obtain the attention level of each high - attention component in each frame of the specified image;

[0013] According to the area of each non - high - attention component in each frame of the specified image where it appears and its distance from the high - attention components, obtain the attention level of each non - high - attention component in each frame of the specified image where it appears;

[0014] Based on the attention level, obtain the target LOD level of each target component in each frame of the specified image where it appears.

[0015] Furthermore, the method for obtaining the attention level of each high - attention component in each frame of the specified image is as follows:

[0016] For any high - attention component and any frame of the specified image, according to the rendering color of the points of the high - attention component in the specified frame image, use the SIFT algorithm to obtain the feature points of the high - attention component in the specified frame image and the two - dimensional coordinates of each feature point;

[0017] According to the three - dimensional coordinates of each vertex of the high - attention component in the mesh model of the specified frame image, the position information of the high - attention component in the specified frame image, and the viewing direction of the user, obtain the display vertices of the high - attention component in the specified frame image and the two - dimensional coordinates of each display vertex;

[0018] Obtain the difference between the number of feature points and the number of displayed vertices as the first difference;

[0019] Take the reciprocal of the result of adding the first difference to the first preset constant as the first degree of attention reference; wherein, the first preset constant is greater than 0;

[0020] According to the two-dimensional coordinates of the feature points and the displayed vertices, match the feature points and the displayed vertices through the nearest neighbor matching algorithm, and obtain the number of successfully matched feature points as the first quantity;

[0021] Obtain the area ratio of the highly concerned component in the specified image of this frame as the second degree of attention reference;

[0022] Take the normalized result of the product of the first quantity, the first degree of attention reference, and the second degree of attention reference as the degree of attention of the highly concerned component in the specified image of this frame.

[0023] Further, the method for obtaining the degree of attention of each non-highly concerned component in each frame of the specified image where it appears is:

[0024] For any non-highly concerned component and any frame of the specified image where the non-highly concerned component appears, obtain the shortest distance between the non-highly concerned component and each highly concerned component in the specified image of this frame as the first distance;

[0025] Take the reciprocal of the result of adding the smallest first distance to the second preset constant as the first degree of attention target; wherein, the second preset constant is greater than 0;

[0026] Obtain the area ratio of the non-highly concerned component in the specified image of this frame as the second degree of attention target;

[0027] Take the normalized result of the product of the first degree of attention target and the second degree of attention target as the reference degree of attention of the non-highly concerned component in the specified image of this frame;

[0028] Take the difference between the total number of levels of the LOD level and the third preset constant as the first value; wherein, the third preset constant must be a positive integer less than the total number of levels of the LOD level and greater than 0;

[0029] Take the ratio of the first value to the total number of levels of the LOD level as the adjustment weight;

[0030] Take the product of the adjustment weight and the reference degree of attention as the degree of attention of the non-highly concerned component in the specified image of this frame.

[0031] Further, the method for obtaining the target LOD level is:

[0032] Indirectly divide the value range of the attention level to obtain multiple attention level intervals; among them, the number of attention level intervals is the same as the total number of levels of the LOD level;

[0033] Correspond one attention level interval to one LOD level, where the LOD level corresponding to the largest attention level interval is the smallest;

[0034] For any target component and any specified image of the frame in which the target component appears, when the attention level of the target component in the specified image of the frame is within a certain attention level interval, use the LOD level corresponding to the attention level interval as the target LOD level of the target component in the specified image of the frame.

[0035] Furthermore, the method for obtaining the rendering accuracy is as follows:

[0036] For any target component, obtain the difference between the target LOD level and the display LOD level of the target component in each specified image of the frame in which the target component appears, and take them all as specific differences;

[0037] Take the result of negatively correlating and normalizing the sum of all specific differences as the rendering accuracy of the target component.

[0038] Furthermore, the method for obtaining the predicted LOD level is as follows:

[0039] For any target component, when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold, according to the display LOD level of the target component in the current frame image and the display LOD levels of the target component in each specified image of the frame in which the target component appears, obtain the predicted LOD level of the target component in the next frame image of the current frame;

[0040] When the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, according to the target LOD level and the display LOD level of the target component in the current frame image, obtain the predicted LOD level of the target component in the next frame image of the current frame.

[0041] Furthermore, the method for obtaining the predicted LOD level of the target component in the next frame image of the current frame when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold, according to the display LOD level of the target component in the current frame image and the display LOD levels of the target component in each specified image of the frame in which the target component appears, is as follows:

[0042] Arrange the specified images in which the target component appears in chronological order from front to back to obtain the specified image sequence of the target component;

[0043] Arrange the display LOD levels of the target component in each specified image where it appears according to the order of the corresponding specified image in the specified image sequence to obtain the display LOD level sequence of the target component;

[0044] Fit the display LOD levels in the display LOD level sequence into a straight line as the target straight line;

[0045] When the slope of the target straight line is greater than 0, set the LOD level change degree of the target component to 1;

[0046] When the slope of the target straight line is equal to 0, set the LOD level change degree of the target component to 0;

[0047] When the slope of the target straight line is less than 0, set the LOD level change degree of the target component to -1;

[0048] Take the sum of the display LOD level of the target component in the current frame image and the LOD level change degree as the predicted LOD level of the target component in the next frame image of the current frame.

[0049] Further, when the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, the method for obtaining the predicted LOD level of the target component in the next frame image of the current frame according to the target LOD level and the display LOD level of the target component in the current frame image is as follows:

[0050] In the current frame image, when the target LOD level of the target component is less than or equal to the display LOD level, take the target LOD level of the target component in the current frame image as the predicted LOD level of the target component in the next frame image of the current frame;

[0051] When the target LOD level of the target component is greater than the display LOD level, take the display LOD level of the target component in the current frame image as the predicted LOD level of the target component in the next frame image of the current frame.

[0052] In a second aspect, another embodiment of the present invention provides a virtual scene real-time rendering system for enhancing immersion, which includes: a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any of the above methods are implemented.

[0053] The present invention has the following beneficial effects:

[0054] The present invention first takes each rendering component in the current frame image as a target component, and then, according to the distribution of each target component in each specified image of each frame, obtains the target LOD level of each target component in each specified image where it appears, and determines the LOD level that each target component should be rendered according to the user's attention in each specified image where it appears, which is conducive to accurately analyzing whether the rendering of each target component meets the user's needs in the subsequent process; furthermore, according to the difference between the target LOD level and the display LOD level of each target component in each specified image where it appears, the rendering accuracy of each target component is obtained, which accurately reflects whether the rendering of each target component meets the user's observation situation, and indirectly infers the degree of re - rendering required for each target component in the subsequent update process; during the real - time rendering process, in order to accurately and efficiently render each target component, the predicted LOD level of each target component in the next - frame image of the current frame is obtained according to the rendering accuracy, the display LOD level of each target component in the specified image where it appears, and the target LOD level and the display LOD level of each target component in the current frame image, accurately determining the LOD level of each target component in the upcoming rendering, indirectly determining the target components that need to be re - rendered and the target components that do not need to be re - rendered, which is conducive to reasonably allocating computing power, reducing unnecessary computing power, and ensuring that the rendering LOD level of the target components meets the user's attention situation, improving the efficiency and accuracy of real - time rendering, and effectively enhancing the user's experience and immersion sense. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following - described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0056] Figure 1 It is a schematic flowchart of a real - time rendering method for a virtual scene to enhance immersion provided by an embodiment of the present invention;

[0057] Figure 2 It is a flowchart of a method for obtaining the target LOD level provided by an embodiment of the present invention;

[0058] Figure 3 It is a flowchart of a method for obtaining the predicted LOD level provided by an embodiment of the present invention;

[0059] Figure 4 It is a structural diagram of a real - time rendering system for a virtual scene to enhance immersion provided by an embodiment of the present invention;

[0060] Figure 5 Schematic diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0061] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following describes in detail a real-time rendering method and system for a virtual scene with enhanced immersion proposed according to the present invention, including its specific implementation manners, structures, features, and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0063] The following specifically describes the specific solutions of a real-time rendering method and system for a virtual scene with enhanced immersion provided by the present invention in combination with the accompanying drawings.

[0064] Embodiment 1:

[0065] The specific scenario of this embodiment is as follows: In the existing method, real-time rendering of museum components in a virtual scene is performed through an LOD model. Only based on the virtual distance between the components in the virtual scene displayed on the display device and the user, or the position of the components in the virtual scene in the image displayed on the display device, different degrees of rendering are given to the components in the virtual scene, without considering the components that the user actually focuses on in the virtual scene. This results in the real-time rendering not being able to meet the user's own interests in the virtual scene, affecting the user's experience and immersion. At the same time, the update of the LOD model may continuously re-render the same rendering component, causing a large amount of computing power to be consumed during the rendering process, and then resulting in obvious delays or stuttering phenomena in the update of the digital museum scene of the user experience, affecting the user's experience and immersion. To improve the user's experience and immersion and perform real-time rendering of the virtual scene of the museum accurately and quickly, in this embodiment, each rendering component in the current frame image is first used as a target component, and then the distribution of each target component in each specified image frame in which it appears is analyzed to obtain the target LOD level that each target component should be rendered in each specified image frame in which it appears; further, the target LOD level of each target component in each specified image frame in which it appears is compared with the display LOD level to obtain the rendering accuracy of each target component, accurately reflecting whether the actually rendered display LOD level of each target component is reasonable, which is beneficial to accurately obtaining the predicted LOD level of each target component in the next frame image of the current frame, determining the rendering components that need to be re-rendered, that is, only rendering and updating some rendering components, improving the speed of real-time rendering and updating of the virtual scene, avoiding spending a lot of time on calculation, making the computing power reasonably allocated, and at the same time ensuring that the predicted LOD level of each target component in the next frame image of the current frame is more in line with the user's attention, effectively improving the user's experience and immersion. Among them, the LOD (Level of Detail) model is a well-known technology and will not be elaborated further.

[0066] The present invention proposes a method for real-time rendering of a virtual scene with enhanced immersion. Please refer to Figure 1 , which shows a schematic flowchart of a method for real-time rendering of a virtual scene with enhanced immersion provided by an embodiment of the present invention. The method includes the following steps:

[0067] Step S1: Obtain each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component.

[0068] Specifically, to clearly illustrate the process of real-time rendering of the virtual museum scene, an embodiment of the present invention analyzes it with one user as an example. All subsequent virtual scenes are for this user, and all subsequent users refer to this user. By wearing a display device for the virtual reality scene, the digital museum scene observed by the user can be generated in real time. The user's eyes are tracked through the camera of the display device, that is, the eye tracking technology is used to obtain the video displayed in the display device at the position where the user's attention is focused in real time. Among them, the eye tracking technology is a well-known technology and will not be elaborated here.

[0069] By processing the configuration file called by the video generated by the display device, the information of the rendered museum scene is obtained, such as the cultural relics scanned by laser, the corridors and aisles constructed by simple models, etc. In this embodiment, each specific scene information is called a rendering component in the virtual scene, that is, a rendering component is a specific physical object. To better perform real-time rendering of the rendering components, multiple LOD levels of each rendering component can be read out first through the display device system. At the same time, each rendering component in each frame image of the museum virtual scene in the display device and the display LOD level of each rendering component, that is, the actual displayed LOD level of each rendering component, are obtained. Among them, the smaller the display LOD level, the richer the rendering details of the corresponding rendering component. In this embodiment, the total number of levels of the LOD level is set to 4 levels, namely 0, 1, 2, and 3. Among them, the rendering component with the LOD level of 0 has the richest rendering details. The implementer can set the total number of levels of the LOD level according to the actual situation and is not limited here.

[0070] Step S2: Take each rendering component in the current frame image as a target component, and obtain the target LOD level of each target component in each specified image where it appears according to the distribution of each target component in each specified image.

[0071] It is known that the virtual scene of the museum observed by the user through the worn display device is coherent, that is, there must be the same rendering components between two adjacent frames of images. In order to accurately and efficiently perform real-time rendering of the virtual scene, in this embodiment, each rendering component in the current frame image is first used as a target component, and then the rendering situation of each target component in the current time period and the user's attention to each target component are analyzed to predict the target LOD level that each target component should be rendered in the current time period. In order to analyze the situation of each target component in the current time period, in this embodiment, the current frame image and a preset number of historical images closest to the current frame image in time are used as designated images, and the distribution of each target component in each frame of designated image where it appears indirectly reflects the attention situation of each target component in the current time period. Furthermore, in this embodiment, according to the distribution of each target component in each frame of designated image, the target LOD level of each target component in each frame of designated image where it appears is obtained. Among them, the preset number is set to 29 in this embodiment, and the implementer can set the size of the preset number according to the actual situation, which is not limited here.

[0072] Preferably, in a realizable manner of this embodiment, for the method of obtaining the target LOD level, please refer to Figure 2 , which shows a flowchart of a method for obtaining a target LOD level provided by this embodiment. The method includes the following steps:

[0073] Step S201: The target components that appear in each frame of the designated image are used as highly concerned components.

[0074] The highly concerned components appear in all the designated images, indicating that the highly concerned components are present in the user's attention field of view during the current time period. It is known that the rendering components concerned by the user continuously appear in the video. Therefore, the rendering components concerned by the user must be among the highly concerned components.

[0075] Step S202: According to the rendering color, position distribution, and area of each highly concerned component in each frame of the designated image, obtain the attention degree of each highly concerned component in each frame of the designated image.

[0076] When a user focuses on a certain rendering component, the rendering details of this rendering component should be more abundant to ensure the user experience and immersion. To accurately obtain the degree of attention of the user to each highly concerned component, and then predict the LOD level that each highly concerned component should be rendered, which is beneficial to subsequent analysis of whether the actual rendering degree of each highly concerned component is accurate, and accurately judge whether the LOD level of each highly concerned component needs to be re-rendered during the rendering update process. Therefore, in this embodiment, the information of each highly concerned component in each frame of the specified image is analyzed. It is known that when the rendering color distribution feature and the vertex distribution feature of a certain highly concerned component in a certain frame of the specified image are more consistent, it indicates that the highly concerned component receives more attention in this frame of the specified image. At the same time, when the proportion of the highly concerned component in this frame of the specified image is larger, it also indicates that the highly concerned component receives more attention in this frame of the specified image. Therefore, in this embodiment, according to the rendering color, position distribution, and area of each highly concerned component in each frame of the specified image, the attention degree of each highly concerned component in each frame of the specified image is obtained.

[0077] Preferably, in a feasible implementation manner of this embodiment, the method for obtaining the attention degree of each highly concerned component in each frame of the specified image is as follows: For any highly concerned component and any frame of the specified image, first, all other regions in this frame of the specified image except the region where the highly concerned component is located are used as the background region, the rendering color of the highly concerned component remains unchanged, and the background regions are all set to white. Then, the feature points and the two-dimensional coordinates of each feature point of the highly concerned component in this frame of the specified image are obtained through the SIFT (Scale-Invariant Feature Transform) algorithm. Among them, the SIFT (Scale-Invariant Feature Transform) algorithm is a well-known technology and will not be elaborated here;

[0078] Retrieve the three-dimensional coordinates of each vertex of the high-concern component in the mesh model of the specified image in this frame. At the same time, retrieve the corresponding position and angle information of the high-concern component in the specified image in this frame, which is used to reflect the generation angle of the rendering, and obtain the rotation matrix of the network model corresponding to the high-concern component. Obtain the rotated vertex position information of all vertices of the high-concern component through the rotation matrix. By tracking the user's eyes, obtain the user's field-of-view direction information, construct a plane perpendicular to the field-of-view direction as the user's observation plane, that is, the specified image in this frame, project the rotated vertex coordinates onto the observation plane, and obtain the display vertices of the high-concern component in the specified image in this frame and the two-dimensional coordinates of each display vertex. It should be noted that vertices that overlap or are occluded after the transformation are not calculated. In this embodiment, only the displayed vertices are analyzed. Therefore, in this embodiment, according to the three-dimensional coordinates of each vertex of the high-concern component in the mesh model of the specified image in this frame, the position information of the high-concern component in the specified image in this frame, and the user's field-of-view direction, obtain the display vertices of the high-concern component in the specified image in this frame and the two-dimensional coordinates of each display vertex;

[0079] It is known that when the number of feature points is more similar to the number of display vertices, the degree of attention actually received by the high - attention component in the specified image of this frame is greater. Therefore, in this embodiment, the absolute value of the difference between the number of feature points and the number of display vertices is obtained as the first difference; then, the reciprocal of the result of adding the first difference and the first preset constant is used as the first attention reference degree, where the first preset constant is greater than 0 to avoid the situation where the first difference is 0 as the denominator. However, in actual situations, the high - attention component in the specified image of this frame may be partially occluded by other rendering components, resulting in errors in the number of feature points obtained, thus making the analysis of the attention degree of the high - attention component in the specified image of this frame inaccurate. To more accurately analyze the attention degree of the high - attention component in the specified image of this frame, in this embodiment, according to the two - dimensional coordinates of the feature points and the display vertices, the feature points and the display vertices are matched through the nearest - neighbor matching algorithm, and the number of successfully matched feature points is obtained as the first quantity; the larger the first quantity, the greater the attention degree of the high - attention component in the specified image of this frame. Among them, the nearest - neighbor matching algorithm is a well - known technology and will not be elaborated further. At the same time, the larger the proportion of the high - attention component in the specified image of this frame, the greater the possibility that the user pays attention to the high - attention component in the specified image of this frame. Therefore, in this embodiment, the area proportion of the high - attention component in the specified image of this frame is used as the second attention reference degree; the greater the second attention reference degree, the greater the attention degree of the high - attention component in the specified image of this frame. To accurately represent the attention degree of the high - attention component in the specified image of this frame, the normalized result of the product of the first quantity, the first attention reference degree, and the second attention reference degree is used as the attention degree of the high - attention component in the specified image of this frame. The greater the attention degree, the richer the details that should be rendered for the high - attention component in the specified image of this frame.

[0080] Among them, the calculation formula for the attention degree of the high - attention component in the specified image is: ; in the formula, is the attention degree of the ath high - attention component in the ith specified image; is the first quantity; is the number of feature points of the ath high - attention component in the ith specified image; is the number of display vertices of the ath high - attention component in the ith specified image; is the first difference; is the first preset constant; is the first attention reference degree of the ath high - attention component in the ith specified image; is the second attention reference degree of the ath high - attention component in the ith specified image; is the absolute - value function; norm is the normalization function.

[0081] This embodiment will Set to 1, and the implementer can set it according to the actual situation The size of which is not limited here.

[0082] So far, the attention degree of each high-attention component in each frame of the specified image is obtained.

[0083] Step S203: According to the area of each non-high-attention component in the target component in each frame of the specified image where it appears and its distance from the high-attention component, obtain the attention degree of each non-high-attention component in each frame of the specified image where it appears.

[0084] It is known that non-high-attention components are rendering components that users do not pay attention to. In actual situations, non-high-attention components are not unnecessary for rendering. The layout of non-high-attention components in the virtual scene still plays an important role, which can make the layout of the virtual scene more reasonable. The lack of rendering of non-high-attention components will also seriously affect the user experience and immersion. Therefore, non-high-attention components also need to be rendered at different LOD levels. When the distance between a certain non-high-attention component and the high-attention component in a certain frame of the specified image where it appears is closer, it means that the non-high-attention component has an obvious impact on the user's virtual scene experience in the frame of the specified image where it appears, and the non-high-attention component needs to be rendered more carefully in the frame of the specified image where it appears, that is, the attention degree of the non-high-attention component in the frame of the specified image where it appears should be greater; at the same time, when the proportion of the non-high-attention component in the frame of the specified image is larger, the proportion affecting the user's vision is larger. In order to improve the user experience, the attention degree of the non-high-attention component in the frame of the specified image should be greater. Therefore, in this embodiment, according to the area of each non-high-attention component in the target component in each frame of the specified image where it appears and its distance from the high-attention component, the attention degree of each non-high-attention component in each frame of the specified image where it appears is obtained.

[0085] Preferably, in an implementable manner of this embodiment, the method for obtaining the attention degree of each non-highly concerned component in each specified image where it appears is as follows: for any non-highly concerned component and any specified image frame where the non-highly concerned component appears, obtain the shortest distance between the non-highly concerned component and each highly concerned component in the specified image frame, and all of them are used as the first distance; among them, the method for obtaining the shortest distance between two regions is a well-known technology and will not be elaborated here. The smaller the first distance, the greater the attention degree of the non-highly concerned component in the specified image. Therefore, in this embodiment, the reciprocal of the sum of the smallest first distance and the second preset constant is used as the first attention target degree; the greater the first attention target degree, the greater the attention degree of the non-highly concerned component in the specified image; among them, the second preset constant is greater than 0 to avoid the situation where the first distance is 0 as the denominator. It is known that the larger the area ratio of the non-highly concerned component in the specified image frame, the greater the attention degree of the non-highly concerned component in the specified image frame. Furthermore, in this embodiment, the area ratio of the non-highly concerned component in the specified image frame is obtained as the second attention target degree; the greater the second attention target degree, the greater the attention degree of the non-highly concerned component in the specified image; in order to accurately obtain the attention degree of the non-highly concerned component in the specified image frame, furthermore, the normalized result of the product of the first attention target degree and the second attention target degree is used as the reference attention degree of the non-highly concerned component in the specified image frame; the greater the reference attention degree, the greater the degree to which the non-highly concerned component is concerned by the user in the specified image frame.

[0086] To avoid the rendering details of non-highly concerned components being too detailed and causing unnecessary waste of computing power, therefore, in this embodiment, the overall attention degree of non-highly concerned components in the specified image frame is reduced, and then the difference between the total number of levels of the LOD level and the third preset constant is obtained as the first value; among them, the third preset constant must be a positive integer less than the total number of levels of the LOD level and greater than 0; then, the ratio of the first value to the total number of levels of the LOD level is used as the adjustment weight; finally, the product of the adjustment weight and the reference attention degree is used as the attention degree of the non-highly concerned component in the specified image frame.

[0087] Among them, the calculation formula for the attention degree of a non-highly concerned component in the specified image where it appears is: ; in the formula, is the attention degree of the q-th non-highly concerned component in the i-th specified image frame; N is the total number of levels of the LOD level; is the smallest first distance of the q-th non-highly concerned component in the i-th specified image frame; is the first attention target degree of the q-th non-highly concerned component in the i-th specified image frame; is the second attention target degree of the q-th non-highly concerned component in the i-th specified image frame; is the second preset constant; is the third preset constant; norm is the normalization function; is the reference attention degree of the q-th non-highly concerned component in the specified image of the i-th frame; is the first value; is the adjustment weight.

[0088] In this embodiment, is set to 0.1, and is set to 1. Implementers can set the sizes of and according to the actual situation, which will not be limited here.

[0089] So far, the attention degree of each non-highly concerned component in each frame of the specified image where it appears is obtained. It should be noted that when a certain non-highly concerned component does not appear in a certain frame of the specified image, the attention degree of the non-highly concerned component in the frame of the specified image is directly defaulted to 0.

[0090] Step S204: Obtain the target LOD level of each target component in each frame of the specified image based on the attention degree.

[0091] Specifically, it can be seen from step S1 that in this embodiment, the total number of levels of the LOD level is set to 4 levels, namely 0, 1, 2, and 3, and the LOD level of 0 has the richest rendering details for the rendering component. In order to obtain the target LOD level of each target component in each frame of the specified image based on the attention degree, this embodiment divides the value range of the attention degree indirectly equally to obtain multiple attention degree intervals; among them, the number of attention degree intervals is the same as the total number of levels of the LOD level; it is known that the value range of the attention degree is 0-1 and the total number of levels of the LOD level is 4. Therefore, there are 4 attention degree intervals, which are respectively , and then one attention degree interval corresponds to one LOD level. Among them, the LOD level corresponding to the largest attention degree interval is the smallest, that is, corresponds to the LOD level of 0, corresponds to the LOD level of 1, corresponds to the LOD level of 2, corresponds to the LOD level of 3. For any target component and any frame of the specified image where the target component appears, when the attention degree of the target component in the frame of the specified image is within a certain attention degree interval, the LOD level corresponding to the attention degree interval is used as the target LOD level of the target component in the frame of the specified image. For example, when the attention degree of the q-th target component in the i-th frame of the specified image is 0.3, the target LOD level of the q-th target component in the i-th frame of the specified image is 2.

[0092] So far, the target LOD level of each target component in each specified image of each frame in which it appears is obtained.

[0093] Step S3: According to the difference between the target LOD level and the display LOD level of each target component in each specified image of each frame in which it appears, obtain the rendering accuracy of each target component.

[0094] It is known that the target LOD level is the LOD level at which the rendering component is ideally rendered considering the actual attention of the user, and the display LOD level is the LOD level directly rendered by the LOD model. In order to accurately analyze whether the actual rendered LOD level of each target component meets the user's attention, in this embodiment, the difference between the target LOD level and the display LOD level of each target component in each specified image of each frame in which it appears is analyzed. When the difference between the target LOD level and the display LOD level of a certain target component in each specified image of each frame in which it appears is smaller, it indicates that the actual rendered LOD level of this target component is more accurate. Furthermore, in this embodiment, according to the difference between the target LOD level and the display LOD level of each target component in the specified image in which it appears, the rendering accuracy of each target component is obtained. The greater the rendering accuracy, the more the display LOD level of the corresponding target component is the same as the target LOD level.

[0095] Preferably, in a feasible implementation manner of this embodiment, the method for obtaining the rendering accuracy is as follows: For any target component, the absolute value of the difference between the target LOD level and the display LOD level of this target component in each specified image of each frame in which it appears is used as a specific difference; the result of adding all the specific differences and then performing negative correlation and normalization is used as the rendering accuracy of this target component. It should be noted that in this embodiment, the result of adding all the specific differences is subjected to negative correlation and normalization processing, where exp is the exponential function with the natural constant as the base; x represents the result of adding all the specific differences.

[0096] So far, the rendering accuracy of each target component is obtained.

[0097] Step S4: According to the rendering accuracy, the display LOD level of each target component in the specified image in which it appears, and the target LOD level and the display LOD level of each target component in the current frame image, obtain the predicted LOD level of each target component in the next frame image of the current frame.

[0098] Specifically, when it is necessary to render and update the target components of the current frame image, the LOD level of each target component in the next frame image of the current frame image can be predicted based on the rendering accuracy of each target component in the current frame image, as well as the target LOD level and the display LOD level of each target component in the current frame image. Considering that the self-changing trend of the LOD level of each target component in the current frame image will have a certain impact on the predicted LOD level of each target component in the next frame of the current frame, therefore, in this embodiment, according to the rendering accuracy of each target component, the display LOD level of each target component in the specified image where it appears, as well as the target LOD level and the display LOD level of each target component in the current frame image, the predicted LOD level of each target component in the next frame image of the current frame is obtained.

[0099] Preferably, in a feasible implementation manner of this embodiment, for the method of obtaining the predicted LOD level, please refer to Figure 3 , which shows a flowchart of a method for obtaining the predicted LOD level provided in this embodiment. The method includes the following steps:

[0100] Step S301: For any target component, when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold, according to the display LOD level of the target component in the current frame image and the display LOD level of the target component in each frame of the specified image where it appears, obtain the predicted LOD level of the target component in the next frame image of the current frame.

[0101] It is known that the greater the rendering accuracy, the more consistent the LOD level of the corresponding target component rendered in the actual situation is with the ideal LOD level. Therefore, for any target component, when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold, the display LOD level of the target component in the current frame image can be used as the predicted LOD level of the target component in the next frame image of the current frame. Considering that the self-changing trend of the LOD level of the target component will also affect the predicted LOD level of the target component in the next frame image of the current frame, in this embodiment, the self-changing trend of the LOD level of the target component can be predicted through the display LOD level of the target component in each frame of the specified image where it appears. Therefore, in this embodiment, according to the display LOD level of the target component in the current frame image and the display LOD level of the target component in each frame of the specified image where it appears, the predicted LOD level of the target component in the next frame image of the current frame is obtained. The specific implementation method is as follows:

[0102] First, arrange the specified images of the target component in chronological order from front to back to obtain the specified image sequence of the target component; then arrange the display LOD levels of the target component in each frame of the specified image in the order of the corresponding specified image in the specified image sequence to obtain the display LOD level sequence of the target component; further, fit the display LOD levels in the display LOD level sequence into a straight line as the target straight line; among them, the method of straight line fitting is a well-known technology and will not be elaborated here. When the slope of the target straight line is greater than 0, the change trend of the display LOD level of the target component is an upward trend, and in this embodiment, the change degree of the LOD level of the target component is set to 1; when the slope of the target straight line is equal to 0, the change trend of the display LOD level of the target component is a stable trend, and in this embodiment, the change degree of the LOD level of the target component is set to 0; when the slope of the target straight line is less than 0, the change trend of the display LOD level of the target component is a downward trend, and in this embodiment, the change degree of the LOD level of the target component is set to -1; finally, take the sum of the display LOD level of the target component in the current frame image and the change degree of the LOD level as the predicted LOD level of the target component in the next frame image of the current frame. It should be noted that if the predicted LOD level of the target component in the next frame image of the current frame exceeds the set LOD level range, then take the nearest boundary LOD level as the predicted LOD level of the target component in the next frame image of the current frame. For example, when the predicted LOD level of the target component in the next frame image of the current frame is -1, then the predicted LOD level of the target component in the next frame image of the current frame is 0; when the predicted LOD level of the target component in the next frame image of the current frame is 4, then the predicted LOD level of the target component in the next frame image of the current frame is 3.

[0103] It should be noted that in this embodiment, the preset rendering accuracy threshold is set to 0.5, and the implementer can set the size of the preset rendering accuracy threshold according to the actual situation, which is not limited here.

[0104] Step S302: When the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, obtain the predicted LOD level of the target component in the next frame image of the current frame according to the target LOD level and the display LOD level of the target component in the current frame image.

[0105] When the rendering accuracy is less than or equal to the preset rendering accuracy threshold, it indicates that the LOD level of the true rendering of the corresponding target component is inaccurate and does not fully consider the user's own attention to the target component. On the premise that the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, in the current frame image, when the target LOD level of the target component is less than or equal to the display LOD level, it indirectly reflects that the true rendering of the current target component is more detailed. In order to maintain high-detail information, in this embodiment, the target LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame. When the target LOD level of the target component is greater than the display LOD level, in order to improve the rendering details of the target component and better meet the user's needs, the display LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame. In the process of re-rendering the rendering components in the next frame image of the current frame in this embodiment, only the target components that need to be rendered and updated are processed, that is, only the target components with different display LOD levels and predicted LOD levels in the current frame image are re-rendered, which is beneficial to the reasonable allocation of computing power, reduces unnecessary computing power at the same time, improves the efficiency of real-time rendering, and ensures that the rendering LOD level of the target component meets the user's attention, effectively improving the user's experience and immersion.

[0106] In summary, this embodiment obtains the rendering components of the virtual scene image and their display LOD levels; takes the rendering components in the current frame image as target components, and obtains their target LOD levels in the specified image according to the distribution of the target components in the specified image; obtains the rendering accuracy according to the difference between the target LOD level and the display LOD level of the target component in the specified image; and obtains the predicted LOD level of the target component in the next frame image of the current frame according to the rendering accuracy, the target LOD level and the display LOD level of the target component. By accurately obtaining the predicted LOD level, the present invention is beneficial to the reasonable allocation of computing power in real-time rendering, makes the rendering degree of the rendering components more in line with the user's feelings, improves the efficiency and accuracy of real-time rendering, and enhances the user's experience and immersion.

[0107] Embodiment 2:

[0108] The present invention also proposes a virtual scene real-time rendering system for enhancing immersion. Please refer to Figure 4 , which shows the structural diagram of a virtual scene real-time rendering system for enhancing immersion provided by an embodiment of the present invention. The system includes: an acquisition module 10, a target LOD level acquisition module 20, a rendering accuracy acquisition module 30, and a predicted LOD level acquisition module 40.

[0109] An acquisition module 10 is configured to acquire each rendering component in each frame image of the virtual museum scene and the display LOD level of each rendering component.

[0110] A target LOD level acquisition module 20 is configured to use each rendering component in the current frame image as a target component, and acquire the target LOD level of each target component in each specified image frame where it appears according to the distribution of each target component in each specified image frame.

[0111] A rendering accuracy acquisition module 30 is configured to acquire the rendering accuracy of each target component according to the difference between the target LOD level and the display LOD level of each target component in each specified image frame where it appears.

[0112] A predicted LOD level acquisition module 40 is configured to acquire the predicted LOD level of each target component in the next frame image of the current frame according to the rendering accuracy, the display LOD level of each target component in the specified image where it appears, and the target LOD level and the display LOD level of each target component in the current frame image.

[0113] It should be noted that: for the system provided in the above embodiments, only the division of the above functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, an enhanced immersion virtual scene real-time rendering system provided in the above embodiments and an embodiment of an enhanced immersion virtual scene real-time rendering method belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be elaborated here.

[0114] Embodiment 3:

[0115] The present invention also provides an enhanced immersion virtual scene real-time rendering device, which includes a memory and a processor. Among them, an executable program code is stored in the memory, and the processor is configured to call and execute the executable program code to execute an enhanced immersion virtual scene real-time rendering method provided in an embodiment of the present application. The device may specifically be a chip, a component or a module. The chip may include a connected processor and a memory; among them, the memory is used to store instructions, and when the processor calls and executes the instructions, the chip may execute an enhanced immersion virtual scene real-time rendering method provided in the above embodiments.

[0116] In addition, embodiments of the present application also protect a computer device. Please refer to Figure 5, the computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402. When the processor 402 executes the computer program 403, the computer device can execute any one of the enhanced immersion virtual scene real-time rendering methods introduced above.

[0117] Embodiment 4:

[0118] This embodiment also provides a computer-readable storage medium. Computer program code is stored in the computer-readable storage medium. When the computer program code runs on a computer, the computer is enabled to execute the above-related method steps to implement an enhanced immersion virtual scene real-time rendering method provided in the above embodiment.

[0119] Embodiment 5:

[0120] This embodiment also provides a computer program product. When the computer program product runs on a computer, the computer is enabled to execute the above-related steps to implement an enhanced immersion virtual scene real-time rendering method provided in the above embodiment.

[0121] Among them, the device, computer-readable storage medium, computer program product, or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be elaborated here.

[0122] It should be noted that the above order of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

Claims

1. A real-time rendering method for virtual scenes to enhance immersion, characterized in that The method includes the following steps: Obtain each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component; Take each rendering component in the current frame image as a target component, and obtain the target LOD level of each target component in each frame of the specified image where it appears according to the distribution of each target component in each frame of the specified image; Obtain the rendering accuracy of each target component according to the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image where it appears; Obtain the predicted LOD level of each target component in the next frame image of the current frame according to the rendering accuracy, the display LOD level of each target component in the specified image where it appears, and the target LOD level and display LOD level of each target component in the current frame image; The method for obtaining the target LOD level is as follows: Take the target components that appear in each frame of the specified image as high - attention components; Obtain the attention degree of each high - attention component in each frame of the specified image according to the rendering color, position distribution, and area of each high - attention component in each frame of the specified image; Obtain the attention degree of each non - high - attention component in each frame of the specified image where it appears according to the area of each non - high - attention component in each frame of the specified image where it appears and its distance from the high - attention components; Obtain the target LOD level of each target component in each frame of the specified image where it appears based on the attention degree; 2. The real-time rendering method of a virtual scene for enhancing immersion according to claim 1, characterized in that The method for obtaining the attention degree of each high - attention component in each frame of the specified image is as follows: For any high - attention component and any frame of the specified image, according to the rendering color of the points of the high - attention component in the frame of the specified image, obtain the feature points of the high - attention component in the frame of the specified image and the two - dimensional coordinates of each feature point through the SIFT algorithm; According to the three - dimensional coordinates of each vertex of the high - attention component in the mesh model of the frame of the specified image, the position information of the high - attention component in the frame of the specified image, and the viewing direction of the user, obtain the display vertices of the high - attention component in the frame of the specified image and the two - dimensional coordinates of each display vertex; Obtain the difference between the number of feature points and the number of display vertices as the first difference; Take the reciprocal of the sum of the first difference and the first preset constant as the first attention reference degree; where the first preset constant is greater than 0; According to the two - dimensional coordinates of the feature points and the display vertices, match the feature points and the display vertices through the nearest - neighbor matching algorithm, and obtain the number of successfully matched feature points as the first number; Obtain the area ratio of the high - attention component in the frame of the specified image as the second attention reference degree; Take the normalized result of the product of the first number, the first attention reference degree, and the second attention reference degree as the attention degree of the high - attention component in the frame of the specified image; 3. A real-time rendering method for a virtual scene enhancing immersion according to claim 1, characterized in that, The method for obtaining the attention degree of each non - high - attention component in each frame of the specified image where it appears is as follows: For any non - high - attention component and any frame of the specified image where the non - high - attention component appears, obtain the shortest distance between the non - high - attention component and each high - attention component in the frame of the specified image as the first distance; Take the reciprocal of the sum of the smallest first distance and the second preset constant as the degree of the first target of concern; where the second preset constant is greater than 0; Obtain the area ratio of the non-high-concern component in the specified image of this frame as the degree of the second target of concern; Take the normalized result of the product of the degree of the first target of concern and the degree of the second target of concern as the reference degree of concern of the non-high-concern component in the specified image of this frame; Take the difference between the total number of levels of the LOD level and the third preset constant as the first value; where the third preset constant must be a positive integer less than the total number of levels of the LOD level and greater than 0; Take the ratio of the first value to the total number of levels of the LOD level as the adjustment weight; Take the product of the adjustment weight and the reference degree of concern as the degree of concern of the non-high-concern component in the specified image of this frame.

4. A real-time rendering method for a virtual scene with enhanced immersion according to claim 1, characterized in that, The method for obtaining the target LOD level is as follows: Divide the value range of the degree of concern at equal intervals to obtain multiple intervals of the degree of concern; where the number of intervals of the degree of concern is the same as the total number of levels of the LOD level; Correspond one interval of the degree of concern to one LOD level, where the LOD level corresponding to the largest interval of the degree of concern is the smallest; For any target component and any specified image frame in which the target component appears, when the degree of concern of the target component in the specified image of this frame is within a certain interval of the degree of concern, take the LOD level corresponding to this interval of the degree of concern as the target LOD level of the target component in the specified image of this frame.

5. A real-time rendering method for a virtual scene enhancing immersion according to claim 1, characterized in that, The method for obtaining the rendering accuracy is as follows: For any target component, obtain the difference between the target LOD level and the displayed LOD level in each specified image frame in which the target component appears as a specific difference; Take the result of negative correlation and normalization of the sum of all specific differences as the rendering accuracy of the target component.

6. The real-time rendering method for virtual scenes enhancing immersion according to claim 1, characterized in that The method for obtaining the predicted LOD level is as follows: For any target component, when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold, obtain the predicted LOD level of the target component in the next frame image of the current frame according to the displayed LOD level of the target component in the current frame image and the displayed LOD levels of the target component in each specified image frame in which the target component appears; When the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, obtain the predicted LOD level of the target component in the next frame image of the current frame according to the target LOD level and the displayed LOD level of the target component in the current frame image.

7. A real-time rendering method for a virtual scene enhancing immersion according to claim 6, characterized in that, The method for obtaining the predicted LOD level of the target component in the next frame image of the current frame when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold according to the displayed LOD level of the target component in the current frame image and the displayed LOD levels of the target component in each specified image frame in which the target component appears is as follows: Arrange the specified images in which the target component appears in chronological order from front to back to obtain the sequence of specified images of the target component; Arrange the display LOD levels of the target component in each specified image where it appears according to the order of the corresponding specified image in the specified image sequence to obtain the display LOD level sequence of the target component; Fit the display LOD levels in the display LOD level sequence into a straight line as the target line; When the slope of the target line is greater than 0, set the LOD level change degree of the target component to 1; When the slope of the target line is equal to 0, set the LOD level change degree of the target component to 0; When the slope of the target line is less than 0, set the LOD level change degree of the target component to -1; Take the sum of the display LOD level of the target component in the current frame image and the LOD level change degree as the predicted LOD level of the target component in the next frame image of the current frame.

8. A real-time rendering method for a virtual scene enhancing immersion according to claim 6, characterized in that The method for obtaining the predicted LOD level of the target component in the next frame image of the current frame according to the target LOD level and the display LOD level of the target component in the current frame image when the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold is as follows: In the current frame image, when the target LOD level of the target component is less than or equal to the display LOD level, take the target LOD level of the target component in the current frame image as the predicted LOD level of the target component in the next frame image of the current frame; When the target LOD level of the target component is greater than the display LOD level, take the display LOD level of the target component in the current frame image as the predicted LOD level of the target component in the next frame image of the current frame.

9. A real-time rendering system for virtual scenes that enhances immersion, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for real-time rendering of a virtual scene with enhanced immersion according to any one of claims 1-8 above.

Images