Virtual scene real-time rendering method and system for enhancing immersion

Abstract

The invention relates to the technical field of virtual scene real-time rendering, in particular to a virtual scene real-time rendering method and system for enhancing immersion. The method comprises the following steps: acquiring a rendering component of a virtual scene image and a display LOD level of the rendering component; taking the rendering component in the current frame image as a target component, and acquiring a target LOD level of the target component in the specified image according to the distribution of the target component in the specified image; obtaining the rendering accuracy according to the difference between the target LOD level and the display LOD level of the target component in the specified image; and according to the rendering accuracy and the target LOD level and the display LOD level of the target component, obtaining a predicted LOD level of the target component in the next frame of image of the current frame. According to the method, the LOD level is accurately obtained and predicted, so that the computing power is reasonably distributed in real-time rendering, meanwhile, the rendering degree of the rendering component better conforms to the user experience, the real-time rendering efficiency and accuracy are improved, and the user experience and immersion are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual scene real-time rendering, and in particular to a virtual scene real-time rendering method and system for enhancing immersion. Background Art

[0002] In order to allow users to more intuitively and conveniently understand the components such as cultural relics and artworks in the museum, users can wear display devices of the virtual museum's real scene to generate the museum scene observed by users in real time. In order to improve the user's experience in the virtual scene, it is necessary to render the museum components in the virtual scene in real time. Real-time rendering allows users to interact with virtual scene images, which is conducive to users' better observation of the components in the museum. At the same time, real-time rendering has many advantages such as fast, strong interactivity, high efficiency, good flexibility, strong authenticity and good immersion. Therefore, real-time rendering is widely used in smart museums. By creating a virtual museum display, users can appreciate the components such as cultural relics and artworks in the museum in a virtual environment.

[0003] In the existing method, real-time rendering is performed through the LOD model, and different degrees of rendering are given to the components in the virtual scene according to the virtual distance between the components in the virtual scene displayed in the display device and the user or the position of the components in the virtual scene in the image displayed by the display device. The smaller the virtual distance corresponding to the rendering component or the closer the position is to the center of the displayed image, the more detailed information the rendering component needs to render, and the lower the LOD level of the rendering component. However, in actual situations, the virtual distance information or position information of the rendering component does not reflect the interaction between the rendering component and the user, and does not consider the components that the user actually pays attention to in the virtual scene. Therefore, the real-time rendering of the existing method cannot meet the user's own interests in the virtual scene, affecting the user's experience and immersion. At the same time, the update of the LOD model may continuously re-render the same rendering component, causing the rendering process to generate a large amount of computing power, which in turn leads to obvious delays or freezes in the update of the digital museum scene of the user experience, affecting the user's experience and immersion. Summary of the invention

[0004] In order to solve the technical problem that in the process of real-time rendering of virtual scenes through LOD models, there may be inaccurate rendering of rendering components and continuous re-rendering of the same rendering component, resulting in a large amount of computing power jamming, leading to poor user experience and immersion, the purpose of the present invention is to provide a method and system for real-time rendering of virtual scenes with enhanced immersion, and the technical solutions adopted are as follows: In a first aspect, an embodiment of the present invention provides a method for real-time rendering of a virtual scene for enhancing immersion, the method comprising the following steps: Obtain each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component; Each rendering component in the current frame image is taken as a target component, and according to the distribution of each target component in each frame of the specified image, the target LOD level of each target component in each frame of the specified image is obtained; Obtaining the rendering accuracy of each target component according to the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image in which the target component appears; According to the rendering accuracy and the display LOD level of each target component in the specified image in which it appears, as well as the target LOD level and display LOD level of each target component in the current frame image, the predicted LOD level of each target component in the next frame image of the current frame is obtained.

[0005] Furthermore, the method for obtaining the target LOD level is: The target components that appear in each frame of the specified image are regarded as high-attention components; According to the rendering color, position distribution and area of ​​each high-attention component in each frame of the specified image, the attention degree of each high-attention component in each frame of the specified image is obtained; According to the area of ​​each non-high attention component in the target component in each specified frame of the image in which it appears and its distance from the high attention component, the attention level of each non-high attention component in each specified frame of the image in which it appears is obtained; Gets the target LOD level of each target component in each frame of the specified image in which it appears based on the degree of attention.

[0006] Furthermore, the method for obtaining the attention level of each high attention component in each frame of a specified image is: For any high-attention component and any frame specified image, according to the rendering color of the midpoint of the high-attention component in the frame specified image, the feature points of the high-attention component in the frame specified image and the two-dimensional coordinates of each feature point are obtained by SIFT algorithm; According to the three-dimensional coordinates of each vertex of the high-attention component in the grid model of the frame specified image, the position information of the high-attention component in the frame specified image, and the user's field of view, the display vertices of the high-attention component in the frame specified image and the two-dimensional coordinates of each display vertex are obtained; Obtain the difference between the number of feature points and the number of displayed vertices as a first difference; The reciprocal of the sum of the first difference and the first preset constant is used as the first reference degree of attention; wherein the first preset constant is greater than 0; According to the two-dimensional coordinates of the feature points and the display vertices, the feature points and the display vertices are matched by using a nearest neighbor matching algorithm, and the number of successfully matched feature points is obtained as a first number; Obtaining the area ratio of the high-attention component in the specified image of the frame as a second attention reference degree; A result of normalizing the product of the first quantity, the first attention reference degree, and the second attention reference degree is used as the attention degree of the high attention component in the designated image of the frame.

[0007] Furthermore, the method for obtaining the attention level of each non-high attention component in each frame of a specified image in which it appears is: For any non-high attention component and any frame specified image where the non-high attention component appears, obtain the shortest distance between the non-high attention component and each high attention component in the frame specified image, and use them as the first distance; The inverse of the sum of the minimum first distance and the second preset constant is used as the first target degree of attention; wherein the second preset constant is greater than 0; Obtaining the area ratio of the non-high attention component in the specified image of the frame as the second attention target degree; Normalizing the product of the first attention target degree and the second attention target degree as the reference attention degree of the non-high attention component in the designated image of the frame; The difference between the total number of LOD levels and the third preset constant is used as the first value; wherein the third preset constant must be a positive integer less than the total number of LOD levels and greater than 0; The ratio of the first value to the total number of LOD levels is used as the adjustment weight; The product of the adjusted weight and the reference attention level is taken as the attention level of the non-high attention component in the specified image of the frame.

[0008] Furthermore, the method for obtaining the target LOD level is: The value range of the attention level is equally and indirectly divided to obtain a plurality of attention level intervals; wherein the number of attention level intervals is the same as the total number of LOD levels; Correspond one attention level interval to one LOD level, where the maximum attention level interval corresponds to the smallest LOD level; For any target component and any frame specified image in which the target component appears, when the attention level of the target component in the frame specified image is within a certain attention level interval, the LOD level corresponding to the attention level interval is used as the target LOD level of the target component in the frame specified image.

[0009] Furthermore, the method for obtaining the rendering accuracy is: For any target component, obtain the difference between the target LOD level and the display LOD level of the target component in each specified frame in which the target component appears, and use them as specific differences; The sum of all specific differences is negatively correlated and normalized to obtain the rendering accuracy of the target component.

[0010] Furthermore, the method for obtaining the predicted LOD level is: For any target component, when the rendering accuracy of the target component is greater than a preset rendering accuracy threshold, according to the display LOD level of the target component in the current frame image and the display LOD level of the target component in each specified frame image in which it appears, obtain the predicted LOD level of the target component in the next frame image of the current frame; When the rendering accuracy of the target component is less than or equal to a preset rendering accuracy threshold, the predicted LOD level of the target component in the next frame image of the current frame is obtained according to the target LOD level and display LOD level of the target component in the current frame image.

[0011] Furthermore, when the rendering accuracy of the target component is greater than a preset rendering accuracy threshold, the method for obtaining the predicted LOD level of the target component in the next frame image of the current frame according to the display LOD level of the target component in the current frame image and the display LOD level of the target component in each specified frame image in which the target component appears is: Arrange the specified images in which the target component appears in a time sequence from front to back to obtain a specified image sequence of the target component; Arrange the display LOD levels of the target component in each frame of the specified image according to the order of the corresponding specified images in the specified image sequence to obtain a display LOD level sequence of the target component; Fitting the display LOD levels in the display LOD level sequence into a straight line as a target straight line; When the slope of the target straight line is greater than 0, the LOD level change degree of the target component is set to 1; When the slope of the target straight line is equal to 0, the LOD level change degree of the target component is set to 0; When the slope of the target straight line is less than 0, the LOD level change degree of the target component is set to -1; The sum of the displayed LOD level of the target component in the current frame image and the degree of change of the LOD level is used as the predicted LOD level of the target component in the next frame image of the current frame.

[0012] Furthermore, when the rendering accuracy of the target component is less than or equal to a preset rendering accuracy threshold, the method for obtaining the predicted LOD level of the target component in the next frame image of the current frame according to the target LOD level and the display LOD level of the target component in the current frame image is: In the current frame image, when the target LOD level of the target component is less than or equal to the display LOD level, the target LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame; When the target LOD level of the target component is greater than the display LOD level, the display LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame.

[0013] In a second aspect, another embodiment of the present invention provides a real-time rendering system for a virtual scene with enhanced immersion, the system comprising: a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the steps of any one of the above methods are implemented.

[0014] The present invention has the following beneficial effects: The present invention firstly takes each rendering component in the current frame image as a target component, and then obtains the target LOD level of each target component in each specified frame image according to the distribution of each target component in each specified frame image, and determines the LOD level at which each target component should be rendered in each specified frame image according to the user's attention situation, which is conducive to the subsequent accurate analysis of whether the rendering situation of each target component meets the user's needs; and then obtains the rendering accuracy of each target component according to the difference between the target LOD level of each target component in each specified frame image and the display LOD level, accurately reflecting whether the rendering situation of each target component meets the user's observation situation, and indirectly infers whether each target component needs to be re-rendered in the subsequent update process. degree; in order to accurately and efficiently render each target component during the real-time rendering process, the predicted LOD level of each target component in the next frame image of the current frame is obtained according to the rendering accuracy and the display LOD level of each target component in the specified image in which it appears, as well as the target LOD level and display LOD level of each target component in the current frame image, and the LOD level of each target component in the next frame image of the current frame is accurately determined, and the target components that need to be re-rendered and the target components that do not need to be re-rendered are indirectly determined, which is conducive to the reasonable allocation of computing power and the reduction of unnecessary computing power. At the same time, it ensures that the rendering LOD level of the target component meets the user's concerns, improves the efficiency and accuracy of real-time rendering, and effectively enhances the user's experience and immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic flow chart of a method for real-time rendering of a virtual scene for enhancing immersion provided by an embodiment of the present invention; Figure 2 A flow chart of a method for obtaining a target LOD level provided by an embodiment of the present invention; Figure 3 A flow chart of a method for obtaining a predicted LOD level provided by an embodiment of the present invention; Figure 4 A structural diagram of a real-time rendering system for a virtual scene with enhanced immersion provided by an embodiment of the present invention; Figure 5 A schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the method and system for real-time rendering of a virtual scene for enhancing immersion proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] The following is a detailed description of a method and system for real-time rendering of a virtual scene for enhancing immersion provided by the present invention with reference to the accompanying drawings.

[0020] Embodiment 1: The specific scenario of this embodiment is: in the existing method, the museum components in the virtual scene are rendered in real time through the LOD model, and the components in the virtual scene are rendered to different degrees only according to the virtual distance between the components in the virtual scene displayed in the display device and the user or the position of the components in the virtual scene in the image displayed by the display device, without considering the components that the user himself actually pays attention to in the virtual scene, resulting in real-time rendering that cannot meet the user's own interests in the virtual scene, affecting the user's experience and immersion. At the same time, the update of the LOD model may continuously re-render the same rendering component, causing the rendering process to generate a large amount of computing power, which in turn leads to obvious delays or freezes in the update of the digital museum scene experienced by the user, affecting the user's experience and immersion. In order to improve the user's experience and immersion, the virtual scene of the museum is rendered accurately and quickly in real time. In this embodiment, each rendering component in the current frame image is first used as a target component, and then the distribution of each target component in each specified frame image is analyzed to obtain the target LOD level that each target component should be rendered in each specified frame image; further, the target LOD level of each target component in each specified frame image is compared with the display LOD level to obtain the rendering accuracy of each target component, accurately reflecting whether the display LOD level actually rendered by each target component is reasonable, which is conducive to accurately obtaining the predicted LOD level of each target component in the next frame image of the current frame, determining the rendering components that need to be re-rendered, that is, only rendering and updating some rendering components, improving the speed of real-time rendering and updating of virtual scenes, avoiding a lot of time spent on calculation, making the computing power reasonably distributed, and ensuring that the predicted LOD level of each target component in the next frame image of the current frame is more in line with the user's concerns, effectively improving the user's experience and immersion. Among them, the LOD (Level of Detail) model is a well-known technology and will not be repeated.

[0021] The present invention proposes a method for real-time rendering of a virtual scene to enhance immersion. Figure 1 , which shows a schematic flow chart of a method for real-time rendering of a virtual scene for enhancing immersion provided by an embodiment of the present invention, the method comprising the following steps: Step S1: Obtain each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component.

[0022] Specifically, in order to clearly explain the process of real-time rendering of the museum virtual scene, the embodiment of the present invention takes a user as an example for analysis. The virtual scenes that appear subsequently are all for this user, and the users that appear subsequently all refer to this user. The user can generate the digital museum scene observed by the user in real time by wearing a display device of the virtual reality scene, and track the user's eyes through the camera of the display device, that is, use eye tracking technology to obtain the video displayed on the display device at the position that the user is concerned about in real time. Among them, eye tracking technology is a well-known technology and will not be described in detail.

[0023] By processing the configuration file called by the video generated by the display device, the rendered museum scene information is obtained, such as laser scanned cultural relics, corridors and aisles constructed by simple models, etc. In this embodiment, each specific scene information is referred to as a rendering component in the virtual scene, that is, a rendering component is a specific physical object. In order to better render the rendering component in real time, the multiple LOD levels of each rendering component can be read out through the display device system first, and at the same time, each rendering component in each frame image of the museum virtual scene in the display device and the display LOD level of each rendering component, that is, the LOD level actually displayed by each rendering component, are obtained. Among them, the smaller the display LOD level, the richer the rendering details of the corresponding rendering component. In this embodiment, the total number of levels of the LOD level is set to 4 levels, namely 0, 1, 2 and 3, among which the rendering details corresponding to the rendering component with LOD level 0 are the richest. The implementer can set the total number of levels of the LOD level according to the actual situation, which is not limited here.

[0024] Step S2: Take each rendering component in the current frame image as a target component, and obtain the target LOD level of each target component in each frame of the specified image according to the distribution of each target component in each frame of the specified image.

[0025] It is known that the virtual scene of the museum observed by the user through the display device worn is coherent, that is, there must be the same rendering component between two adjacent frames of images. In order to accurately and efficiently render the virtual scene in real time, this embodiment first takes each rendering component in the current frame image as the target component, and then analyzes the rendering of each target component in the current time period and the user's attention to each target component, and predicts the target LOD level that each target component should be rendered in the current time period. In order to analyze the situation of each target component in the current time period, this embodiment takes the current frame image and the preset number of historical images closest to the current frame image in time as designated images, and indirectly reflects the attention of each target component in the current time period according to the distribution of each target component in each designated image frame that appears. Then, this embodiment obtains the target LOD level of each target component in each designated image frame that appears according to the distribution of each target component in each designated image frame. Among them, this embodiment sets the preset number to 29, and the implementer can set the size of the preset number according to the actual situation, which is not limited here.

[0026] Preferably, in one possible implementation of this embodiment, the method for obtaining the target LOD level can be found in Figure 2 , which shows a flow chart of a method for obtaining a target LOD level provided in this embodiment, the method comprising the following steps: Step S201: The target component that appears in each frame of the specified image is regarded as a high-attention component.

[0027] The high-attention component appears in all the specified images, indicating that the high-attention component exists in the user's field of view in the current time period. It is known that the rendering component that the user is concerned about appears continuously in the video, so the rendering component that the user is concerned about must be among the high-attention components.

[0028] Step S202: Obtain the attention level of each high-attention component in each frame of the specified image according to the rendering color, position distribution and area of ​​each high-attention component in each frame of the specified image.

[0029] When a user pays attention to a certain rendering component, the rendering details of the rendering component should be richer to ensure the user's experience and immersion. In order to accurately obtain the user's attention level to each high-attention component, and then predict the LOD level that each high-attention component should be rendered, it is beneficial to the subsequent analysis of whether the actual rendering level of each high-attention component is accurate, and accurately judge whether the LOD level of each high-attention component needs to be re-rendered during the rendering update process. Therefore, this embodiment analyzes the information of each high-attention component in each frame of the specified image. It is known that when the rendering color distribution characteristics and vertex distribution characteristics of a certain high-attention component in a certain frame of the specified image are more consistent, it means that the high-attention component has received more attention in the specified image of the frame. At the same time, when the proportion of the high-attention component in the specified image of the frame is larger, it also means that the high-attention component has received more attention in the specified image of the frame. Therefore, this embodiment obtains the attention level of each high-attention component in each frame of the specified image according to the rendering color, position distribution and area of ​​each high-attention component in each frame of the specified image.

[0030] Preferably, in a method that can be implemented in this embodiment, the method for obtaining the degree of attention of each high-attention component in each frame of a specified image is: for any high-attention component and any frame of a specified image, firstly, all areas except the area where the high-attention component is located in the frame of the specified image are used as background areas, the rendering color of the high-attention component remains unchanged, and the background areas are set to white, and then the feature points of the high-attention component in the frame of the specified image and the two-dimensional coordinates of each feature point are obtained through the SIFT (Scale-Invariant Feature Transform) algorithm. Among them, the SIFT (Scale-Invariant Feature Transform) algorithm is a well-known technology and will not be described in detail; The three-dimensional coordinates of each vertex of the high-attention component in the grid model of the specified image of the frame are retrieved, and the corresponding position angle information of the high-attention component in the specified image of the frame is retrieved to reflect the generation angle of rendering, and the rotation matrix of the network model corresponding to the high-attention component is obtained. The three-dimensional coordinates of all vertices of the high-attention component are obtained through the rotation matrix to obtain the rotated vertex position information. By tracking the user's eyeball, the user's field of vision direction information is obtained, and a plane perpendicular to the field of vision direction is constructed as the user's observation plane, that is, the specified image of the frame. The rotated vertex coordinates are projected on the observation plane to obtain the display vertices of the high-attention component in the specified image of the frame and the two-dimensional coordinates of each display vertex. It should be noted that the vertices that overlap or are blocked after the transformation are not calculated, and this embodiment only analyzes the displayed vertices. Therefore, this embodiment obtains the display vertices of the high-attention component in the specified image of the frame and the two-dimensional coordinates of each display vertex according to the three-dimensional coordinates of each vertex of the high-attention component in the grid model of the specified image of the frame, the position information of the high-attention component in the specified image of the frame, and the field of vision direction of the user; It is known that when the number of feature points and display vertices is more similar, the actual attention level of the high-attention component in the specified image of the frame is greater. Therefore, this embodiment obtains the absolute value of the difference between the number of feature points and the number of display vertices as the first difference; then the inverse of the result of adding the first difference to the first preset constant is used as the first attention reference level; wherein the first preset constant is greater than 0 to avoid the situation where the first difference is 0 as the denominator. However, in actual situations, the high-attention component may be partially blocked by other rendering components in the specified image of the frame, resulting in errors in the number of feature points obtained, thereby inaccurate analysis of the attention level of the high-attention component in the specified image of the frame. In order to more accurately analyze the attention level of the high-attention component in the specified image of the frame, this embodiment matches the feature points and the display vertices according to the two-dimensional coordinates of the feature points and the display vertices through the nearest neighbor matching algorithm, and obtains the number of successfully matched feature points as the first number; the larger the first number, the greater the attention level of the high-attention component in the specified image of the frame. wherein the nearest neighbor matching algorithm is a well-known technology and will not be described in detail. At the same time, the larger the proportion of the high-attention component in the specified image of the frame, the greater the possibility that the user will pay attention to the high-attention component in the specified image of the frame. In this embodiment, the area proportion of the high-attention component in the specified image of the frame is used as the second attention reference degree; the larger the second attention reference degree, the greater the attention degree of the high-attention component in the specified image of the frame. In order to accurately represent the attention degree of the high-attention component in the specified image of the frame, the product of the first quantity, the first attention reference degree and the second attention reference degree is normalized as the attention degree of the high-attention component in the specified image of the frame. The greater the attention degree, the richer the details that need to be rendered in the specified image of the frame should be.

[0031] The calculation formula for the attention level of a high-attention component in a specified image is: ; In the formula, Specify the image attention level for the a-th high attention component in the i-th frame; is the first quantity; Specify the number of feature points in the image for the a-th high-attention component in the i-th frame; Specify the number of displayed vertices in the image for the a-th high-attention component in the i-th frame; is the first difference; is the first preset constant; Assign the first attention reference degree of the image to the a-th high attention component at the i-th frame; Specify a second attention reference level of the image for the a-th high attention component in the i-th frame; is the absolute value function; norm is the normalization function.

[0032] This embodiment will Set to 1. The implementer can set it according to the actual situation. The size is not limited here.

[0033] At this point, the attention level of each high-attention component in each frame of the specified image is obtained.

[0034] Step S203: Obtain the attention level of each non-high attention component in each designated image frame in which it appears, based on the area of ​​each non-high attention component in the target component in each designated image frame in which it appears and its distance from the high attention component.

[0035] It is known that non-high attention components are rendering components that users do not pay attention to. In actual situations, non-high attention components do not need to be rendered. Non-high attention components still play an important role in the layout of virtual scenes, which can make the layout of virtual scenes more reasonable. The lack of rendering of non-high attention components will also seriously affect the user's experience and immersion. Therefore, non-high attention components also need to be rendered at different LOD levels. When a non-high attention component is closer to a high attention component in a specified frame image, it means that the non-high attention component has a significant impact on the user's experience of the virtual scene in the specified frame image. The non-high attention component needs to be rendered more carefully in the specified frame image, that is, the degree of attention to the non-high attention component in the specified frame image should be greater; at the same time, when the proportion of the non-high attention component in the specified frame image is larger, the proportion of affecting the user's vision is larger. In order to improve the user experience, the degree of attention to the non-high attention component in the specified frame image should be greater. Therefore, this embodiment obtains the attention level of each non-high attention component in each specified frame in which it appears based on the area of ​​each non-high attention component in the target component in each specified frame in which it appears and its distance from the high attention component.

[0036] Preferably, in a method that can be implemented in this embodiment, the method for obtaining the degree of attention of each non-high attention component in each frame of the specified image in which it appears is: for any non-high attention component and any frame of the specified image in which the non-high attention component appears, obtain the shortest distance between the non-high attention component and each high attention component in the frame of the specified image, all as the first distance; wherein, the method for obtaining the shortest distance between two areas is a well-known technology and will not be repeated. The smaller the first distance, the greater the degree of attention of the non-high attention component in the specified image. Therefore, this embodiment uses the reciprocal of the sum of the minimum first distance and the second preset constant as the first attention target degree; the greater the first attention target degree, the greater the degree of attention of the non-high attention component in the specified image; wherein, the second preset constant is greater than 0, to avoid the situation where the first distance is 0 as the denominator. It is known that the larger the area ratio of the non-high attention component in the specified image of the frame, the greater the degree of attention of the non-high attention component in the specified image of the frame, and thus this embodiment obtains the area ratio of the non-high attention component in the specified image of the frame as the second attention target degree; the larger the second attention target degree, the greater the degree of attention of the non-high attention component in the specified image; in order to accurately obtain the degree of attention of the non-high attention component in the specified image of the frame, the product of the first attention target degree and the second attention target degree is normalized as the reference degree of attention of the non-high attention component in the specified image of the frame; the larger the reference degree of attention, the greater the degree of attention of the non-high attention component by the user in the specified image of the frame; In order to avoid excessively detailed rendering details of non-high-attention components, which would cause unnecessary waste of computing power, this embodiment overall reduces the attention level of non-high-attention components in the specified image of the frame, and then obtains the difference between the total number of levels of the LOD level and the third preset constant as the first value; wherein the third preset constant must be a positive integer less than the total number of levels of the LOD level and greater than 0; then the ratio of the first value to the total number of levels of the LOD level is used as the adjustment weight; finally, the product of the adjustment weight and the reference attention level is used as the attention level of the non-high-attention component in the specified image of the frame.

[0037] The calculation formula for the attention level of a non-high attention component in a specified image is: ; In the formula, Specify the image attention level for the qth non-high attention component at the i-th frame; N is the total number of LOD levels; Assign the smallest first distance in the image to the qth non-high attention component at the i-th frame; Assign the first attention target degree of the image to the qth non-high attention component at the i-th frame; Assign a second attention target degree of the image to the qth non-high attention component at the i-th frame; is the second preset constant; is the third preset constant; norm is the normalization function; Assign the reference attention level of the image to the qth non-high attention component at the i-th frame; is the first value; To adjust the weight.

[0038] This embodiment will Set to 0.1, Set to 1. The implementer can set it according to the actual situation. and The size is not limited here.

[0039] At this point, the attention level of each non-high attention component in each frame of the specified image in which it appears is obtained. It should be noted that when a non-high attention component does not appear in a frame of the specified image, the attention level of the non-high attention component in the frame of the specified image is directly assumed to be 0.

[0040] Step S204: Obtain the target LOD level of each target component in each frame of the specified image based on the degree of attention.

[0041] Specifically, from step S1, it can be seen that the present embodiment sets the total number of LOD levels to 4 levels, namely 0, 1, 2 and 3, and LOD level 0 is the most detailed rendering of the rendering component. In order to obtain the target LOD level of each target component in each frame of a specified image based on the degree of attention, the present embodiment divides the value range of the degree of attention equally and indirectly to obtain multiple degree of attention intervals; wherein the number of degree of attention intervals is the same as the total number of LOD levels; it is known that the value range of the degree of attention is 0-1 and the total number of LOD levels is 4, therefore, there are 4 degree of attention intervals, namely , and then correspond a focus interval to a LOD level, where the maximum focus interval corresponds to the smallest LOD level, that is The corresponding LOD level is 0. The corresponding LOD level is 1. The corresponding LOD level is 2. The corresponding LOD level is 3. For any target component and any frame specified image where the target component appears, when the attention level of the target component in the frame specified image is within a certain attention level interval, the LOD level corresponding to the attention level interval is used as the target LOD level of the target component in the frame specified image. For example, when the attention level of the qth target component in the i-frame specified image is 0.3, the target LOD level of the qth target component in the i-frame specified image is 2.

[0042] At this point, the target LOD level of each target component in each frame of the specified image is obtained.

[0043] Step S3: Obtain the rendering accuracy of each target component according to the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image in which the target component appears.

[0044] It is known that the target LOD level is the LOD level that the rendering component renders in the ideal considering the actual concerns of the user, and the display LOD level is the LOD level directly rendered through the LOD model. In order to accurately analyze whether the LOD level actually rendered by each target component meets the user's concerns, this embodiment analyzes the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image that appears. The smaller the difference between the target LOD level and the display LOD level of a certain target component in each frame of the specified image that appears, the more accurate the LOD level actually rendered by the target component is. This embodiment obtains the rendering accuracy of each target component based on the difference between the target LOD level and the display LOD level of each target component in the specified image that appears. The greater the rendering accuracy, the more the display LOD level of the corresponding target component is the same as the target LOD level.

[0045] Preferably, in a method that can be implemented in this embodiment, the rendering accuracy is obtained by: for any target component, obtaining the absolute value of the difference between the target LOD level and the display LOD level of the target component in each frame of the specified image, all of which are taken as specific differences; negatively correlating and normalizing the sum of all specific differences as the rendering accuracy of the target component. It should be noted that in this embodiment, The sum of all specific differences is negatively correlated and normalized, where exp is an exponential function with a natural constant as the base; x represents the sum of all specific differences.

[0046] At this point, the rendering accuracy of each target component is obtained.

[0047] Step S4: Based on the rendering accuracy and the display LOD level of each target component in the specified image in which it appears, as well as the target LOD level and display LOD level of each target component in the current frame image, obtain the predicted LOD level of each target component in the next frame image of the current frame.

[0048] Specifically, when the target component of the current frame image needs to be rendered and updated, the LOD level of each target component in the current frame image in the next frame image can be predicted based on the rendering accuracy of each target component in the current frame image, and the target LOD level and display LOD level of each target component in the current frame image. Considering that the change trend of the LOD level of each target component in the current frame image will have a certain impact on the predicted LOD level of each target component in the next frame of the current frame, therefore, this embodiment obtains the predicted LOD level of each target component in the next frame image of the current frame based on the rendering accuracy of each target component and the display LOD level of each target component in the specified image in which it appears, as well as the target LOD level and display LOD level of each target component in the current frame image.

[0049] Preferably, in one possible implementation of this embodiment, the method for obtaining the predicted LOD level can be found in Figure 3 , which shows a flow chart of a method for obtaining a predicted LOD level provided in this embodiment, the method comprising the following steps: Step S301: For any target component, when the rendering accuracy of the target component is greater than a preset rendering accuracy threshold, the predicted LOD level of the target component in the next frame image of the current frame is obtained based on the display LOD level of the target component in the current frame image and the display LOD level of the target component in each specified frame in which it appears.

[0050] It is known that the greater the rendering accuracy, the more consistent the LOD level rendered by the corresponding target component in actual situation is with the ideal LOD level. Therefore, for any target component, when the rendering accuracy of the target component is greater than the preset rendering accuracy threshold, the display LOD level of the target component in the current frame image can be used as the predicted LOD level of the target component in the next frame image of the current frame. Considering that the LOD level change trend of the target component itself will also affect the predicted LOD level of the target component in the next frame image of the current frame, this embodiment can predict the LOD level change trend of the target component itself through the display LOD level of the target component in each frame of the specified image. Therefore, this embodiment obtains the predicted LOD level of the target component in the next frame image of the current frame based on the display LOD level of the target component in the current frame image and the display LOD level of the target component in each frame of the specified image. The specific implementation method is as follows: First, the specified images in which the target component appears are arranged in chronological order from front to back to obtain a specified image sequence of the target component; then, the display LOD levels of the target component in each frame of the specified image in which the target component appears are arranged according to the order of the corresponding specified images in the specified image sequence to obtain a display LOD level sequence of the target component; further, the display LOD levels in the display LOD level sequence are fitted into a straight line as the target straight line; wherein, the method of straight line fitting is a well-known technology and will not be described in detail. When the slope of the target straight line is greater than 0, the display LOD level change trend of the target component is an upward trend, and the LOD level change degree of the target component is set to 1 in this embodiment; when the slope of the target straight line is equal to 0, the display LOD level change trend of the target component is a stable trend, and the LOD level change degree of the target component is set to 0 in this embodiment; when the slope of the target straight line is less than 0, the display LOD level change trend of the target component is a downward trend, and the LOD level change degree of the target component is set to -1 in this embodiment; finally, the display LOD level of the target component in the current frame image and the LOD level change degree are added as the predicted LOD level of the target component in the next frame image of the current frame. It should be noted that if the predicted LOD level of the target component in the next frame image of the current frame exceeds the set LOD level range, the nearest boundary LOD level is used as the predicted LOD level of the target component in the next frame image of the current frame. For example, when the predicted LOD level of the target component in the next frame image of the current frame is -1, the predicted LOD level of the target component in the next frame image of the current frame is 0; when the predicted LOD level of the target component in the next frame image of the current frame is 4, the predicted LOD level of the target component in the next frame image of the current frame is 3.

[0051] It should be noted that, in this embodiment, the preset rendering accuracy threshold is set to 0.5, and the implementer may set the value of the preset rendering accuracy threshold according to actual conditions, which is not limited here.

[0052] Step S302: When the rendering accuracy of the target component is less than or equal to a preset rendering accuracy threshold, the predicted LOD level of the target component in the next frame image of the current frame is obtained according to the target LOD level and display LOD level of the target component in the current frame image.

[0053] When the rendering accuracy is less than or equal to the preset rendering accuracy threshold, it means that the LOD level of the actual rendering of the corresponding target component is inaccurate, and the user's concern for the target component is not fully considered. Under the premise that the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, in the current frame image, when the target LOD level of the target component is less than or equal to the display LOD level, it indirectly reflects that the actual rendering of the current target component is more detailed. In order to maintain high-detail information, this embodiment uses the target LOD level of the target component in the current frame image as the predicted LOD level of the target component in the next frame image of the current frame. When the target LOD level of the target component is greater than the display LOD level, in order to improve the rendering details of the target component and better meet the needs of the user, the display LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame. In the process of re-rendering the rendering components in the next frame image of the current frame, this embodiment only processes the target components that need to be updated for rendering, that is, only the target components whose display LOD level in the current frame image is different from the predicted LOD level are re-rendered. This is conducive to the reasonable allocation of computing power, while reducing unnecessary computing power, improving the efficiency of real-time rendering, and at the same time ensuring that the rendering LOD level of the target component meets the user's concerns, effectively improving the user's experience and immersion.

[0054] In summary, the present embodiment obtains the rendering component of the virtual scene image and its display LOD level; takes the rendering component in the current frame image as the target component, and obtains its target LOD level in the specified image according to the distribution of the target component in the specified image; obtains the rendering accuracy according to the difference between the target LOD level and the display LOD level of the target component in the specified image; obtains the predicted LOD level of the target component in the next frame image of the current frame according to the rendering accuracy, the target LOD level of the target component and the display LOD level. The present invention is conducive to the reasonable allocation of computing power in real-time rendering by accurately obtaining the predicted LOD level, and at the same time makes the rendering level of the rendering component more in line with the user's experience, improves the efficiency and accuracy of real-time rendering, and enhances the user's experience and immersion.

[0055] Embodiment 2: The present invention also proposes a virtual scene real-time rendering system for enhancing immersion, see Figure 4 , which shows a structural diagram of a real-time rendering system for a virtual scene with enhanced immersion provided by an embodiment of the present invention. The system includes: an acquisition module 10, a target LOD level acquisition module 20, a rendering accuracy acquisition module 30 and a predicted LOD level acquisition module 40.

[0056] The acquisition module 10 is used to acquire each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component.

[0057] The target LOD level acquisition module 20 is used to take each rendering component in the current frame image as a target component, and obtain the target LOD level of each target component in each frame of the specified image according to the distribution of each target component in each frame of the specified image.

[0058] The rendering accuracy acquisition module 30 is used to acquire the rendering accuracy of each target component according to the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image in which the target component appears.

[0059] The predicted LOD level acquisition module 40 is used to obtain the predicted LOD level of each target component in the next frame image of the current frame based on the rendering accuracy and the display LOD level of each target component in the specified image in which it appears, as well as the target LOD level and display LOD level of each target component in the current frame image.

[0060] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the real-time rendering system of a virtual scene with enhanced immersion and the real-time rendering method of a virtual scene with enhanced immersion provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0061] Embodiment 3: The present invention also proposes a device for real-time rendering of virtual scenes with enhanced immersion, the device comprising a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a method for real-time rendering of virtual scenes with enhanced immersion provided in an embodiment of the present application. The device can be specifically a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a method for real-time rendering of virtual scenes with enhanced immersion provided in the above embodiment.

[0062] In addition, the present application embodiment also protects a computer device, see Figure 5The computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program 403, the computer device can execute any one of the above-mentioned methods for real-time rendering of virtual scenes that enhance immersion.

[0063] Embodiment 4: This embodiment also provides a computer-readable storage medium, which stores a computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a real-time rendering method of a virtual scene with enhanced immersion provided in the above embodiment.

[0064] Embodiment 5: This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement a method for real-time rendering of a virtual scene with enhanced immersion provided in the above embodiment.

[0065] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0066] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0067] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for real-time rendering of a virtual scene for enhancing immersion, characterized in that: The method comprises the following steps: Obtain each rendering component in each frame image of the museum virtual scene and the display LOD level of each rendering component; Each rendering component in the current frame image is taken as a target component, and according to the distribution of each target component in each frame of the specified image, the target LOD level of each target component in each frame of the specified image is obtained; Obtaining the rendering accuracy of each target component according to the difference between the target LOD level and the display LOD level of each target component in each frame of the specified image in which the target component appears; According to the rendering accuracy and the display LOD level of each target component in the specified image in which it appears, as well as the target LOD level and display LOD level of each target component in the current frame image, the predicted LOD level of each target component in the next frame image of the current frame is obtained.

2. A method for real-time rendering of a virtual scene for enhancing immersion according to claim 1, characterized in that: The method for obtaining the target LOD level is: The target components that appear in each frame of the specified image are regarded as high-attention components; According to the rendering color, position distribution and area of ​​each high-attention component in each frame of the specified image, the attention degree of each high-attention component in each frame of the specified image is obtained; According to the area of ​​each non-high attention component in the target component in each specified frame of the image in which it appears and its distance from the high attention component, the attention level of each non-high attention component in each specified frame of the image in which it appears is obtained; Gets the target LOD level of each target component in each frame of the specified image in which it appears based on the degree of attention.

3. A method for real-time rendering of a virtual scene for enhancing immersion according to claim 2, characterized in that: The method for obtaining the attention level of each high attention component in each frame of a specified image is: For any high-attention component and any frame specified image, according to the rendering color of the midpoint of the high-attention component in the frame specified image, the feature points of the high-attention component in the frame specified image and the two-dimensional coordinates of each feature point are obtained by SIFT algorithm; According to the three-dimensional coordinates of each vertex of the high-attention component in the grid model of the frame specified image, the position information of the high-attention component in the frame specified image, and the user's field of view, the display vertices of the high-attention component in the frame specified image and the two-dimensional coordinates of each display vertex are obtained; Obtain the difference between the number of feature points and the number of displayed vertices as a first difference; The reciprocal of the sum of the first difference and the first preset constant is used as the first reference degree of attention; wherein the first preset constant is greater than 0; According to the two-dimensional coordinates of the feature points and the display vertices, the feature points and the display vertices are matched by using a nearest neighbor matching algorithm, and the number of successfully matched feature points is obtained as a first number; Obtaining the area ratio of the high-attention component in the specified image of the frame as a second attention reference degree; A result of normalizing the product of the first quantity, the first attention reference degree, and the second attention reference degree is used as the attention degree of the high attention component in the designated image of the frame.

4. A method for real-time rendering of a virtual scene for enhancing immersion as claimed in claim 2, characterized in that: The method for obtaining the attention level of each non-high attention component in each frame of a specified image in which it appears is: For any non-high attention component and any frame specified image where the non-high attention component appears, obtain the shortest distance between the non-high attention component and each high attention component in the frame specified image, all as the first distance; The inverse of the sum of the minimum first distance and the second preset constant is used as the first target degree of attention; wherein the second preset constant is greater than 0; Obtaining the area ratio of the non-high attention component in the specified image of the frame as the second attention target degree; Normalizing the product of the first attention target degree and the second attention target degree as the reference attention degree of the non-high attention component in the designated image of the frame; The difference between the total number of LOD levels and the third preset constant is used as the first value; wherein the third preset constant must be a positive integer less than the total number of LOD levels and greater than 0; The ratio of the first value to the total number of LOD levels is used as the adjustment weight; The product of the adjusted weight and the reference attention level is taken as the attention level of the non-high attention component in the specified image of the frame.

5. The method for real-time rendering of a virtual scene for enhancing immersion according to claim 2, characterized in that: The method for obtaining the target LOD level is: The value range of the attention level is equally and indirectly divided to obtain a plurality of attention level intervals; wherein the number of attention level intervals is the same as the total number of LOD levels; Correspond one attention level interval to one LOD level, where the maximum attention level interval corresponds to the smallest LOD level; For any target component and any frame specified image in which the target component appears, when the attention level of the target component in the frame specified image is within a certain attention level interval, the LOD level corresponding to the attention level interval is used as the target LOD level of the target component in the frame specified image.

6. The method for real-time rendering of a virtual scene for enhancing immersion according to claim 1, characterized in that: The method for obtaining the rendering accuracy is: For any target component, obtain the difference between the target LOD level and the display LOD level of the target component in each specified frame in which the target component appears, and use them as specific differences; The sum of all specific differences is negatively correlated and normalized to obtain the rendering accuracy of the target component.

7. The method for real-time rendering of a virtual scene for enhancing immersion according to claim 1, characterized in that: The method for obtaining the predicted LOD level is: For any target component, when the rendering accuracy of the target component is greater than a preset rendering accuracy threshold, according to the display LOD level of the target component in the current frame image and the display LOD level of the target component in each specified frame image in which it appears, obtain the predicted LOD level of the target component in the next frame image of the current frame; When the rendering accuracy of the target component is less than or equal to a preset rendering accuracy threshold, the predicted LOD level of the target component in the next frame image of the current frame is obtained according to the target LOD level and display LOD level of the target component in the current frame image.

8. A method for real-time rendering of a virtual scene for enhancing immersion according to claim 7, characterized in that: When the rendering accuracy of the target component is greater than a preset rendering accuracy threshold, the method for obtaining the predicted LOD level of the target component in the next frame image of the current frame according to the display LOD level of the target component in the current frame image and the display LOD level of the target component in each specified frame image in which the target component appears is: Arrange the specified images in which the target component appears in a time sequence from front to back to obtain a specified image sequence of the target component; Arranging the display LOD levels of the target component in each frame of the specified image according to the order of the corresponding specified images in the specified image sequence to obtain a display LOD level sequence of the target component; Fitting the display LOD levels in the display LOD level sequence into a straight line as a target straight line; When the slope of the target straight line is greater than 0, the LOD level change degree of the target component is set to 1; When the slope of the target straight line is equal to 0, the LOD level change degree of the target component is set to 0; When the slope of the target straight line is less than 0, the LOD level change degree of the target component is set to -1; The sum of the displayed LOD level of the target component in the current frame image and the degree of change of the LOD level is used as the predicted LOD level of the target component in the next frame image of the current frame.

9. The method for real-time rendering of a virtual scene for enhancing immersion according to claim 7, characterized in that: When the rendering accuracy of the target component is less than or equal to the preset rendering accuracy threshold, the method for obtaining the predicted LOD level of the target component in the next frame image of the current frame according to the target LOD level and the display LOD level of the target component in the current frame image is: In the current frame image, when the target LOD level of the target component is less than or equal to the display LOD level, the target LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame; When the target LOD level of the target component is greater than the display LOD level, the display LOD level of the target component in the current frame image is used as the predicted LOD level of the target component in the next frame image of the current frame.

10. A virtual scene real-time rendering system for enhancing immersion, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When executing the computer program, the processor implements the steps of a method for real-time rendering of a virtual scene for enhancing immersion as described in any one of claims 1 to 9.

Images