Intelligent display system based on MR application scene

Abstract

The invention relates to the technical field of MR intelligent display, in particular to an intelligent display system based on an MR application scene. Comprising a server, a rendering monitoring module and an intelligent display module, the method comprises the following steps: respectively carrying out quantitative analysis on rendering performance and rendering requirements of MR scene application to obtain a rendering requirement value and a rendering performance value, calculating to obtain a rendering requirement ratio according to the rendering requirement value, and judging whether a rendering strategy needs to be optimized according to the rendering requirement ratio; performing comprehensive analysis on the region parameters of each circular ring region to obtain a complex value, performing numerical analysis on the complex value and the radius of each circular ring to obtain a resolution ratio drop ratio of each circular ring region, keeping the highest resolution of the circular region closest to the user, and sequentially adjusting other circular ring regions according to the distances between the other circular ring regions and the user; and after each adjustment, returning to the rendering monitoring module for re-analysis, and further adjusting according to the feedback of the rendering demand ratio to form a closed-loop feedback mechanism, thereby being capable of adapting to the user demand and scene change display in real time.

Description

Technical Field

[0001] The present invention relates to the field of MR intelligent display technology, and in particular to an intelligent display system based on MR application scenarios. Background Art

[0002] MR application scenarios refer to various practical application scenarios created by combining the virtual world with the real world under the Mixed Reality (MR) technology. In these scenarios, MR not only overlays virtual information into the real environment, but also allows interaction between virtual and real objects. MR allows users to see and interact with objects in the real environment, rather than being completely in a virtual world, which can increase the user's sense of reality and immersion.

[0003] Currently, virtual objects in MR scenes need to be seamlessly integrated with the real world, which requires a large amount of image calculations, including lighting calculations, texture mapping, shadow rendering, reflection / refraction processing, etc. These computing tasks need to be completed in a very short time, which affects the balance between picture quality and rendering speed, causing the system to freeze or delay, especially when dealing with a large number of virtual objects or highly detailed scenes. This not only affects the user experience, but may also cause dizziness and other discomfort. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent display system based on MR application scenarios to solve the problems of image calculation complexity, rendering delay and user experience caused by the seamless integration of virtual objects and the real world in MR application scenarios.

[0005] The object of the present invention can be achieved by the following technical solutions: An intelligent display system based on MR application scenarios, comprising: a server, a rendering monitoring module and an intelligent display module;

[0006] The server collects scene parameters of MR application scenes and system performance parameters in real time and saves them;

[0007] The rendering monitoring module quantitatively analyzes the rendering requirements and rendering performance based on the scene parameters and performance parameters of the MR application scenario to obtain the rendering requirement value and the rendering performance value, and performs numerical analysis on them to obtain the rendering requirement ratio U; if U ≥ 1, a rendering adjustment instruction is generated to the intelligent display module;

[0008] The intelligent display module performs regional resolution adjustment optimization based on the received rendering adjustment instruction, specifically:

[0009] 1-1: With the user as the center, draw the viewing range of the MR application scenario and divide it into a circle closest to the user and several rings, specifically: the first circle, the first ring, the second ring...the Qth ring, where Q is a positive integer, indicating the total number of rings;

[0010] 1-2: Obtain the regional parameters of each annular area. The specific annular area parameters include the number of triangle faces, resolution, number of light sources and number of special effects, and record them as X area, L area, G area and D area respectively; normalize the number of triangle faces X area, resolution L area, number of light sources G area and number of special effects D area of ​​each area and take their values, and calculate the values ​​in a formula to obtain the complex value Kσ of each annular area. The specific calculation formula is:

[0011] Kσ=σ1×X 区域 +σ2×L 区域 +σ3×G 区域 +σ4×D 区域

[0012] Among them, σ1, σ2, σ3, and σ4 are set proportional constants respectively;

[0013] 1-3: The radius Rq of the circular ring area is taken as the distance between the circular ring area and the user, and it is normalized with the corresponding complexity value Kσ and its value is taken. The value is calculated and analyzed to obtain the resolution reduction ratio of each circular ring area. Resolution reduction ratio refers to the ratio of reducing resolution in image or scene rendering; the specific calculation formula is:

[0014]

[0015] Among them, θ1 and θ2 are the set proportional coefficients respectively;

[0016] 1-4: Sort the circular areas in order of their distance from the user from far to near, and adjust each circular area according to the corresponding resolution reduction ratio in order. After each adjustment, return to the rendering monitoring module for re-analysis to obtain the rendering requirement ratio. When the rendering requirement ratio U<1 is met, the adjustment is stopped.

[0017] 1-5: After each adjustment, the resolution difference between two adjacent rings is calculated. When the difference is greater than the set difference threshold, the interpolation technology is used to fill in frames at the adjacent positions of the two rings.

[0018] Preferably, the specific division method of dividing the viewing range into regions is:

[0019] The viewing range is divided into several distance segments, each segment corresponds to a concentric ring area, so that the viewing range can be divided into a circle closest to the user and several ring areas. The specific radius of each ring is Rq, where q = 1, 2, 3...Q, Q is a positive integer, Q is the total number of rings, and q is the serial number of any one of the rings; the specific radius calculation formula is:

[0020] Rq=R0×log2(q+1)

[0021] Among them, R0 is the radius of the first circle, and the line of sight range is divided into: the first circle, the first ring, the second ring...the Qth ring.

[0022] Preferably, the rendering requirement analysis based on scene parameters is specifically as follows:

[0023] 3-1: Retrieve the scene parameters of the MR application scene. The specific scene parameters include the number of triangle faces, the number of texture maps and the resolution of each texture map, the number of illumination sources, the type of special effects and the number of special effects corresponding to each type of special effect, and the viewing distance range;

[0024] 3-2: Analyze the types of special effects in the MR application scenarios and the number of special effects corresponding to each type of special effect to measure the complexity of the special effects and obtain the special effect value accordingly;

[0025] 3-3: The number of triangle faces, the resolution of each texture map, the number of illumination sources and the viewing distance range are recorded as X, Lj, G and W respectively, and they are normalized with the special effect value SV and their values ​​are taken. The rendering requirement value XR is obtained by numerical analysis, where j = 1, 2, 3...J, J is a positive integer, J is the total number of texture maps, and j is the serial number of any texture map; the specific calculation formula is:

[0026]

[0027] Among them, β1, β2, β3, β4, and β5 are the set proportional constants respectively.

[0028] Preferably, the specific steps of measuring the complexity of the special effects and obtaining the special effects value are as follows:

[0029] Set each special effect type to correspond to a special effect base value, extract the special effect type and compare it with all the set special effect types to match the corresponding special effect base value, thereby obtaining the special effect base value and special effect quantity corresponding to each special effect type, and record them as Vi and Si respectively, where i = 1, 2, 3...I, I is a positive integer, I is the total number of special effect types, and i is any special effect type; normalize the special effect base value Vi and the special effect quantity Si and take their values, and calculate the values ​​in a formula to obtain the special effect value SV; the specific calculation formula is:

[0030]

[0031] Preferably, the rendering performance analysis based on the performance parameters is specifically as follows:

[0032] 5-1: Retrieve performance parameters, including frame rate, rendering time, disk read and write speed, memory usage, and CPU utilization;

[0033] 5-2: Get the frame rate per second, build a two-dimensional rectangular coordinate system with time as the horizontal coordinate and frame rate as the vertical coordinate, input the frame rate into the coordinate axis according to the corresponding number of seconds, and record the position of the frame rate in the coordinate axis as the frame rate point, use a smooth curve to connect each frame rate point in sequence to draw a curve of frame rate change over time, and perform frame rate trend analysis to obtain the frame rate monitoring value;

[0034] 5-3: Retrieve the rendering time of each frame, establish a two-dimensional rectangular coordinate system, mark each time interval in the horizontal axis in order, and the vertical axis is the quantity; when the rendering time of a frame is in one of the time intervals, the quantity corresponding to the time interval increases by one, and the rendering time of each frame is input into the coordinate axis to construct a rendering histogram. The peak in the rendering histogram is a rectangle with a fixed width and the quantity corresponding to the time interval as the height, and the quantity corresponding to the time interval is used as the peak height; perform graphical analysis on the rendering histogram to obtain the rendering monitoring value;

[0035] 5-4: Normalize the frame rate monitoring value BP, rendering monitoring value CH, memory usage Y1 and CPU utilization Y2 and take their values. Perform formulaic calculation and analysis on the values ​​to obtain the rendering performance value CY. The specific calculation formula is:

[0036]

[0037] Among them, δ1, δ2, δ3, and δ4 are the set proportional constants respectively.

[0038] Preferably, the specific steps of frame rate trend analysis are:

[0039] Draw a tangent line of the curve at each frame rate point, calculate the slope of the tangent line, sum the slopes greater than zero to obtain the frame rate increase value, and sum the slopes less than zero and take the absolute value to obtain the frame rate decrease value;

[0040] The frame rate Fa, slope Pa, frame rate increase value B1 and frame rate decrease value B2 are normalized and their values ​​are taken. The values ​​are analyzed and calculated by formula to obtain the frame rate monitoring value BP. The specific calculation formula is:

[0041]

[0042] Among them, α1, α2, and α3 are the set proportional constants respectively.

[0043] Preferably, the specific steps of performing graphical analysis on the rendering histogram are:

[0044] 7-1: Count the number of peaks in the rendered histogram;

[0045] 7-2: Assume that there is a duration threshold value denoted as T, and mark the duration threshold value on the horizontal axis; mark the peak on the right side of the duration threshold value as the long-tail peak, and mark the peak on the left side of the duration threshold value as the short-tail peak; calculate the average of the upper and lower limits of the duration interval corresponding to each long-tail peak to obtain the horizontal midpoint value of the long-tail peak, and retrieve the peak height of the long-tail peak; normalize the peak height H3 and the horizontal midpoint value H2 of each long-tail peak and take their values, and perform formulaic calculation and analysis on the values ​​to obtain the long-tail value Hlongtail; the specific calculation formula is:

[0046]

[0047] Thus, the long tail value of each long tail peak can be obtained, and the sum of them is calculated to obtain the total value of the long tail and recorded as ZHlongtail; 7-3: Similarly, each short tail peak is processed in the same way as 7-2 to obtain each short tail value Hshorttail, and the sum of them is calculated to obtain the total value of the short tail and recorded as ZHshorttail;

[0048] 7-3: The peak number H1, the total value of the long tail is recorded as ZHlongtail, and the total value of the short tail is recorded as ZHshorttail. They are normalized and their values ​​are taken. The values ​​are calculated by formula to obtain the rendering monitoring value CH. The specific calculation formula is:

[0049]

[0050] Among them, γ1, γ2, and γ3 are proportional constants set respectively.

[0051] Beneficial effects of the present invention:

[0052] 1. Quantitatively analyze the rendering performance and rendering requirements of MR scene applications to obtain rendering requirement values ​​and rendering performance values, and calculate the rendering requirement ratio based on them. According to the rendering requirement ratio, determine whether the current frame rate is high enough and whether the rendering is stable enough, so as to decide whether the rendering strategy needs to be optimized; provide data support for the optimization and adjustment strategy of MR scenes;

[0053] 2. The number of triangle faces, resolution, number of light sources and number of animations in each circular area are comprehensively analyzed to obtain the complexity value, and then the complexity value and the radius of each circular area are numerically analyzed to obtain the resolution reduction ratio of each circular area. The highest resolution of the circular area closest to the user is maintained, and the other circular areas are adjusted in sequence according to their distance from the user. After each adjustment, it is returned to the rendering monitoring module for re-analysis, and further adjustments are made based on the feedback of the rendering demand ratio (U), forming a closed-loop feedback mechanism that can adapt to user needs and scene changes in real time to maintain the best rendering effect and system performance;

[0054] 3. The interpolation technology eliminates the resolution differences between adjacent circles, reducing the visual abruptness and making the user's visual experience in different areas more coherent. This smooth transition reduces user fatigue and enhances the sense of immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a schematic diagram of system module connection of the present invention;

[0056] Figure 2 is the rendering histogram of the present invention. DETAILED DESCRIPTION

[0057] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0059] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0060] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0061] See also Figure 1 As shown, the present invention is an intelligent display system based on MR application scenarios, including: a server, a rendering monitoring module and an interactive display module;

[0062] The server collects scene parameters of MR application scenes and system performance parameters in real time and saves them;

[0063] The rendering monitoring module analyzes the rendering load of the current MR application scenario to determine whether frame rendering adjustment is required; specifically:

[0064] Retrieve scene parameters and performance parameters of the MR application scene. Specific scene parameters include the number of triangle faces, the number of texture maps and the resolution of each texture map, the number of illumination sources, the type of special effects, and the number of special effects and the viewing distance range corresponding to each type of special effect. Performance parameters include frame rate, rendering time (the time required for single-frame rendering from start to finish), disk read and write speed, latency (the time delay from user interaction to display response, usually in milliseconds), memory usage (the occupancy of system and video memory when rendering MR scenes), and CPU utilization (the occupancy of the CPU when rendering and processing MR scenes).

[0065] It should be noted that: (1) Number of triangle faces: In the actual rendering process, the polygons of virtual objects will be divided into triangles for processing. The reason is that triangles are the simplest and most stable shapes in computer graphics. All complex polygons will eventually be divided into multiple triangles when rendering. Therefore, the more triangle faces there are, the more vertices, edges and surfaces need to be calculated, which will increase the computational load of rendering and thus increase the complexity of the scene. (2) Texture map: Texture map refers to the image attached to the surface of the virtual object to simulate the visual details in the real world, such as color, gloss, bumps, etc. The higher the resolution of the texture map used by the virtual object, the larger the amount of texture data that needs to be processed during rendering, and the greater the rendering difficulty. (3) Number of scene lighting sources: The number of light sources in the scene will significantly affect the complexity of lighting calculations, especially light sources with shadows, reflections or refractions. The more light sources in the scene, the greater the rendering pressure. (4) Number of special effects: The more special effects are applied in the scene, such as particle systems, reflections, refractions, motion blur, ambient occlusion, etc., the greater the rendering complexity.

[0066] The scene parameters of the MR application scene are analyzed based on the scene parameters to determine the scene rendering difficulty, specifically:

[0067] Scene parameters include the number of triangles, the number of texture maps and the resolution of each texture map, the number of illumination sources, the type of special effects and the number of special effects corresponding to each type of special effect, and the viewing distance range (i.e., the maximum distance from the user to the farthest object in the scene);

[0068] Each special effect type is set to correspond to a special effect base value. It should be noted that different types of special effects have different rendering difficulties and corresponding special effect base values. Simple color change special effects have little impact on performance, while special effects such as ambient occlusion or global illumination involve more complex lighting calculations and consume more system resources. The particle system needs to simulate the movement of hundreds or thousands of particles in real time, which will greatly increase the computing burden of the graphics card, while dynamic blur requires calculating the object movement trajectory and blur effect in each frame. For example: the special effect base value of simple lighting effect is 40, the special effect base value of dynamic blur is 80, the special effect base value of particle system is 100, and the special effect base value of ambient occlusion is 120. Extract the special effect type and compare it with all the set special effect types to match the corresponding special effect base value, thereby obtaining the special effect base value and the number of special effects corresponding to each special effect type, and record them as Vi and S respectively. i, where i = 1, 2, 3 ... I, I is a positive integer, I is the total number of special effect types, and i is any one of the special effect types; the special effect base value Vi and the special effect quantity Si are normalized and their values ​​are taken, and the values ​​are calculated formulaically to obtain the special effect value SV; the specific calculation formula is:

[0069]

[0070] The number of triangle faces, the resolution of each texture map, the number of illumination sources and the viewing distance range are recorded as X, Lj, G and W respectively, and they are normalized with the special effect value SV and their values ​​are taken. The rendering requirement value XR is obtained by numerical analysis, where j = 1, 2, 3...J, J is a positive integer, J is the total number of texture maps, and j is the serial number of any texture map; the specific calculation formula is:

[0071]

[0072] Among them, β1, β2, β3, β4, and β5 are respectively set proportional constants. From the formula, it can be seen that when the number of triangles is larger, the texture map resolution is larger, the number of light sources is more, the special effect value is larger, and the viewing distance range is larger, it means that the rendering complexity of the MR application scene is greater, and the corresponding rendering demand value is larger;

[0073] The performance parameters of the MR application scenario are analyzed based on the performance parameters to determine the performance tolerance, specifically:

[0074] Performance parameters include frame rate, rendering time (the time required for single-frame rendering from start to finish), disk read and write speed, memory usage (the occupancy of system and video memory when rendering MR scenes) and CPU utilization (the occupancy of CPU when rendering and processing MR scenes);

[0075] The frame rate is recorded as Fa, where the unit of the frame rate is frames per second, which means the number of image frames rendered and displayed per second; a = 1, 2, 3 ... A, A is a positive integer, A represents the total number of seconds, and a represents the serial number of any second; a two-dimensional rectangular coordinate system is constructed with time as the horizontal coordinate and frame rate as the vertical coordinate, the frame rate is input into the coordinate axis according to the corresponding number of seconds, and the position of the frame rate in the coordinate axis is recorded as the frame rate point, and a smooth curve is used to connect each frame rate point in sequence to draw a curve of frame rate changing with time; a tangent line is drawn at each frame rate point, and the slope of the tangent line is calculated and recorded as Pa; it should be noted that , the lower the frame rate, the more likely the system is overloaded, and the rendering load needs to be adjusted; if the frame rate fluctuates greatly, it means that the rendering is unstable, and the system performance may change dramatically between different frames, which will affect the user's visual experience; the slopes greater than zero are summed up to obtain the frame rate increase value recorded as B1, and the slopes less than zero are summed up and the absolute value is taken to obtain the frame rate decrease value recorded as B2; the frame rate Fa, slope Pa, frame rate increase value B1 and frame rate decrease value B2 are normalized and their values ​​are taken, and the values ​​are analyzed by formula calculation to obtain the frame rate monitoring value BP. The specific calculation formula is:

[0076]

[0077] Among them, α1, α2, and α3 are respectively set proportional constants. From the formula, we can know that when the frame rate per second is larger and the frame rate fluctuation is smaller, the frame rate monitoring value is larger, indicating that the rendering is smoother;

[0078] Get the rendering time of each frame; Figure 2 As shown, several duration intervals are set (the duration intervals here are usually in milliseconds, for example, a certain duration interval is 10-12 milliseconds, which is specifically set by those skilled in the art according to the needs of the application scenario), a two-dimensional rectangular coordinate system is established, and each duration interval is marked on the horizontal axis in order, and the vertical axis is the quantity; when the rendering duration of a frame is in one of the duration intervals, the quantity corresponding to the duration interval is increased by one, and the rendering duration of each frame is input into the coordinate axis to construct a rendering histogram. The peak in the rendering histogram is a rectangle drawn with a fixed width and the quantity corresponding to the time interval as the height, and the quantity corresponding to the time interval is used as the peak height; the specific analysis of the rendering histogram is as follows:

[0079] The number of peaks in the statistical rendering histogram is recorded as H1. When the number of peaks is small, it means that the rendering is more concentrated and the fluctuation is smaller; when the number of peaks is large, it means that the distribution of rendering time spans multiple intervals and is not concentrated, indicating that the stability of the rendering frame is poor. Frequent rendering fluctuations will cause the screen to freeze, and users may feel obvious delays and visual discomfort;

[0080] It is assumed that there is a duration threshold denoted as T (usually the duration threshold is 20 milliseconds, and the specific value is set by the person skilled in the art according to the scene requirements). When the rendering duration is greater than the duration threshold, it indicates that there is a delay; the duration threshold is marked on the horizontal axis, the peak on the right side of the duration threshold is denoted as the long-tail peak, and the peak on the left side of the duration threshold is denoted as the short-tail peak; the upper and lower limits of the duration interval corresponding to each long-tail peak are averaged to obtain the horizontal midpoint value of the long-tail peak and denoted as H2, and the peak height of the long-tail peak is retrieved and denoted as H3; the peak height H3 and the horizontal midpoint value H2 of each long-tail peak are normalized and their values ​​are taken, and the values ​​are analyzed by formula calculation to obtain the long-tail value Hlongtail; the specific calculation formula is:

[0081]

[0082] From the formula, we can see that since the long-tail peak is on the right side of the time threshold, the horizontal midpoint value of the long-tail peak is greater than the time threshold, so When the horizontal center value of the long-tail peak is larger, it means that the peak is more biased to the right, and the time interval corresponding to the peak is obviously longer (for example, more than 20 milliseconds). The larger the peak height is, the insufficient rendering performance is, and the smaller the long-tail value is. Thus, the long-tail value of each long-tail peak can be obtained, and the sum of them is calculated to obtain the total long-tail value recorded as ZH long-tail;

[0083] Similarly, the upper and lower limits of the corresponding time intervals of each short-tail peak are averaged to obtain the horizontal midpoint value of the short-tail peak, recorded as H4, and the peak height of the short-tail peak is retrieved and recorded as H5; the peak height H5 and the horizontal midpoint value H4 of each short-tail peak are normalized and their values ​​are taken, and the values ​​are analyzed and calculated by formula to obtain the short-tail value Hshorttail; the specific calculation formula is:

[0084]

[0085] From the formula, we can see that since the short-tail peak is on the left side of the time threshold, the horizontal midpoint value of the short-tail peak is less than the time threshold, so The smaller the horizontal midpoint value of the short-tail peak, the larger the peak height, indicating that the rendering frame is completed in a shorter time, the more stable the rendering performance, and the larger the short-tail value; thus, the short-tail value of each short-tail peak can be obtained, and the sum of the short-tail values ​​is calculated to obtain the total short-tail value recorded as ZH short-tail;

[0086] The peak number H1, the total long tail value ZHlongtail, and the total short tail value ZHshorttail are normalized and their values ​​are taken. The rendering monitoring value CH is obtained by formula calculation. The specific calculation formula is:

[0087]

[0088] Among them, γ1, γ2, and γ3 are respectively set proportional constants. From the formula, we can know that when the number of peaks is smaller, the total value of the short tail is larger, and the total value of the long tail is larger, it means that the rendering efficiency is higher, the single frame rendering time is shorter, and the rendering monitoring value is larger;

[0089] The frame rate monitoring value BP, rendering monitoring value CH, memory usage Y1 and CPU utilization Y2 are normalized and their values ​​are taken. The rendering performance value CY is obtained by formula calculation and analysis of the values. The specific calculation formula is:

[0090]

[0091] Among them, δ1, δ2, δ3, and δ4 are respectively set proportional constants. From the formula, it can be seen that when the frame rate monitoring value is larger, the rendering monitoring value is larger, the memory usage is smaller, and the CPU utilization is smaller, the rendering performance value is larger;

[0092] When the rendering requirement value XR and the rendering performance value CY are set by the formula Calculate and obtain the rendering demand ratio U, where μ1 and μ2 are respectively set proportional constants; if U < 1, it means that the rendering performance of the current system is sufficient to support the current rendering demand, will not cause delay or freeze, and maintain the stability of the frame rate, and no adjustment is required; if U ≥ 1, a rendering adjustment instruction is generated to the intelligent display module;

[0093] By quantitatively analyzing the rendering performance and rendering requirements of MR scene applications, we can obtain the rendering requirement value and rendering performance value, and calculate the rendering requirement ratio based on this. Based on the rendering requirement ratio, we can judge whether the current frame rate is high enough and whether the rendering is stable enough, so as to decide whether the rendering strategy needs to be optimized; providing data support for the optimization and adjustment strategy of MR scenes.

[0094] The intelligent display module dynamically adjusts the MR application scene based on the received rendering adjustment instruction to achieve dynamic adjustment of rendering optimization, making the presentation of the MR application scene smoother; specifically:

[0095] 1-1: With the user as the center, draw the viewing range of the MR application scenario, and divide the viewing range into several distance segments, each segment corresponds to a concentric ring area, so that the viewing range can be divided into a circle closest to the user and several ring areas, and the specific radius of each ring is Rq, where q = 1, 2, 3...Q, Q is a positive integer, Q is the total number of rings, and q is the serial number of any one of the rings; the specific radius calculation formula is:

[0096] Rq=R0×log2(q+1)

[0097] Where R0 is the radius of the first circle (the circle closest to the user), and the logarithmic function is used to gradually widen the concentric rings, which can smoothly increase the viewing distance, and the central area is relatively dense, providing a basis for achieving smooth resolution transition; thus, the viewing distance range can be divided into: the first circle, the first ring, the second ring... the Qth ring, where the values ​​of R0 and Q are specifically set by those skilled in the art according to actual needs, and the rings here are all concentric rings, and the specific center of the circle is the user's position;

[0098] 1-2: Obtain the regional parameters of each annular area. The specific annular area parameters include the number of triangle faces, resolution, number of light sources and number of special effects, and record them as X area, L area, G area and D area respectively; normalize the number of triangle faces X area, resolution L area, number of light sources G area and number of special effects D area of ​​each area and take their values, and calculate the values ​​in a formula to obtain the complex value Kσ of each annular area. The specific calculation formula is:

[0099] Kσ=σ1×X 区域 +σ2×L 区域 +σ3×G 区域 +σ4×D 区域

[0100] Among them, σ1, σ2, σ3, and σ4 are respectively set proportional constants. From the formula, we can see that when the number of triangles in the area is more, the resolution is higher, the number of light sources is more, and the number of animations is more, the greater the complexity of the area, the greater the complexity value;

[0101] 1-3: The radius Rq of the circular ring area is taken as the distance between the circular ring area and the user, and it is normalized with the corresponding complexity value Kσ and its value is taken. The value is calculated and analyzed to obtain the resolution reduction ratio of each circular ring area. Resolution reduction ratio refers to the ratio of reducing resolution in image or scene rendering; the specific calculation formula is:

[0102]

[0103] θ1 and θ2 are the set proportional coefficients. The larger the radius of the ring, the greater the distance from the user, and the greater the reduction in resolution. The greater the complexity, the smaller the reduction in resolution, which can maintain the delicate and realistic visual experience.

[0104] 1-4: Sort the circular areas in order of their distance from the user from far to near, and adjust each circular area according to the corresponding resolution reduction ratio in order. After each adjustment, return to the rendering monitoring module for re-analysis to obtain the rendering demand ratio, until the rendering demand ratio U < 1, then stop adjusting; It should be noted that the range occupied by the circle closest to the user, that is, the core area or focus area of ​​the user's line of sight, maintains the highest resolution and frame rate, and does not adjust the resolution to ensure the delicacy and coherence of the visual experience and avoid affecting the user's immersion;

[0105] 1-5: After each adjustment, the resolution difference between two adjacent rings is calculated. When the difference is greater than the set difference threshold, the interpolation technology is used to fill in frames at the adjacent positions of the two rings to avoid obvious resolution faults between the rings and reduce the visual abruptness caused by the resolution change;

[0106] The complexity value is obtained by comprehensively analyzing the number of triangle faces, resolution, number of light sources and number of animations in each circular area. The complexity value and the radius of each circular area are then numerically analyzed to obtain the resolution reduction ratio of each circular area. The highest resolution of the circular area closest to the user is maintained, and the other circular areas are adjusted in turn according to their distance from the user. After each adjustment, they are returned to the rendering monitoring module for re-analysis, and further adjustments are made based on the feedback of the rendering demand ratio (U), forming a closed-loop feedback mechanism that can adapt to user needs and scene changes in real time to maintain the best rendering effect and system performance. At the same time, the resolution difference between adjacent circular areas is eliminated through interpolation technology, which reduces the visual abruptness and makes the user's visual experience in different areas more coherent. This smooth transition reduces user fatigue and enhances immersion.

[0107] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. Intelligent display system based on MR application scenarios, including: Rendering monitoring module and intelligent display module; characterized by: The rendering monitoring module quantitatively analyzes the rendering requirements and rendering performance based on the scene parameters and performance parameters of the MR application scenario to obtain the rendering requirement value and the rendering performance value, and performs numerical analysis on them to obtain the rendering requirement ratio U; if U ≥ 1, a rendering adjustment instruction is generated to the intelligent display module; The intelligent display module performs regional resolution adjustment optimization based on the received rendering adjustment instruction, specifically: 1-1: With the user as the center, draw the viewing range of the MR application scenario and divide it into a circle closest to the user and several rings, specifically: the first circle, the first ring, the second ring...the Qth ring, where Q is a positive integer, indicating the total number of rings; 1-2: Obtain the regional parameters of each annular area, which include the number of triangle faces, resolution, number of light sources and number of special effects, and analyze them to obtain the complex value of each annular area; 1-3: The radius of the circular ring area is used as the distance between the circular ring area and the user, and the radius is analyzed with the corresponding complexity value to obtain the resolution reduction ratio of each circular ring area; 1-4: Sort the circular areas in order of their distance from the user from far to near, and adjust each circular area according to the corresponding resolution reduction ratio in order. After each adjustment, return to the rendering monitoring module for re-analysis to obtain the rendering requirement ratio. When the rendering requirement ratio U<1 is met, the adjustment is stopped. 1-5: After each adjustment, the resolution difference between two adjacent rings is calculated. When the difference is greater than the set difference threshold, the interpolation technology is used to fill in frames at the adjacent positions of the two rings.

2. The intelligent display system based on MR application scenario according to claim 1 is characterized in that: The specific division method of regional division of sight range is as follows: The viewing range is divided into several distance segments, each segment corresponds to a concentric ring area, so that the viewing range can be divided into a circle closest to the user and several ring areas. The specific radius of each ring is Rq, where q = 1, 2, 3...Q, Q is a positive integer, Q is the total number of rings, and q is the serial number of any one of the rings; the specific radius calculation formula is: Rq=R0×log2(q+1) Among them, R0 is the radius of the first circle, and the line of sight range is divided into: the first circle, the first ring, the second ring...the Qth ring.

3. The intelligent display system based on MR application scenario according to claim 1 is characterized in that: The rendering requirements analysis based on scene parameters is as follows: 3-1: Retrieve the scene parameters of the MR application scene. The specific scene parameters include the number of triangle faces, the number of texture maps and the resolution of each texture map, the number of illumination sources, the type of special effects and the number of special effects corresponding to each type of special effect, and the viewing distance range; 3-2: Analyze the types of special effects in the MR application scenarios and the number of special effects corresponding to each type of special effect to measure the complexity of the special effects and obtain the special effect value accordingly; 3-3: The number of triangle faces, the resolution of each texture map, the number of lighting sources, the viewing distance range and the special effect value are normalized and analyzed to obtain the rendering requirement value.

4. The intelligent display system based on MR application scenario according to claim 3 is characterized in that: The specific steps to measure the complexity of the special effects and obtain the special effects value are as follows: Set each special effect type to correspond to a special effect base value, extract the special effect type and compare it with all the set special effect types to match the corresponding special effect base value, thereby obtaining the special effect base value and the number of special effects corresponding to each special effect type, and normalize them to calculate the special effect value.

5. The intelligent display system based on MR application scenario according to claim 1, characterized in that: The rendering performance analysis based on performance parameters is as follows: 5-1: Retrieve performance parameters, including frame rate, rendering time, disk read and write speed, memory usage, and CPU utilization; 5-2: Get the frame rate per second, build a two-dimensional rectangular coordinate system with time as the horizontal coordinate and frame rate as the vertical coordinate, input the frame rate into the coordinate axis according to the corresponding number of seconds, and record the position of the frame rate in the coordinate axis as the frame rate point, use a smooth curve to connect each frame rate point in sequence to draw a curve of frame rate change over time, and perform frame rate trend analysis to obtain the frame rate monitoring value; 5-3: Retrieve the rendering time of each frame, establish a two-dimensional rectangular coordinate system, mark each time interval in the horizontal axis in order, and the vertical axis is the quantity; when the rendering time of a frame is in one of the time intervals, the quantity corresponding to the time interval increases by one, and the rendering time of each frame is input into the coordinate axis to construct a rendering histogram. The peak in the rendering histogram is a rectangle with a fixed width and the quantity corresponding to the time interval as the height, and the quantity corresponding to the time interval is used as the peak height; perform graphical analysis on the rendering histogram to obtain the rendering monitoring value; 5-4: Normalize the frame rate monitoring value, rendering monitoring value, memory usage and CPU utilization and take their numerical values, perform formula calculation and analysis on the numerical values ​​to obtain the rendering performance value.

6. The intelligent display system based on MR application scenario according to claim 5, characterized in that: The specific steps for frame rate trend analysis are as follows: Draw a tangent line of the curve at each frame rate point, calculate the slope of the tangent line, sum the slopes greater than zero to obtain the frame rate increase value, and sum the slopes less than zero and take the absolute value to obtain the frame rate decrease value; The frame rate, slope, frame rate increase value and frame rate decrease value are normalized and their values ​​are taken, and the values ​​are analyzed by formula calculation to obtain the frame rate monitoring value.

7. The intelligent display system based on MR application scenario according to claim 6, characterized in that: The specific steps for graphical analysis of the rendering histogram are: 7-1: Count the number of peaks in the rendered histogram; 7-2: Assume that there is a duration threshold value denoted as T, and mark the duration threshold value on the horizontal axis; mark the peak on the right side of the duration threshold value as the long-tail peak, and mark the peak on the left side of the duration threshold value as the short-tail peak; calculate the average of the upper and lower limits of the duration interval corresponding to each long-tail peak to obtain the horizontal midpoint value of the long-tail peak, and retrieve the peak height of the long-tail peak; normalize the peak height H3 and the horizontal midpoint value H2 of each long-tail peak and take their values, and perform formulaic calculation and analysis on the values ​​to obtain the long-tail value H 长尾 ; The specific calculation formula is: Thus, the long tail values ​​of each long tail peak can be obtained, and the sum of them is calculated to obtain the total long tail value, which is recorded as ZH 长尾 ; 7-3: Similarly, each short-tail peak is treated the same as 7-2 to obtain each short-tail value H 短尾 , and sum them up to get the short-tail total value, recorded as ZH 短尾 ; 7-3: Record the number of peaks H1 and the total value of the long tail as ZH 长尾 , the short-tail total value is recorded as ZH 短尾 Perform normalization processing and take its value, and perform formula calculation on the value to obtain the rendering monitoring value CH. The specific calculation formula is: Among them, γ1, γ2, and γ3 are the set proportional constants respectively.

Images