Image rendering method and system, VR equipment, device and readable storage medium

Abstract

The present application discloses an image rendering method and system, VR equipment, apparatus and readable storage medium, which pre-renders the background of an image to be displayed to generate a background panorama; determines the background image that the user is watching; calculates the user's gaze point; determines the foreground fovea area according to the gaze point, renders the foreground fovea area at a first rendering resolution to generate a foreground fovea image, determines the foreground non-fovea area according to the gaze point, renders the foreground non-fovea area at a second rendering resolution to generate a foreground non-fovea image; and fuses the images to generate a final rendered image. According to the technical solution provided in the embodiment of the present application, by dividing the image to be displayed into different ranges, pre-rendering the background image to render and save it, dividing the foreground image into different areas, and rendering it at different rendering resolutions, the viewing needs of the user are met while reducing the rendering resource requirements.

Description

Technical Field

[0001] The present invention generally relates to the field of VR, and in particular to an image rendering method and system, VR equipment, a device and a readable storage medium. Background Art

[0002] With the continuous development of VR (Virtual Reality) display technology, VR display resolution continues to increase, and VR content needs to be presented at higher resolution and higher frame rate, which means that more GPU (Graphics Processing Unit) rendering resources are consumed, but the GPU graphics processing capability has not increased at the same rate, which limits the presentation and generation of VR ultra-high resolution and ultra-high frame rate content.

[0003] At present, all major manufacturers hope to solve these problems through foveated rendering technology. Foveated rendering is a new type of graphics computing technology that greatly reduces the complexity of GPU computing by reducing the resolution of the image around the foveated point. The existing foveated rendering technology performs 100% high-definition rendering of the area where the human eye is looking, and the areas outside this area are rendered by gradually reducing the resolution. This rendering method can reduce the rendering requirements of the GPU to a certain extent, but as the display resolution of near-eye display screens continues to increase to 16K or higher, the rendering of content in low-definition areas will still bring a heavy burden to the GPU. The current foveated rendering method cannot well solve the rendering requirements of VR content with higher resolution and higher frame rate. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an image rendering method and system, VR equipment, VR device and readable storage medium.

[0005] In a first aspect, an image rendering method is provided, comprising:

[0006] Acquire an image to be displayed, and perform background pre-rendering on the image to be displayed to generate a background panorama;

[0007] Acquire the user's head posture information, and determine the position the user is viewing based on the head posture information, and determine the background image corresponding to the position;

[0008] Calculate the user's gaze point;

[0009] Determine a foreground foveal area according to the gaze point, render the foreground foveal area according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal area according to the gaze point, render the foreground non-foveal area according to a second rendering resolution to generate a foreground non-foveal image;

[0010] The background image, the foreground foveated image and the foreground non-foveated image are fused to generate a final rendered image.

[0011] In a second aspect, an image rendering system is provided, comprising:

[0012] A background content pre-rendering module, used to obtain an image to be displayed, and perform background pre-rendering on the image to be displayed to generate a background panorama;

[0013] A viewing angle calculation module is used to obtain the user's head posture information, and calculate the viewing angle range of the user's current viewing according to the head posture information, and determine the user's viewing position according to the viewing angle range, and determine the background image corresponding to the position;

[0014] A gaze point calculation module, used to calculate the user's gaze point;

[0015] a foreground image generation module, configured to determine a foreground foveal area according to the gaze point, render the foreground foveal area according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal area according to the gaze point, render the foreground non-foveal area according to a second rendering resolution to generate a foreground non-foveal image;

[0016] The image fusion module is used to fuse the background image, the foreground foveated image and the foreground non-foveated image to generate a final rendered image.

[0017] According to a third aspect, a VR device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method is implemented when the processor executes the program.

[0018] In a fourth aspect, a VR display device is provided, wherein the VR display device is used to display an image processed using the above method.

[0019] According to a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program is used to implement the above method.

[0020] According to the technical solution provided in the embodiment of the present application, the image to be displayed is divided into a background image, a foreground foveated image and a foreground non-foveated image, and the background image is rendered and saved in a pre-rendering manner to avoid occupying too many resources in the subsequent real-time rendering process. At the same time, the foreground image is divided into different areas and rendered with different rendering resolutions. While meeting the viewing needs of the human eye, the requirements for GPU rendering capabilities when high-resolution and high-frame rate content are generated at the same time are reduced, thereby meeting the viewing needs of users while reducing the rendering resource requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0022] Figure 1 This is a flow chart of the image rendering method in this embodiment;

[0023] Figure 2 Schematic diagram of the structure of the image rendering system in this embodiment;

[0024] Figure 3 This is a schematic diagram of the image rendering process in this embodiment;

[0025] Figure 4 This is a flow chart of posture information calculation in this embodiment;

[0026] Figure 5 This is a flowchart of calculating the user's gaze point in this embodiment;

[0027] Figure 6 Schematic diagram of a device structure in this embodiment. DETAILED DESCRIPTION

[0028] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] Please refer to Figure 1 , this embodiment provides an image rendering method, comprising the steps of:

[0031] Acquire an image to be displayed, and perform background pre-rendering on the image to be displayed to generate a background panorama;

[0032] Obtaining the user's head posture information, and determining the user's viewing position based on the head posture information, and determining the background image corresponding to the position;

[0033] Calculate the user's gaze point;

[0034] Determine a foreground foveal area according to the gaze point, render the foreground foveal area according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal area according to the gaze point, render the foreground non-foveal area according to a second rendering resolution to generate a foreground non-foveal image;

[0035] The background image, the foreground foveated image and the foreground non-foveated image are fused to generate a final rendered image.

[0036] In this embodiment, the image to be displayed is divided into a background image, a foreground foveated image, and a foreground non-foveated image, and the background image is rendered and saved in a pre-rendering manner to avoid occupying too many resources in the subsequent real-time rendering process. At the same time, the foreground image is divided into different areas and rendered with different rendering resolutions. While meeting the viewing needs of the human eye, the requirements for GPU rendering capabilities when high-resolution and high-frame rate content are generated at the same time are reduced, thereby meeting the viewing needs of users while reducing the rendering resource requirements.

[0037] Furthermore, performing background pre-rendering on the image to be displayed to generate a background panorama includes:

[0038] measuring the depth of field of an image within an image to be displayed;

[0039] If the measured depth of field is less than the set depth of field threshold, the image content is judged to be foreground content; if the measured depth of field is greater than the set depth of field threshold, the image content is judged to be background content;

[0040] The background content is pre-rendered to generate a background panorama, and the background panorama is cached.

[0041] like Figure 4 As shown, in this embodiment, the background of the entire VR image to be displayed is first pre-rendered, and the pre-rendered background image is cached. When the user actually uses it later, the background image only needs to call out the cached image to determine the current viewing background, so as to avoid the subsequent real-time rendering, because the background needs to be rendered and takes up too many resources for calculation; for rendering the background content in the VR content image, it is first necessary to distinguish the foreground content and background content in the acquired VR content image, so as to pre-render the background content. In this embodiment, the depth of field of the image in the VR content image is measured and judged, for example, the depth of field threshold is set to d, and the depth of field greater than d is the foreground image, and the depth of field less than or equal to d is the background image. The background image is pre-rendered to generate a background panorama. The pre-rendering here refers to rendering the background image with a lower rendering resolution. In this embodiment, the foreground and background are determined by measuring the depth of field, wherein the depth of field can be judged sequentially for the pixels of the image to be displayed, or the image to be displayed is classified, and the depth of field of different categories of images is judged respectively.

[0042] Furthermore, obtaining the user's head posture information includes:

[0043] A weighted average calculation is performed based on the gravity acceleration of the previous frame and the gravity acceleration of the current frame to obtain a calculated value of gravity acceleration;

[0044] Calculating an attitude error based on an error between a calculated value of gravity acceleration and a reference value of gravity acceleration;

[0045] Calculate the predicted value of the current frame posture based on the previous frame posture information, angular velocity measurement value and Kalman filter algorithm;

[0046] Based on the posture error, the predicted value of the current frame posture is corrected to calculate the user head posture information of the current frame.

[0047] In this embodiment, it is necessary to determine the content that the user is watching based on the user's head posture information. Usually, the field of view of the human eye is about 90°. The scene image that the user is currently watching is determined based on the user's head posture and the corresponding field of view, and the scene image that the user is currently watching is rendered. In this embodiment, the user's field of view is set to 90°-120°, that is, one-quarter to one-third of the content image in the VR content map. The above embodiment provides specific calculation steps for the user's head posture information.

[0048] like Figure 5 As shown, further, calculating the user's gaze point includes: obtaining an eye image of the user;

[0049] Grayscale the eye image;

[0050] Calculate the threshold of the pupil area based on the grayscale processed eye image;

[0051] Perform binarization based on the threshold of the pupil area to calculate the pupil area;

[0052] Perform edge detection based on the pupil area after binary processing and calculate the pupil edge point;

[0053] Perform ellipse fitting based on pupil edge points to calculate the center of the pupil;

[0054] The coordinates of the user's gaze point are calculated based on the center of the pupil.

[0055] In the above embodiment, the image currently being watched by the user is determined by the user's head posture information, and the foreground image in the image needs to be further rendered, and the foreground image needs to be divided into the image range that the user is looking at and the image range that the user is not looking at. Different images use different rendering resolutions, so that the image that the user is looking at is rendered with the maximum resolution, ensuring that the user has a good user experience. At the same time, the rendering resolution of the image that the user is not looking at is relatively low, which can reduce the consumption of rendering resources. In this embodiment, in order to determine the image range that the user is looking at, it is necessary to calculate the user's gaze point, which is specifically achieved by taking a photo of the user's eye image, and determining the area of ​​the human eye pupil through steps such as grayscale processing and binarization analysis. In this embodiment, an infrared camera is specifically used to take a photo of the user's eye image, and infrared light can also be used to supplement infrared light, so that the captured user eye image is more accurate. Preferably, an 850nm infrared light is used in conjunction with the photo. The pupil center is further calculated by fitting the pupil into an ellipse and calculating the center of the ellipse structure, which is the center of the pupil. The coordinates of the user's gaze point are then calculated based on the center of the pupil. Specifically, the coordinates of the pupil center are input into the calibrated sight line calculation model to obtain the coordinates of the gaze point. The sight line calculation model is:

[0056] X=a+b·x+c·y+d·x 2 +e·y 2 ;

[0057] Y=f+g·x+h·y+1·x 2 +1·y 2 ;

[0058] Where (X, Y) is the fixation point coordinate, (x, y) is the pupil coordinate; the parameters ae and fj can be solved by the five-point calibration method, that is, the least square method is used according to five sets of known corresponding fixation point coordinates (X, Y) and pupil coordinates (x, y).

[0059] The determination of the above-mentioned gaze point coordinates can further determine the foreground fovea area and the foreground non-fovea area. Preferably, the range of about 10° around the gaze point coordinates can be set as the foreground fovea area, and the range of 10° up, down, left, and right of the gaze point coordinates can be set as the foreground fovea area, and the range of 10° above, below, left, and right of the gaze point coordinates can be set as the foreground fovea area, and the range outside 10° can be set as the foreground non-fovea area. This range can be expanded and can be set to 10°-20°.

[0060] Furthermore, the first rendering resolution is 100%, the second rendering resolution is 60%-40%, and the pre-rendering resolution does not exceed 50%.

[0061] In this embodiment, different rendering resolutions are used to render the foreground foveated image, the foreground non-foveated image and the background image respectively. The foreground foveated image on which the human eye focuses is rendered at a first rendering resolution, wherein the setting value is 100%. The foreground non-foveated image on which the human eye is less focused is rendered at a second rendering resolution, wherein the setting value is 60%-40%, preferably 50%. For the pre-rendering of the background image, the resolution does not exceed 50%, so that while meeting the viewing needs of the human eye, the requirements for rendering capabilities when generating high-resolution and high-frame rate content are reduced.

[0062] Furthermore, “performing image fusion on the background image, the foreground foveated image and the foreground non-foveated image” includes:

[0063] Determine matching areas of the foreground foveated image and the foreground non-foveated image within the background image;

[0064] Clear the pixels in the matching area of ​​the background image;

[0065] The foreground foveated image and the foreground non-foveated image are fitted to the matching area of ​​the background image.

[0066] In this embodiment, it is ultimately necessary to display a complete image to the user, so the rendered foreground foveated image, foreground non-foveated image and background image are fused. The method adopted in this embodiment is to directly remove the pixels in the corresponding area of ​​the background image, and then paste the foreground foveated image and the foreground non-foveated image at the corresponding positions of the background image, and then directly display them to the user.

[0067] like Figure 3 As shown, a schematic diagram of the image rendering process in this embodiment is given. The picture in the figure is displayed as a grayscale image, which is actually a color image. The VR display image is divided into three parts, a background image, a foreground foveated image and a foreground non-foveated image. The final image generation is divided into two steps, one is the background content generation process, and the other is the foreground content generation process, wherein the background image is pre-rendered to form a background panorama, and the background panorama is cached, and in the subsequent steps, the actual viewing field image is determined according to the position of the user's human eyes; at the same time, according to the position of the human eye gaze and the gaze point of the human eye, it is determined which of the foreground images currently gazed by the human eye are foveated images and which are non-foveated images. Figure 3 The bird image shown in the figure is a foveated image, and the airplane image is a non-foveated image. After rendering at different resolutions, the background image, the bird image, and the airplane image are fused to produce the final displayed image.

[0068] like Figure 2As shown, this embodiment also provides an image rendering system, including: a background content pre-rendering module 10, used to obtain an image to be displayed, and perform background pre-rendering on the image to be displayed to generate a background panorama;

[0069] The viewing angle calculation module 20 is used to obtain the head posture information of the user, calculate the viewing angle range of the user according to the head posture information, determine the viewing position of the user according to the viewing angle range, and determine the background image corresponding to the position;

[0070] A gaze point calculation module 30, used to calculate the gaze point of the user;

[0071] a foreground image generating module 40, configured to determine a foreground foveal region according to the gaze point, render the foreground foveal region according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal region according to the gaze point, render the foreground non-foveal region according to a second rendering resolution to generate a foreground non-foveal image;

[0072] The image fusion module 50 is used to fuse the background image, the foreground foveated image and the foreground non-foveated image to generate a final rendered image.

[0073] This embodiment also provides an image rendering system, which pre-renders and caches the background image to avoid occupying too many resources in the subsequent real-time rendering process. At the same time, the foreground image is divided into different areas and rendered using different rendering resolutions. While meeting the viewing needs of the human eye, it reduces the requirements for GPU rendering capabilities when high-resolution and high-frame rate content are generated simultaneously, meeting the viewing needs of users while reducing the rendering resource requirements. At the same time, the foreground image generation module in this embodiment also specifically includes a foreground foveated image generation unit and a foreground non-foveated image generation unit, which are respectively used to render the foreground foveated image and the foreground non-foveated image at different rendering resolutions; it can also include a cache module to cache the generated background panorama.

[0074] On the other hand, this embodiment also provides a VR device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the image rendering method as described above is implemented.

[0075] On the other hand, this embodiment also provides a VR display device for displaying the image processed by the above-mentioned image rendering method.

[0076] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing the device of the embodiment of the present application.

[0077] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 603 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0078] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 606 including a network interface card such as a LAN card, a modem, etc. The communication section 606 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0079] In particular, according to the embodiment of VR gaze point rendering disclosed in this application, the above reference Figure 1 The described process may be implemented as a computer software program. For example, embodiments of VR foveated rendering disclosed in the present application include a computer program product including a computer program tangibly embodied on a machine-readable medium, the computer program including a computer program for executing Figure 1 In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 606 and / or installed from the removable medium 611.

[0080] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device.

[0081] In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various VR foveated rendering embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, and the aforementioned module, program segment, or a portion of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified functions or operations, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The units or modules involved in the embodiments described in this application may be implemented by software or hardware. The units or modules described may also be set in a processor, for example, it may be described as: a processor includes a first acquisition module, a second acquisition module and a calculation module.

[0084] As another aspect, the present application further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the aforementioned device in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the image rendering method described in the present application, specifically executing:

[0085] Acquire an image to be displayed, and perform background pre-rendering on the image to be displayed to generate a background panorama;

[0086] Acquire the user's head posture information, and determine the position the user is viewing based on the head posture information, and determine the background image corresponding to the position;

[0087] Calculate the user's gaze point;

[0088] Determine a foreground foveal area according to the gaze point, render the foreground foveal area according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal area according to the gaze point, render the foreground non-foveal area according to a second rendering resolution to generate a foreground non-foveal image;

[0089] The background image, the foreground foveated image and the foreground non-foveated image are fused to generate a final rendered image.

[0090] In summary, the image rendering method, system, VR device, VR apparatus and storage medium provided in the embodiments of the present application divide the image to be displayed into a background image, a foreground foveated image and a foreground non-foveated image, and render and save the background image in a pre-rendering manner to avoid occupying too many resources in the subsequent real-time rendering process. At the same time, the foreground image is divided into different areas and rendered with different rendering resolutions. While meeting the viewing needs of the human eye, the requirements for GPU rendering capabilities when high-resolution and high frame rate content are generated at the same time are reduced, thereby meeting the viewing needs of users while reducing the rendering resource requirements.

[0091] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

Claims

1. An image rendering method, characterized in that: include: Acquire an image to be displayed, and perform background pre-rendering on the image to be displayed at a pre-rendering resolution to generate a background panorama; Acquire the user's head posture information, and determine the position the user is viewing based on the head posture information, and determine the background image corresponding to the position; Calculate the user's gaze point; Using different rendering resolutions to render the foreground foveated image, the foreground non-foveated image and the background image respectively; Determine a foreground foveal area according to the gaze point, render the foreground foveal area according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal area according to the gaze point, render the foreground non-foveal area according to a second rendering resolution to generate a foreground non-foveal image; Performing image fusion on the background image, the foreground foveated image and the foreground non-foveated image to generate a final rendered image; Wherein, the first rendering resolution is 100%, the second rendering resolution is 60%-40%, and the pre-rendering resolution does not exceed 50%; The image fusion of the background image, the foreground foveated image and the foreground non-foveated image comprises: Determine matching areas of the foreground foveated image and the foreground non-foveated image within the background image; Clear the pixels of the matching area in the background image; The foreground foveated image and the foreground non-foveated image are fitted to the matching area of ​​the background image.

2. The image rendering method according to claim 1, characterized in that: The generating a background panorama by performing background pre-rendering on the image to be displayed comprises: measuring the depth of field of an image within the image to be displayed; If the measured depth of field is less than the set depth of field threshold, the image content is judged to be foreground content; if the measured depth of field is greater than the set depth of field threshold, the image content is judged to be background content; The background content is pre-rendered to generate the background panorama, and the background panorama is cached.

3. The image rendering method according to claim 1, characterized in that: The obtaining of the user's head posture information comprises: Perform weighted average calculation based on the gravity acceleration of the previous frame and the gravity acceleration of the current frame to obtain the gravity acceleration calculation value; Calculating an attitude error based on an error between a calculated value of gravity acceleration and a reference value of gravity acceleration; Calculate the predicted value of the current frame posture based on the previous frame posture information, angular velocity measurement value and Kalman filter algorithm; Based on the posture error, error correction is performed on the predicted value of the current frame posture to calculate the user head posture information of the current frame.

4. The image rendering method according to claim 1, characterized in that: Calculating the user's gaze point includes: acquiring an eye image of the user; Performing grayscale processing on the eye image; Calculate the threshold of the pupil area based on the grayscale processed eye image; Perform binarization based on the threshold of the pupil area to calculate the pupil area; Perform edge detection based on the pupil area after binary processing and calculate the pupil edge point; Performing ellipse fitting based on the pupil edge points to calculate the center of the pupil; The coordinates of the user's gaze point are calculated based on the center of the pupil.

5. An image rendering system, characterized in that: include: A background content pre-rendering module is used to obtain an image to be displayed, and perform background pre-rendering on the image to be displayed at a pre-rendering resolution to generate a background panorama; A viewing angle calculation module is used to obtain the user's head posture information, calculate the viewing angle range of the user's current viewing according to the head posture information, determine the user's viewing position according to the viewing angle range, and determine the background image corresponding to the position; A gaze point calculation module, used to calculate the user's gaze point; The foreground image generation module is used to render the foreground foveated image, the foreground non-foveated image and the background image respectively using different rendering resolutions. The foreground image generation module is used to determine a foreground foveal area according to the gaze point, render the foreground foveal area according to a first rendering resolution to generate a foreground foveal image, determine a foreground non-foveal area according to the gaze point, and render the foreground non-foveal area according to a second rendering resolution to generate a foreground non-foveal image; An image fusion module, used for fusing the background image, the foreground foveated image and the foreground non-foveated image to generate a final rendered image; Wherein, the first rendering resolution is 100%, the second rendering resolution is 60%-40%, and the pre-rendering resolution does not exceed 50%; The image fusion module is specifically used for: Determine matching areas of the foreground foveated image and the foreground non-foveated image within the background image; Clear the pixels of the matching area in the background image; The foreground foveated image and the foreground non-foveated image are fitted to the matching area of ​​the background image.

6. A VR device, characterized in that: The VR device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any one of claims 1 to 4 when executing the program.

7. A VR display device, characterized in that: The VR display device is used to display images processed using the method described in any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is used to implement the method according to any one of claims 1 to 4.

Images