Methods, apparatuses, devices, media, and products for rendering images

Abstract

Embodiments of the present disclosure relate to a method, device, equipment, medium and product for rendering an image. The method comprises obtaining rendering instructions of a two-dimensional image on a panel viewed by a user in a virtual space in response to a change in the position of the user. The method further comprises determining a distance of the user to the panel. The method further comprises determining a magnification at which a resolution of the two-dimensional image is to be adjusted based at least on the distance. The method further comprises re-rendering the two-dimensional image for display on the panel based on the rendering instructions and the magnification. Through the method, the rendering effect of the two-dimensional image on the panel can be accurately adjusted according to the change in the position of the user, thereby optimizing the resolution and improving the user experience.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of computer graphics, and more particularly to methods, apparatuses, devices, media, and products for rendering images. Background Technology

[0002] With the rapid development of smart devices and the advancement of computer graphics technology, users' demands for visual experiences are constantly increasing. The widespread adoption of mobile terminals, Augmented Reality (AR), and Virtual Reality (VR) applications has made high-quality image rendering technology crucial for enhancing user experience. Modern users not only focus on device performance but also expect interfaces to have higher resolution, smoother animations, and more realistic visual presentations.

[0003] Meanwhile, mobile devices have limited hardware resources compared to desktops. Therefore, optimizing rendering efficiency and reducing computational overhead while maintaining high-definition image quality has become a crucial research direction in the industry. Furthermore, with the increasing demands for multitasking and cross-platform interaction, the fusion rendering of different types of interface content (such as 2D user interfaces (UI) and 3D scenes) is becoming a current technological trend. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, device, medium, and product for rendering images.

[0005] According to a first aspect of this disclosure, a method for rendering an image is provided. The method includes acquiring rendering instructions for a two-dimensional image on a panel viewed by the user in a virtual space in response to a change in the user's position. The method further includes determining the distance from the user to the panel. The method also includes determining a magnification by which the resolution of the two-dimensional image should be adjusted, at least based on the distance. The method re-renders the two-dimensional image based on the rendering instructions and the magnification for display on the panel.

[0006] According to a second aspect of this disclosure, an apparatus for rendering an image is provided. The apparatus includes a rendering instruction acquisition module configured to acquire rendering instructions for a two-dimensional image on a panel viewed by the user in a virtual space in response to a change in the user's position; a distance determination module configured to determine the distance from the user to the panel; a resolution adjustment module configured to determine a magnification by which the resolution of the two-dimensional image should be adjusted, at least based on the distance; and an image display module configured to re-render the two-dimensional image for display on the panel based on the rendering instructions and the magnification.

[0007] In a third aspect of this disclosure, an electronic device is provided, including at least one processor; and a storage device for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method according to the first aspect of this disclosure.

[0008] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0009] In a fifth aspect of this disclosure, a computer program product is provided. This computer program product includes a computer program that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0010] It should be understood that the content described in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0012] Figure 1 The illustration shows a schematic diagram of an example environment in which some embodiments of the present disclosure may be implemented;

[0013] Figure 2 The illustration shows a schematic diagram of an example method for rendering an image according to some embodiments of the present disclosure;

[0014] Figure 3 The illustration shows a flowchart of an example process for rendering an image according to some embodiments of the present disclosure;

[0015] Figure 4 The illustration shows a schematic diagram of an example of a high-definition strategy in a rendered image according to some embodiments of the present disclosure;

[0016] Figure 5 The illustration shows a schematic diagram of an example of high-resolution rendering in a rendered image according to some embodiments of the present disclosure;

[0017] Figure 6 The illustration shows a schematic diagram of an example of high / low resolution compositing in a rendered image according to some embodiments of the present disclosure;

[0018] Figure 7The illustration shows a flowchart of an example method for adjusting resolution magnification according to some embodiments of the present disclosure;

[0019] Figure 8 The illustration shows a schematic diagram of an example of the field of view of a vertical perspective according to some embodiments of the present disclosure;

[0020] Figure 9 The illustration shows a schematic diagram of an example of a high-definition area range in a vertical view according to some embodiments of the present disclosure;

[0021] Figure 10 The illustration shows a schematic block diagram of an apparatus for rendering an image according to some embodiments of the present disclosure;

[0022] Figure 11 A schematic block diagram of an example device suitable for implementing various embodiments of the present disclosure is illustrated. Detailed Implementation

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0026] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0030] As mentioned above, with the widespread adoption of smart devices and advancements in computer graphics technology, users' demand for high-quality visuals is growing. Improving the rendering quality of 2D UIs has become a significant technical challenge in order to provide clearer and smoother interface displays. However, achieving high-quality rendering while maintaining high performance remains a challenge on some electronic devices.

[0031] Typically, 2D UI rendering on mobile devices relies on the system's graphics rendering engine. However, in real-world applications, 2D UIs may need to be displayed or transformed in 3D space. Traditional rendering methods can result in inconsistent resolution or blurred images when blending 2D UIs with 3D scenes. Furthermore, the limitations of mobile device computing resources mean that achieving efficient, high-definition user interface rendering remains a significant challenge.

[0032] For example, in a multitasking operating system that supports spatial computing, users can run multiple 2D applications simultaneously and project their interfaces into 3D space, such as displaying multiple floating windows on a VR device or holographic projection interface. However, because 2D UIs require projection transformation, the resolution may decrease due to perspective scaling, resulting in blurry text and interface elements. Furthermore, when displaying menus, notifications, or interactive buttons in virtual space, these UI elements may remain unchanged depending on the user's viewing angle and distance. This can cause the currently rendered user interface to appear blurry and fonts difficult to read when viewed closely, thus affecting the user experience.

[0033] To address this, embodiments of this disclosure propose a method for rendering images. In this method, a computing device determines whether the user's position has changed. When the user's position changes, the computing device can obtain rendering instructions for a two-dimensional image on a panel viewed by the user in virtual space. The computing device then determines the distance from the user to the panel. Based on this distance, the computing device then determines the scaling factor for adjusting the resolution of the two-dimensional image. The computing device can then use the rendering instructions and the scaling factor to re-render the two-dimensional image for display on the panel. This method dynamically adjusts the resolution scaling factor at different distances and angles from the user to the panel, ensuring that the focused area when the user views the panel is always clear and free of jagged edges, thus improving the user experience.

[0034] The embodiments of this disclosure will now be described in further detail with reference to the accompanying drawings. Figure 1 The illustration shows an example environment in which the devices and / or methods of embodiments of the present disclosure may be implemented. In environment 100, computing device 102 can render images at different resolutions through preset operations.

[0035] Examples of computing device 102 include, but are not limited to, head-mounted devices (such as VR glasses), personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multiprocessor systems, consumer electronics, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, etc.

[0036] like Figure 1 As shown, users can view objects in a virtual 3D world. When the user moves, changes in the user's position 104 need to be detected. In one example, when the computing device 102 is a head-mounted device, it can acquire user position data and detect changes in the user's position in the 3D virtual space. In another example, the computing device 102 can also receive user position data transmitted from other devices and then detect changes in the user's position 104. When the user's position 104 changes, the computing device 102 acquires rendering instructions 110 for the 2D image on the panel viewed by the user in the virtual space. The 3D virtual space at this time can be a virtual environment constructed by, for example, VR glasses, AR devices, or other immersive display devices.

[0037] The computing device 102 then further determines the panel being viewed by the user. This panel is a window used to present information. For example, in three-dimensional space, a panel can be used to present a 2D application. The computing device 102 can determine the distance 106 between the user and the panel. Therefore, the obtained distance 106 between the user and the panel can be used to determine the adjustment of the resolution magnification and to prepare for image rendering. Additionally, the computing device 102 can further determine the target angle between the user's line of sight and the panel. For example, the user's line of sight can be a ray emanating from the midpoint between the user's two eyes in the three-dimensional virtual world and directed towards the display panel, and the angle between this ray and the panel can be used as the target angle.

[0038] Next, the computing device 102 can use the determined user-to-panel distance 106 to determine the resolution adjustment factor 108 of the two-dimensional image displayed on the panel. Additionally, the computing device 102 can also use the user-to-panel distance 106 and the target angle between the user's line of sight and the panel to determine the resolution adjustment factor 108 of the two-dimensional image displayed on the panel. In this process, the computing device 102 can determine the intersection point of the user's line of sight and the display panel. The user's line of sight at this point is a ray emanating from the midpoint between the user's two eyes in the three-dimensional virtual world and projected onto the display panel. This ray can be tracked using eye-tracking technology and acquired by the computing device. The intersection point of this ray and the display panel is the focal point of the user's eyes.

[0039] In one example, the computing device 102 can define a region centered on this point and encompassing a certain number of pixels within the panel. The size and shape of this region can be set to any suitable shape and size. Additionally, the size of this region is determined by its distance from the panel and the device's display parameters. Therefore, the computing device 102 can define a region centered on this intersection point and encompassing a predetermined number of pixels, using this region as a reference region. Within this region, local sampling is performed to acquire image details. For example, if the selected sampling region is 200 pixels, it means that a rectangular region of 200 pixels by 200 pixels near the ray point will be selected for image sampling.

[0040] Once the local sampling area is determined, the distance 106 from the user to the panel, or further combined with the target angle between the user's line of sight and the panel, can be used to map this area from virtual space onto the actual display screen. The actual size of the image area projected onto the screen may vary due to factors such as viewing distance, screen size, and resolution. To accurately adjust the image display effect, the ratio between the projected area and the total screen area needs to be calculated. For example, if the projected area is 300*200 pixels and the screen resolution is 1920*1080 pixels, the ratio can be calculated as 300*200 / 1920*1080. Finally, the computing device determines the resolution adjustment factor 108 based on the screen occupancy ratio, the image resolution, and the screen resolution. Any suitable method can be used to determine the resolution adjustment factor using the screen occupancy ratio, the image resolution, and the screen resolution; this disclosure does not limit this method.

[0041] Finally, the computing device 102 re-renders the 2D image based on the rendering instructions 110 and the calculated resolution adjustment factor 108 for image display 112 on the panel. In one example, the entire 2D image is re-rendered at the resolution adjustment factor 108. In another example, the resolution can be adjusted only for the region of interest (ROI) that the user is looking at, while maintaining the previous resolution for other areas, also known as the target region. For example, when the angle between the user's line of sight and the panel is 90 degrees, i.e., when the user is directly facing the panel, a radiating cone can be formed at a certain angle extending outward from the center of the line of sight. This cone interacts with the panel in a circular area. When the angle between the user's line of sight and the panel is not 90 degrees, the target region on the panel is an ellipse.

[0042] The computing device 102 renders the target image portion of the 2D image corresponding to the target area using a certain resolution, based on rendering instructions. This image portion is the part that the user's current gaze is primarily focused on, so a higher resolution is used to satisfy the user's viewing experience. Additionally, the computing device 102 can render the remaining image portion of the 2D graphic using the previous resolution and rendering instructions. Finally, the target image portion and the remaining portion are combined to generate the final 2D image. The computing device 102 then performs spatial effect processing and sampling on the 2D image, enhancing the image's visual effects by simulating real-world physical phenomena, while simultaneously optimizing rendering skills through sampling techniques. Spatial effect processing improves the image's immersion and realism, while sampling techniques ensure more accurate presentation of these effects even with limited resources.

[0043] This method allows the computing device to determine the user's distance from the panel and the viewing angle by acquiring the user's position changes in the virtual space. This allows the device to adjust the resolution of the two-dimensional image displayed on the panel and then re-render it to enhance the immersiveness and realism of the image. While meeting the user's visual needs, this method effectively improves the utilization of system resources and image rendering efficiency, thus enhancing the user experience.

[0044] The above combination Figure 1 The following is a schematic diagram illustrating an example environment in which some embodiments of this disclosure may be implemented, in conjunction with... Figure 2 A schematic diagram illustrating an example method for rendering an image according to some embodiments of the present disclosure. Figure 2 The method in can be derived from Figure 1 The computing device 102 or any suitable computing device in the system shall execute the test.

[0045] like Figure 2 In example method 200, at block 202, computing device 102, in response to a change in the user's position, acquires rendering instructions for a two-dimensional image on a panel viewed by the user in virtual space. A change in the user's position typically refers to a change in position, orientation, and viewpoint relative to virtual or real space in environments such as virtual reality (VR) and augmented reality (AR). In one example, the computing device receives data tracked and detected in real time by sensors (such as accelerometers, gyroscopes, etc.) to determine the user's motion. When the computing device detects a change in the user's virtual space position, the system needs to adjust the image content displayed to the user. Computing device 102 can acquire rendering instructions for this two-dimensional image, which describe how to render the panel image in the virtual environment as seen by the user.

[0046] At frame 204, the computing device 102 determines the distance between the user and the panel. To adjust image rendering, the computing device 102 can calculate this distance. This distance affects the clarity, resolution, and display area of ​​the content. For example, when the user is closer to the panel, more detail is observed in the pattern on the panel; in this case, the resolution of the pattern on the panel should be increased to avoid the user seeing jagged pixels. When the user is farther from the panel, the resolution can be appropriately reduced to save performance overhead. Simultaneously, the distance between the user and the panel also significantly affects the user's field of view. When the user's viewing angle remains constant, the area of ​​the panel in the user's field of view decreases as the user approaches the panel, and increases as the user moves away from the panel.

[0047] At frame 206, computing device 102 determines the scaling factor for adjusting the resolution of the two-dimensional image, at least based on distance. After obtaining the distance between the user and the panel, this distance can be further used to calculate the scaling factor or adjustment ratio for the resolution of the two-dimensional image at this time for the user.

[0048] In some embodiments, the computing device 102 can use distance-related algorithms or formulas to calculate the required adjustment factor for the resolution of the 2D image for the user. Additionally, the angle between the user's line of sight and the panel also affects the display of the panel in the user's line of sight. When the angle between the user's line of sight and the panel is not 90 degrees, the displayed area may lack detail due to the deviation in line of sight. The computing device needs to optimize the display area using techniques such as perspective correction, dynamic resolution adjustment, and local high-definition rendering to ensure the user obtains the best visual experience. Therefore, in some embodiments, the computing device further determines the target angle between the user's line of sight and the panel. Then, the computing device 102 uses the distance from the user to the panel and the target angle to calculate the required adjustment factor for the resolution of the 2D image for the user.

[0049] When calculating the resolution adjustment factor of a 2D image using the user's distance to the panel and the target angle, the intersection point between the user's line of sight and the panel can be determined before determining the resolution adjustment factor. This intersection point is usually the focal point of the user's eyes and can be obtained through eye-tracking technology. Once the intersection point is determined, a reference region can be defined around this point, such as an area centered on the intersection point and including a predetermined number of pixels, which contains the area of ​​user focus. The size of the reference region can be preset. Next, the computing device uses the distance and target angle to calculate the proportion of the reference region on the actual display screen. For example, the computing device can use the distance and target angle to calculate the projection area of ​​the reference region on the display screen, such as using a Model-View-Projection (MVP) matrix combined with the distance and target angle to calculate the projection area of ​​the reference region on the display screen. After the reference region is projected onto the display screen, the computing device calculates the occupancy ratio between the projection area and the total area of ​​the display screen, and then uses this occupancy ratio, the original resolution of the image, and the resolution of the display screen to determine the resolution adjustment factor.

[0050] Finally, at box 208, computing device 102 re-renders the 2D image based on the rendering instructions and scaling factor for display on the panel. Having obtained the rendering instructions and the scaling factor to which the resolution should be adjusted, computing device 102 can use this information to re-render the image displayed on the panel.

[0051] In some embodiments, to reduce resource consumption, the image resolution can be adjusted only for the region of interest that the user's eyes are focused on. For example, based on the user's eye tracking and gaze method, it is necessary to first determine the target region related to the user's gaze. After measuring and determining the gaze direction using eye tracking technology, the target region can be determined based on the gaze direction and a predetermined angle, which is the opening angle of the cone, for example, a predetermined angle of 30 degrees. The target region is determined using a cone geometry model. For example, if the user's viewpoint is the starting point of the cone, and the predetermined angle is the opening angle of the cone, expanding outward to form a cone, then the intersection of this cone and the panel, i.e., the plane where the cone intersects the panel, is the target region. Then, the image portion corresponding to the target region is rendered based on the previously calculated magnification. Because this image portion is the area of ​​most interest to the user, dynamic resolution adjustment is required to ensure clear display. In addition to the target region, there are other regions in the 2D image, which are generally areas that the user is not currently interested in, and these can be rendered according to the previous resolution and rendering instructions. Finally, the computing device combines these two parts, using precise compositing techniques to ensure a smooth transition, and the details of each region meet the user's detail requirements. After the above steps, the computing device will generate an updated two-dimensional image, which includes the target area whose resolution is dynamically adjusted according to the user's line of sight, as well as the remaining area at normal resolution.

[0052] This method uses precise eye tracking and gaze analysis to re-render 2D images based on the user's angle and gaze direction, providing the user with a clear visual experience. This better meets the user's requirements for high-definition and immersive 3D virtual space and improves the user experience.

[0053] The above combination Figure 2 Schematic diagrams of example methods for rendering images according to some embodiments of the present disclosure are described below, in conjunction with... Figure 3 A flowchart describing an example process for rendering an image according to some embodiments of the present disclosure. Figure 3 Example process 300 in the example can be derived from Figure 1 The computing device 102 or any suitable computing device in the system can be used for processing.

[0054] At box 304, computing device 102 uses eye-tracking (ET) data to analyze the user's gaze trajectory and uses collision detection technology to determine whether the user's gaze intersects with the interactive panel (virtual interface), thereby identifying the user's attention area and performing corresponding rendering and interaction optimization for that area. At box 306, the global position of the interactive panel in 3D space is represented using world coordinates to determine the panel's absolute position and orientation in the virtual environment. The user's eye-tracking (ET) coordinates are calculated based on the panel's local coordinate system, not the global coordinate system. By transforming the position of the eye-tracking point in world coordinates and mapping it to the panel's local coordinate system, the user's gaze point on the panel can be accurately located. Then, based on eye-tracking data and collider checks in 3D space, the intersection area between the user's gaze and the interactive panel can be determined, and the panel's position in 3D coordinates can be calculated. Simultaneously, transforming the eye-tracking gaze data to the panel's local coordinate system provides accurate reference data for subsequent high-definition rendering and interaction optimization. Finally, in the high-definition rendering of the UI, a high-definition strategy 308 needs to be applied, followed by high-definition rendering 310 and high / low-definition compositing 312. This information will be combined... Figure 4 , Figure 5 , Figure 6 and Figure 7 Describe it.

[0055] The following is combined Figure 4 Describe the high-definition strategy, in which Figure 4The illustration shows an example of a high-definition strategy in a rendered image according to some embodiments of the present disclosure. In example 400, the high-definition strategy at box 402 is a comprehensive strategy system including a trigger strategy 404, a scaling factor calculation 406, and a high-definition region calculation 408. At the trigger strategy in box 404, the trigger strategy is delayed triggering. At the delayed triggering in box 410, when the user interacts with the application and a change in the interactive panel's screen is detected, the version changes, thus triggering an update. At box 406, the scaling factor between the two screens is calculated based on the resolution of the partially sampled area of ​​the eye-tracking system and the resolution projected onto the display screen (such as the screen of VR glasses). Partial sampling of the eye-tracking area refers to using a higher resolution in the area where the gaze is concentrated, while using a lower resolution in the surrounding areas to save computing resources. For example, at box 412, at a distance of 1.9 meters from the window, the resolution is 1600*900, and the resolution projected onto the display screen is calculated. At a distance of 1 meter from the window, due to the closer viewing distance, a higher resolution may be required. Based on the resolution of the eye-tracking area and the resolution projected onto the display screen, the scaling factor calculated at box 406 can further determine the visualization factor at box 416 that adjusts the image display ratio according to the scaling factor, as well as the size of the rendering area and the visualization area for detail display.

[0056] Finally, the high-definition area is calculated at box 408. First, the position of the display panel in actual space is determined, i.e., the distance and relative angle between the panel and the user's center of vision. This position is crucial for determining the range of high-definition rendering; incorrect positioning may affect the alignment of eye tracking with the high-definition area. Combining the pose of the eye-tracking (ET) system and the radiation range of the user's field of view, the area requiring high-definition detail can be calculated. In box 414, the panel area is calculated based on the intersection of the ray of the ET radiation angle and the panel. Specifically, the field of view angle range formed by the gaze direction obtained from the eye-tracking (ET) system and the ray extending from the eye along the ET radiation angle intersect the panel at a point. This point represents the precise location of the user's gaze point, i.e., the focal point. The system generates a circle or ellipse centered on this ray. When the ray extending from the user is perpendicular to the panel, the resulting panel area is a circle centered on that ray. When the ray extending from the user's center of vision is not perpendicular to the panel, the resulting panel area is an ellipse centered on that ray.

[0057] Figure 5 The diagram shows Figure 3The example below illustrates high-definition rendering. In example 500, at box 502, high-definition rendering comprises two parts: region redrawing 504 and view rendering 506. Region redrawing 504 involves redrawing the high-definition display area on the panel after determining the high-definition display area to ensure that the area the user is looking at always maintains the highest quality display. View rendering calculates the view that needs to be rendered in high definition, i.e., the high-definition view, based on the high-definition area. The user's gaze point is obtained through an eye-tracking system, which calculates the size and position of the gaze area and dynamically renders high-definition details based on that area. In this way, only the gaze area uses a high-resolution image, while non-gaze areas use a lower resolution, thereby optimizing rendering performance.

[0058] Figure 6 The diagram shows Figure 3 An example of high / low resolution compositing. In example 600, the high / low resolution compositing at box 602 includes region redrawing (3D overlay enhancement) 604 and view rendering compositing 606. 3D overlay is a technique that enhances the user's visual experience by overlaying 3D images onto a real-world scene. The combination of high / low resolution compositing and 3D overlay enhancement means that enhanced 3D elements can be displayed at high resolution in the area the user is looking at (such as the corresponding pattern that the user needs to interact with), while the remaining unimportant background parts are displayed at low resolution. View rendering compositing 606 refers to the process of compositing the view in the high-definition area with the image in the low-definition area onto the screen. The final displayed image ensures that the view seen by the user presents clear details in the high-definition area, while other parts can maintain a smooth viewing experience, reducing performance consumption to some extent.

[0059] Figure 7 The illustration shows a flowchart of an example method for adjusting resolution magnification according to some embodiments of the present disclosure. This method can be... Figure 1 The calculation is performed by computing device 102 or any suitable device. In example 700, at 702, the computing device can determine the intersection point of the line of sight and the panel. For example, first determine the line of sight emanating from the viewpoint and directed towards the panel, which has an intersection point with the panel. At 704, the computing device determines a reference area centered on the intersection point. For example, a 200-pixel * 200-pixel square reference area is determined centered on the intersection point. Then, at 706, the reference area is projected onto the projection area of ​​the display screen. For example, the reference area is projected onto the projection area of ​​the display screen through a matrix transformation. Then, at 708, the ratio of the projection area to the display screen is calculated. The occupancy ratio of the reference area on the display screen can be determined based on the relative ratio of the number of pixels in the projection area to the number of pixels on the display screen. Then, at 710, the required adjustment factor is calculated based on this ratio, the resolution of the image in the reference area, and the resolution of the display screen.

[0060] Figure 8 The illustration shows an example diagram of an example method for determining the field of view of a vertical perspective according to some embodiments of the present disclosure. In example 800, when the angle between the user's line of sight and the panel is 90 degrees, and the cone angle of the cone forming the field of view is 15 degrees, the target area focused by the user is a circle centered on a ray emanating from the user's line of sight perpendicular to the panel, i.e., the area where the cone coincides with the panel.

[0061] Figure 9 The illustration shows an example method for defining a high-definition area range from a vertical viewing angle according to some embodiments of the present disclosure. In Example 900, when a user views a high-definition pattern on a panel, the area within the square is the area of ​​primary user attention and is the target high-definition area. Other areas are areas of less user attention and can be rendered using the original resolution. Finally, the high-definition area is combined with the original low-resolution rendered portion to form a complete two-dimensional image.

[0062] Figure 10 A schematic block diagram of an apparatus for rendering images according to some embodiments of the present disclosure is shown. Figure 10 As shown, the device 1000 includes a rendering instruction acquisition module 1002, configured to acquire rendering instructions for a two-dimensional image on a panel viewed by the user in a virtual space in response to a change in the user's position; a distance determination module 1004, configured to determine the distance from the user to the panel; a resolution adjustment module 1006, configured to determine the scaling factor for adjusting the resolution of the two-dimensional image, at least based on the distance; and an image display module 1008, configured to re-render the two-dimensional image for display on the panel based on the rendering instructions and the scaling factor.

[0063] In some embodiments, the resolution adjustment module 1006 includes: a target angle determination module configured to determine a target angle between the user's line of sight and the panel; and a first magnification determination module configured to determine the magnification to be adjusted for the resolution of the two-dimensional image based on the distance and the target angle.

[0064] In some embodiments, the first magnification determination module includes: an intersection point determination module configured to determine the intersection point of the line of sight and the panel; a reference area determination module configured to determine a reference area centered on the intersection point and including a predetermined number of pixels; an occupancy ratio determination module configured to determine the occupancy ratio of the reference area on the display screen based on distance and target angle; and a second magnification determination module configured to determine the magnification based on the occupancy ratio, the resolution of the image, and the resolution of the screen.

[0065] In some embodiments, the occupancy ratio determination module includes: a projection area determination module configured to determine a projection area of ​​a reference area projected onto a display screen based on a distance and a target angle; and an occupancy ratio determination module configured to determine the ratio of the projection area to the display screen as the occupancy ratio.

[0066] In some embodiments, the image display module 1008 includes: a corresponding angle target area determination module, configured to determine a target area in the panel corresponding to a predetermined angle based on a predetermined angle related to eye tracking; and a two-dimensional graphics re-rendering module, configured to re-render two-dimensional graphics based on rendering instructions, magnification, and the target area.

[0067] In some embodiments, the corresponding angle target area determination module includes: a gaze direction determination module configured to determine the gaze direction of the user's gaze; and a target area determination module configured to determine a target area on the panel based on the gaze direction and a predetermined angle.

[0068] In some embodiments, the target area determination module includes: a cone perspective determination module configured to determine a cone extending outward from the line of sight and the viewpoint by a predetermined angle; an interaction area determination module configured to determine the interaction area between the cone and the panel based on the target angle; and a target area selection module configured to determine the interaction area as the target area.

[0069] In some embodiments, the two-dimensional image re-rendering module is configured as follows: a target image portion rendering module, configured to render a target image portion in a two-dimensional graphic corresponding to the target region based on a resolution and rendering instructions having the magnification; a remaining image rendering module, configured to render a remaining image portion in a two-dimensional graphic based on the previous resolution and rendering instructions; and an image combination module, configured to generate a two-dimensional graphic by combining the target image portion and the remaining image portion.

[0070] In some embodiments, the apparatus 1000 further includes: a two-dimensional image rendering module configured to render a two-dimensional image based on rendering instructions; and an instruction caching module configured to store rendering instructions in a cache.

[0071] In some embodiments, the apparatus 1000 further includes a two-dimensional image processing module configured to perform spatial effect processing and sampling on the two-dimensional image.

[0072] Figure 11 A schematic block diagram of an example device 1100 that can be used to implement embodiments of the present disclosure is shown. Figure 1The computing device 102 can be implemented using device 1100. As shown, device 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 1102 or loaded from storage unit 1108 into random access memory (RAM) 1103. The RAM 1103 can also store various programs and data required for the operation of device 1100. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0073] Multiple components in device 1100 are connected to I / O interface 1105, including: input unit 1106, such as keyboard, mouse, etc.; output unit 1107, such as various types of monitors, speakers, etc.; storage unit 1108, such as disk, optical disk, etc.; and communication unit 1109, such as network card, modem, wireless transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0074] The various processes and procedures described above, such as method 200 and process 300, can be executed by processing unit 1101. For example, in some embodiments, method 200 and process 300 can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by CPU 1101, one or more actions of the example methods 200 and process 300 described above can be performed.

[0075] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0076] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0077] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0078] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0079] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0083] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for rendering an image, comprising: In response to a change in the user's position, obtain rendering instructions for the two-dimensional image on the panel viewed by the user in the virtual space; Determine the distance from the user to the panel; Based at least on the distance, determine the magnification by which the resolution of the two-dimensional image should be adjusted; as well as Based on the rendering instructions and the magnification, the two-dimensional image is re-rendered for display on the panel.

2. The method according to claim 1, wherein determining the magnification by which the resolution of the two-dimensional image is to be adjusted comprises: Determine the target angle between the user's line of sight and the panel; Based on the distance and the target angle, determine the magnification by which the resolution of the two-dimensional image should be adjusted.

3. The method according to claim 2, wherein determining the magnification by which the resolution of the two-dimensional image should be adjusted based on the distance and the target angle includes: Determine the intersection point between the line of sight and the panel; Determine a reference region centered on the intersection point and including a predetermined number of pixels; Based on the distance and the target angle, determine the occupancy ratio of the reference area on the display screen; as well as The magnification is determined based on the occupancy ratio, the resolution of the image, and the resolution of the display screen.

4. The method of claim 3, wherein determining the occupancy ratio of the reference area on the display screen comprises: Based on the distance and the target angle, the projection area of ​​the reference area onto the display screen is determined; as well as The ratio of the projection area to the display screen is defined as the occupancy ratio.

5. The method of claim 1, wherein re-rendering the two-dimensional image for display on the panel comprises: Based on a predetermined angle related to eye tracking, a target area in the panel corresponding to the predetermined angle is determined; as well as Based on the rendering instructions, the magnification, and the target region, the two-dimensional image is re-rendered.

6. The method of claim 5, wherein determining the target area in the panel corresponding to the predetermined angle comprises: Determine the direction of the user's gaze; as well as The target area on the panel is determined based on the line of sight and the predetermined angle.

7. The method of claim 6, wherein determining the target area on the panel based on the viewing direction and the predetermined angle comprises: Determine a cone that extends outward from the predetermined angle with the line of sight as its center and the viewpoint as its starting point; Based on the target angle, the interaction area between the cone and the panel is determined; as well as The interactive area is defined as the target area.

8. The method of claim 5, wherein re-rendering the two-dimensional image based on the rendering instructions, the magnification, and the target region comprises: Based on the resolution with the magnification and the rendering instructions, render the target image portion in the two-dimensional image corresponding to the target region; Based on the previous resolution and the rendering instructions, render the remaining image portion of the two-dimensional image; as well as The two-dimensional image is generated by combining the target image portion and the remaining image portion.

9. The method according to claim 1, further comprising: Render the two-dimensional image based on the rendering instructions; as well as The rendering instructions are stored in the cache.

10. The method of claim 1, wherein the panel is for a two-dimensional application.

11. The method of claim 1, further comprising: The two-dimensional image is processed for spatial effects and sampled.

12. An apparatus for rendering an image, comprising: The rendering instruction acquisition module is configured to acquire rendering instructions for the two-dimensional image on the panel viewed by the user in the virtual space in response to a change in the user's position. A distance determination module is configured to determine the distance from the user to the panel; A resolution adjustment module is configured to determine, at least based on the distance, the magnification by which the resolution of the two-dimensional image should be adjusted; as well as The image display module is configured to re-render the two-dimensional image for display on the panel based on the rendering instructions and the magnification.

13. An electronic device, comprising: At least one processor; as well as A storage device for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method according to any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1-11 when executed by a processor.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images