Method, apparatus, device, storage medium and program product for image rendering

Abstract

Embodiments of this disclosure relate to methods, apparatus, devices, storage media, and program products for image rendering. The proposed method includes: dividing a screen area into multiple grids; determining first evaluation information for the multiple grids based on current motion information of the multiple grids, the first evaluation information indicating whether pixels corresponding to the multiple grids will move outside the screen area; determining second evaluation information for the multiple grids based on historical motion information of the multiple grids, the second evaluation information indicating the smoothness of the motion of the multiple grids; determining a rendering strategy based at least on the first and second evaluation information, the rendering strategy indicating the regional attributes of an additional region to be rendered, the additional region being outside the screen area; generating a rendering frame based on the rendering strategy, the rendering frame covering the screen area and the additional region; and performing interpolation processing on the rendering frame to generate an interpolated frame. In this manner, embodiments of this disclosure can dynamically adjust the additional region to be rendered, improving the quality of image rendering.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein relate generally to the field of computers, and more particularly to methods, apparatus, devices, and computer-readable storage media for image rendering. Background Technology

[0002] Frame extrapolation is a technique that predicts and generates new image frames based on currently rendered image frames and information such as motion vectors. For example, in game scenes, extrapolation relies on motion vectors and depth information from historical game frames, using methods such as reprojection and motion prediction to generate game frames for future moments. Improving the quality of extrapolated frame generation is a key concern. Summary of the Invention

[0003] In a first aspect of this disclosure, an image rendering method is provided. The method includes: dividing a screen area into multiple grids; determining first evaluation information for the multiple grids based on current motion information of the multiple grids, the first evaluation information indicating whether pixels corresponding to the multiple grids will move outside the screen area; determining second evaluation information for the multiple grids based on historical motion information of the multiple grids, the second evaluation information indicating the smoothness of the motion of the multiple grids; determining a rendering strategy based at least on the first and second evaluation information, the rendering strategy indicating the regional attributes of an additional region to be rendered, the additional region being outside the screen area; generating a rendering frame based on the rendering strategy, the rendering frame covering the screen area and the additional region; and performing interpolation processing on the rendering frame to generate an interpolated frame.

[0004] In a second aspect of this disclosure, an apparatus for image rendering is provided. The apparatus includes: a partitioning module configured to partition a screen area into multiple grids; a first determining module configured to determine first evaluation information for the multiple grids based on current motion information of the multiple grids, the first evaluation information indicating whether pixels corresponding to the multiple grids will move outside the screen area; a second determining module configured to determine second evaluation information for the multiple grids based on historical motion information of the multiple grids, the second evaluation information indicating the smoothness of motion of the multiple grids; a third determining module configured to determine a rendering strategy based at least on the first and second evaluation information, the rendering strategy indicating regional attributes of an additional region to be rendered, the additional region being outside the screen area; a rendering module configured to generate a rendering frame based on the rendering strategy, the rendering frame covering the screen area and the additional region; and a generating module configured to perform interpolation frame processing on the rendering frame to generate an interpolated frame.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] Based on this approach, embodiments of this disclosure can analyze whether multiple grids are about to move off-screen based on their current motion information within the screen area, prioritizing the rendering of grids that are about to move off-screen. Furthermore, embodiments of this disclosure can analyze the stability of the motion of multiple grids based on their historical motion information, prioritizing the rendering of grids with less stable motion. Thus, embodiments of this disclosure can dynamically allocate image rendering computational resources through a rendering strategy determined at least based on first and second evaluation information, allowing computational resources to be directed towards additional areas that require rendering, thereby improving the utilization of computational resources. In addition, embodiments of this disclosure can generate rendering frames based on this rendering strategy, thereby rendering more appropriate additional areas during the rendering frame generation process. Furthermore, embodiments of this disclosure can ensure that the interpolated frames generated based on the rendering frames not only reference the portion of the rendering frame covering the screen area but also appropriate additional areas, thereby improving the quality of the interpolated frames and the overall image rendering quality. Therefore, embodiments of this disclosure can dynamically adjust the additional areas to be rendered, improving the utilization of computational resources and enhancing the quality of image rendering.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram is shown of an example environment in which embodiments of the present disclosure may be implemented;

[0012] Figure 2 A schematic diagram illustrating an example process for processing some regions according to some embodiments of the present disclosure is shown;

[0013] Figure 3 A flowchart illustrating an example process for image rendering according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic diagram illustrating example processes for some region processing according to other embodiments of the present disclosure is shown;

[0015] Figure 5 A schematic structural block diagram of an example apparatus for image rendering according to some embodiments of the present disclosure is shown;

[0016] Figure 6 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0021] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0022] As mentioned above, extrapolation is a technique that predicts and generates new image frames based on currently rendered image frames and information such as motion vectors. For example, in game scenes, extrapolation relies on motion vectors and depth information from historical game frames, using methods such as reprojection and motion prediction to generate game frames for future moments. Compared to interpolation, extrapolation typically does not reference actual image frames from future moments, thus achieving higher rendering speeds. Therefore, improving the frame generation quality of extrapolated frames is also desirable.

[0023] This disclosure proposes an image rendering scheme. According to this scheme, a screen area can be divided into multiple grids. Furthermore, based on the current motion information of the multiple grids, first evaluation information of the multiple grids can be determined, indicating whether the pixels corresponding to the multiple grids will move outside the screen area. Further, based on the historical motion information of the multiple grids, second evaluation information of the multiple grids can be determined, indicating the smoothness of the motion of the multiple grids. Further, based at least on the first and second evaluation information, a rendering strategy can be determined, indicating the regional attributes of an additional region to be rendered, which is outside the screen area. Additionally, based on the rendering strategy, a rendering frame can be generated, covering both the screen area and the additional region. Additionally, extrapolation frame processing can be performed on the rendering frame to generate an extrapolated frame.

[0024] Based on this approach, embodiments of this disclosure can analyze whether multiple grids are about to move off-screen based on their current motion information within the screen area, prioritizing the rendering of grids that are about to move off-screen. Furthermore, embodiments of this disclosure can analyze the smoothness of the motion of multiple grids based on their historical motion information, prioritizing the rendering of grids with less stable motion. Thus, embodiments of this disclosure can dynamically allocate image rendering computational resources through a rendering strategy determined at least based on first and second evaluation information, allowing computational resources to be directed towards additional areas that require more rendering, thereby improving the utilization of computational resources. In addition, embodiments of this disclosure can generate rendering frames based on this rendering strategy, thereby rendering more appropriate additional areas during the rendering frame generation process. Furthermore, embodiments of this disclosure can ensure that the interpolated frames generated based on the rendering frames not only reference the portion of the rendering frame covering the screen area but also reference appropriate additional areas, thereby improving the quality of the interpolated frames and the overall image rendering quality.

[0025] Therefore, the embodiments of this disclosure can dynamically adjust the additional area to be rendered, improve the utilization of computing resources, and improve the quality of image rendering.

[0026] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0027] Example environment:

[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include image processing device 110.

[0029] In this example environment 100, the image processing device 110 may first render to generate image frame 120, and then perform at least extrapolation frame processing on image frame 120 to generate a new image frame 130. As an example, image frame 120 and image frame 130 may represent different scenes in a virtual scene, which may switch automatically or be switched based on user operation. As an example, such a virtual scene may include a game scene, a virtual reality scene, a virtual augmented reality scene, etc., and the scenes in the virtual scene can be implemented by presenting the rendered image frames in a screen area. It should be understood that the image content shown in image frame 120 and image frame 130 is for illustrative purposes only and is not intended to be limiting.

[0030] In some examples, the image rendering process described above can be implemented at the image processing device 110. The image processing device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the image processing device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0031] In some examples, the image rendering process described above can be implemented at the server. The image processing device 110 can acquire the image frame rendered by the server through a communication connection and present the rendered image frame on the interface of the image processing device 110. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. The image processing device 110 may include, for example, a computing system / server, such as a mainframe, edge computing node, computing device in a cloud environment, etc.

[0032] A communication connection can be established between the server and the image processing device 110. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, the server and the image processing device 110 can achieve signaling interaction through the communication connection between them.

[0033] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0034] The following description will continue with reference to the accompanying drawings to further describe some exemplary embodiments of this disclosure.

[0035] Traditionally, with the development of real-time rendering technology, some virtual scenes have begun to use extrapolation frame technology to achieve scene rendering. Extrapolation frame technology can predict and generate new image frames based on information from historical frames, thereby improving the screen display refresh rate and reducing the rendering burden.

[0036] Typically, extrapolation techniques do not reference information from historical frames, but rather rely on information such as motion vectors, depth information, and scene rendering results from the previous frame or several historical frames. For example, extrapolation techniques can predict the content of a scene at a future moment or over a future period of time through reprojection or motion prediction. Taking the current frame as frame N as an example, extrapolation techniques can predict the image frame of frame N+0.5 based on the complete information of frame N and the preceding image frames (for ease of description, in the implementation of extrapolation, the extrapolated frame of frame N+0.5 is referred to as the next frame of the current frame in the embodiments disclosed below).

[0037] Due to the lack of a reference to a real frame at a "future moment" (e.g., information from the N+1th frame), image frames generated by extrapolation techniques exhibit significant errors in areas such as edges. For example, in some scenes, some objects in the newly generated image frame will slide out of or about to enter the viewport. In such scenarios, extrapolation methods such as optical flow-based reprojection or texture inference cannot obtain reliable real pixels corresponding to these objects as a basis for sampling, thus increasing the probability of problems such as edge transformation distortion, structural warping, object stretching, jitter, or ghosting. Furthermore, in situations such as rapid camera translation or rotation, or foreground objects moving rapidly close to the screen edge, motion vectors in edge regions are more prone to jumps, noise, and occlusion changes, thereby reducing the estimation stability of extrapolation techniques.

[0038] Based on this, some embodiments of this disclosure can additionally draw a portion of the area outside the screen area when generating the current rendering frame. Therefore, when performing extrapolation frame processing on the rendering frame to generate the next frame, the embodiments of this disclosure can refer to this additional area, thereby improving the quality of image rendering.

[0039] As an example, the process of drawing an additional region outside the screen area can be called overscanning. This can provide more pixels as reference for the process of generating the next frame by performing interpolation on the current rendered frame. Again, taking the current frame as frame N, some examples can use overscanning to supplement the interpolated frame with richer edge data (e.g., rendering content drawn outside the screen area). Again, taking the current frame as frame N, overscanning can provide more pixel references for predicting frame N+0.5, thereby reducing edge distortion during interpolation and improving the accuracy of the interpolation.

[0040] The following embodiments of this disclosure are exemplified by using the image processing device 110 as a terminal device. Some embodiments of image rendering implemented through a server (e.g., applied to cloud gaming, cloud virtual reality scenarios, etc.) can be referred to the following description of this disclosure, and will not be repeated here.

[0041] Figure 2 A schematic diagram of an example process 200 for processing some regions according to some embodiments of the present disclosure is shown. Process 200 can be implemented at an image processing device 110. Reference is made below. Figure 1 Describe process 1, 200.

[0042] like Figure 2 As shown, some embodiments of this disclosure may add an additional region 220 outside the screen region 210 to obtain a rendering region 230, where region 230 may include the screen region 210 and the additional region 220. Therefore, compared to the technical solution of the image processing device 110 generating only a rendering frame 211 for the screen region 210, some embodiments of this disclosure can additionally draw the additional region 220 outside the screen region 210 to obtain a rendering frame 231 with a larger drawing area. As an example, rendering frame 231 may include an image frame 120 overlaid in the screen region 210 and an extended image overlaid in the additional region 220.

[0043] Furthermore, the image processing device 110 can perform interpolation frame processing on the second rendering frame 231, and can use the portion of the second rendering frame 231 corresponding to the additional region 220 as a reference to generate a higher quality interpolation frame.

[0044] In some cases, such an additional region 220 can be a fixed extension size. For example, such an additional region 220 can be an area that extends outward by a specified size from the screen region 210. However, since the additional region 220 is fixedly and evenly distributed in one or more directions to be extended, the interpolation frame stability of the edge region is weak in scenarios such as high-speed horizontal translation and character sliding along the edge.

[0045] To further improve the quality of interpolated frames, in some embodiments, the image processing device 110 may determine a more reasonable additional area based on analysis of the content in screen region 210. Reference will be made below. Figure 3 This document describes some example processes for image rendering according to embodiments of the present disclosure. Figure 3 A flowchart of an example process 300 for image rendering according to some embodiments of the present disclosure is shown. Process 300 can be implemented at image processing device 110. Reference is made below. Figure 1 Describe process two, 300.

[0046] like Figure 3 As shown, in step 310, the image processing device 110 divides the screen area into multiple grids.

[0047] As an example, screen area 210 can be an area supported by the display unit of image processing device 110 (e.g., screen, monitor, etc.). Furthermore, image processing device 110 can divide screen area 210 into multiple grids using any appropriate image partitioning method.

[0048] As an example, each of the multiple grids may include one or more pixels in screen area 210. The different grids are independent of each other and may include different pixels. In some examples, such multiple grids may be multiple fixed-size grids, for example, each grid may be 8×8 pixels, 16×16 pixels, 8×16 pixels, etc. In other examples, such multiple grids may also be multiple grids of different sizes; for example, the size of the grid in the central portion of screen area 210 may be larger than the size of the grid in the edge portion of screen area 210, to support the image processing device 110 in more fine-grained analysis of the edge regions in screen area 210.

[0049] As an example, Figure 4 A schematic diagram of an example process 3400 for processing some regions according to other embodiments of the present disclosure is shown. Process 3400 can be implemented at image processing device 110. Figure 4 As shown, the image processing device 110 can divide the screen area 210 into multiple grids, such as grid 1 311, grid 2 312, grid 3 313, grid 4 314, grid 5 315, etc. Based on this, the image processing device 110 can analyze such multiple grids, thereby determining a more reasonable additional area 220 through localized analysis of the screen area 210.

[0050] In step 320, the image processing device 110 determines first evaluation information for the multiple grids based on the current motion information of the multiple grids. The first evaluation information indicates whether the pixels corresponding to the multiple grids will move outside the screen area.

[0051] As an example, during the generation of the current rendering frame, the image processing device 110 can read data such as motion vector buffers, depth buffers, and one or more historical frames. Furthermore, the image processing device 110 can determine the current motion information of multiple meshes based on such information as motion vector buffers, depth buffers, and one or more historical frames.

[0052] In some examples, the image processing device 110 can analyze each of the multiple grids to determine the current motion information of each grid. Specifically, the image processing device 110 can determine the motion vectors of multiple pixels in a first grid; and determine the current motion information of the first grid based on the motion vectors of the multiple pixels.

[0053] Taking grid 313 as an example, the image processing device 110 can statistically analyze information such as the current motion vectors of multiple pixels in grid 313 and the rate of change of the current motion vectors relative to the previous frame or multiple historical frames. Furthermore, the image processing device 110 can determine the current motion information of grid 313 based on the motion vectors of multiple pixels.

[0054] As an example, current motion information can indicate the average motion speed of multiple pixels, the average motion direction of multiple pixels, and so on. For instance, image processing device 110 can statistically analyze the average value, amplitude, direction, and rate of change of the motion vectors of multiple pixels in grid 313 compared to previous frames or more historical frames to determine information such as the motion trend of multiple pixels in grid 313.

[0055] Alternatively or additionally, such current motion information may also include other statistical data determined based on the current motion vectors of multiple pixels. For example, the image processing device 110 may use the motion direction with the largest proportion among the multiple motion directions of multiple pixels as the main motion direction of grid 313, and determine the current motion information of grid 313 based on such the motion direction with the largest proportion, etc.

[0056] Additionally, the image processing device 110 can determine the motion vectors of other grids (e.g., grid 1 311, grid 2 312, grid 4 314, grid 5 315, etc.) in the plurality of grids based on a processing procedure similar to that of grid 3 313. Additionally, the image processing device 110 can determine first evaluation information for the plurality of grids based on the current motion information of the plurality of grids. For example, the image processing device 110 can determine a first evaluation score indicating that each grid in the plurality of grids will move outside the screen area 210 based on the average motion speed, average motion direction, etc., of each grid in the plurality of grids.

[0057] As an example, such as Figure 1 As shown, when the image moves to the right, the right-hand area of ​​image frame 120 may include the grid that will move outside the screen area 210. In subsequent frames, the grid that will move outside the screen area 210 will display new image content. Thus, the image processing device 110 can understand the current speed and direction of multiple grids and provide a data basis for subsequent trend prediction.

[0058] In some embodiments, in order to improve the accuracy of the first evaluation information, the image processing device 110 may determine the predicted position information of the multiple grids at a future time based on the current motion information of the multiple grids; and determine the first evaluation information of the multiple grids based on whether the predicted position information exceeds the screen area.

[0059] As an example, the image processing device 110 can determine the predicted unknown information of multiple grids at future times using any appropriate trend prediction method. The first evaluation information may include the probability that the corresponding grid exceeds the screen area 210, the direction in which the corresponding grid exceeds the screen area 210, the predicted distance at which the corresponding grid will exceed the screen area 210, and so on.

[0060] Taking grid 313 as an example again, the image processing device 110 can determine the difference between the current motion vector and the motion vector of multiple pixels in grid 313 and the previous frame. Furthermore, the image processing device 110 can use such difference as a predicted acceleration to determine the predicted displacement of a subsequent frame (e.g., a frame at a future time, which may be a time that is slightly later than the current time) by using the current motion vector and the predicted acceleration of multiple pixels in grid 313.

[0061] Therefore, the image processing device 110 can determine the predicted position information of grid 313 at a future time by predicting the displacement, and determine whether the predicted position information exceeds the screen area 210. Furthermore, the image processing device 110 can determine the first evaluation information of grid 313 based on the determination result of whether the predicted position information exceeds the screen area 210.

[0062] For example, if multiple pixels of grid 313 are currently moving upward (i.e., the average direction of movement is to the right), and the predicted displacement exceeds the distance between grid 313 and the upper boundary of screen area 210 (i.e., the predicted position information indicates that it is outside screen area 210), then the image processing device 110 can mark grid 313 as "potentially crossing the boundary from the right in the future" as at least part of the first evaluation information of grid 313.

[0063] Based on a similar evaluation process for grid 311, the image processing device 110 can determine first evaluation information for multiple grids. Therefore, embodiments of this disclosure can predict portions of multiple grids that will extend beyond the screen area 210, thus avoiding insufficient data after pixels cross boundaries when rendering the corresponding additional areas, thereby improving the quality of image rendering.

[0064] In step 330, the image processing device 110 determines second evaluation information of the multiple grids based on the historical motion information of the multiple grids. The second evaluation information indicates the smoothness of the motion of the multiple grids.

[0065] In some examples, historical motion information from multiple grids can indicate motion information corresponding to one or more historical frames, such as the historical motion direction, historical motion speed, historical rate of change, etc. of multiple pixels within each grid.

[0066] Furthermore, the image processing device 110 can analyze historical motion information of multiple grids to assess the smoothness of the motion of the multiple grids. As an example, the second evaluation information may include scores on the smoothness of the motion of the multiple grids, markers indicating whether it is smooth, etc.

[0067] In some examples, the image processing device 110 can determine whether the motion vectors of pixels change significantly between multiple frames to assess the smoothness of motion across multiple grids. Specifically, for a second grid among multiple grids, the image processing device 110 can determine a first difference between the current motion information and historical motion information of the second grid; and based on the first difference, determine a second evaluation information for the second grid.

[0068] As an example, the first difference can be determined by an appropriate vector interpolation method. The second grid and the first grid described above can be the same grid or different grids among multiple grids. Taking grid four 314 as an example, the image processing device 110 can determine the first difference between the current motion information of grid four 314 and the historical motion information at a historical moment. Thus, the image processing device 110 can measure the temporal consistency of grid four 314 based on the first difference.

[0069] As an example, the image processing device 110 can determine second comment information for grid four 314 based on a first difference. As an example, the image processing device 110 can compare the first difference with a preset difference, and if the first difference exceeds the preset difference, mark grid four 314 as a "high-risk area" as at least part of such second comment information.

[0070] Alternatively, the image processing device 110 can determine variance information of multiple motion vectors of multiple pixels in the second grid; and based on the variance information, determine second evaluation information of the second grid.

[0071] As an example, such variance information can measure whether the differences between multiple motion vectors of multiple pixels in a grid are large, thereby determining the spatial consistency of the grid. For instance, if the differences between multiple motion vectors of multiple pixels in a grid are large, i.e., the variance is large, the grid has low stability, and therefore, subsequent rendering processes can prioritize rendering extended areas of that grid.

[0072] Alternatively or additionally, the image processing device 110 can determine the depth gradient (e.g., a rate of change representing depth) of one of a plurality of grids to assess the probability of occlusion flipping of that grid, and determine second motion information of the plurality of grids based on the depth gradients of the plurality of grids. As an example, a larger depth gradient can characterize that a foreground object in that grid is about to enter or leave the screen boundary. For example, if the depth gradient of grid four 314 exceeds a preset rate of change, the image processing device 110 can mark such a grid as a "high-risk area" as at least part of the second comment information of grid four 314.

[0073] In this way, the image processing device 110 can prioritize expanding the less stable parts of the screen area 210 to avoid problems such as image structural damage and stretching during edge transformation, thereby improving the image rendering quality.

[0074] It should be understood that the image processing device 110 can determine the second motion information of multiple grids based on one or more of the first difference, variance information, and depth gradient mentioned above. For example, the image processing device 110 can determine the variance information of multiple motion vectors of multiple pixels in the second grid; and determine the second evaluation information of the second grid based on the first difference and variance information. As an example, the specific implementation of the image processing device 110 determining the second motion information of multiple grids based on one or more of the first difference, variance information, and depth gradient mentioned above can be achieved through appropriate combinations of the embodiments described above, and the embodiments of this disclosure will not be repeated here.

[0075] In step 340, the image processing device 110 determines a rendering strategy based at least on the first evaluation information and the second evaluation information. The rendering strategy indicates the regional attributes of the additional region to be rendered, which is outside the screen area.

[0076] In some examples, such region attributes may include extended attribute information about the additional region, such as the direction of expansion, the extent of expansion, etc. For example, the direction of expansion may include one or more directions. The extent of expansion may correspond to different pixel widths.

[0077] In some embodiments, in order to improve the accuracy of determining the additional region, the image processing device 110 may determine grid description information of multiple grids to quantify the first evaluation information and the second evaluation information, thereby more accurately determining the extent of the additional region.

[0078] Specifically, the image processing device 110 can determine mesh description information for multiple meshes based at least on first evaluation information and second evaluation information. The mesh description information indicates at least one of the following: the direction of movement of the mesh, the boundary that the mesh will cross, the movement speed of the mesh, the movement stability of the mesh, and the importance of the objects associated with the mesh. Based on the mesh description information, the device determines a target expansion strategy for the multiple meshes. The target expansion strategy indicates the expansion direction and expansion magnitude of each mesh. Based on the target expansion strategy, the device determines the region attributes of the additional region to be rendered. The region attributes include the position and size of the additional region.

[0079] As an example, the mesh description information may include individual mesh description information for each mesh, which may indicate at least one of the following for each mesh: direction of motion, boundary to be crossed, motion speed, motion stability, and importance of associated objects.

[0080] Continue to refer to Figure 4 Taking grid 313 as an example, the image processing device 110 can determine the average motion direction of grid 313 based at least on the first evaluation information, as the motion direction of grid 313 in the grid description information. Alternatively or additionally, the image processing device 110 can determine the average motion speed of grid 313 based at least on the first evaluation information, as the motion direction of grid 313 in the grid description information. Alternatively or additionally, the image processing device 110 can determine the predicted position information of multiple pixels in grid 313 based at least on the first evaluation information, to determine the boundary (e.g., right boundary) that grid 313 will cross.

[0081] Alternatively or additionally, the image processing device 110 may determine at least one of the motion vector differences between multiple pixels in grid 313, the first difference between the current motion information and the historical motion information of grid 313, based at least on the second evaluation information, to determine the motion stability (e.g., the smoothness of motion) of grid 313.

[0082] Alternatively or additionally, the image processing device 110 can also determine the importance of objects in grid 313 based on first evaluation information and second evaluation information. For example, the image processing device 110 can identify object information represented by multiple pixels in grid 313 and determine the type of objects in grid 313 based on the object information. As an example, different object types can be preset with different levels of importance. Thus, the image processing device 110 can determine the importance of objects in grid 313 based on the type of objects in grid 313.

[0083] Based on this, the image processing device 110 can comprehensively evaluate the expansion requirements of each mesh in multiple meshes during the rendering process from multiple dimensions. As an example, the image processing device 110 determines the target expansion strategy for multiple meshes based on mesh description information.

[0084] As an example, such a target expansion strategy can be set according to the actual needs of the scenario. For example, such a target expansion strategy can indicate the expansion direction and expansion magnitude of each grid. As an example, such expansion direction is determined by the aforementioned motion direction, the boundary to be crossed, predicted position information, etc., and can, for example, indicate the direction of expansion from the corresponding grid beyond the screen area 210. In some examples, the expansion direction may include one or more directions.

[0085] As an example, such expansion magnitude can be determined by information such as the direction of motion, the speed of the mesh movement, predicted position information, and distance from the boundary, which may indicate, for example, the width of the pixels to be expanded. In some examples, such expansion magnitude can be quantified by a rating system, with different expansion magnitudes corresponding to different pixel widths. For example, expansion magnitudes could include levels 0, 1, 2, 3, and so on.

[0086] like Figure 4 As shown, taking an additional region 220 including region 321, region 322, and region 323 as an example, the region attributes of the additional region can include the expansion direction of region 321, region 322, and region 323 relative to the screen region 210, which can be to the right. The expansion width of region 321 in the additional region 220 can be greater than the expansion width of region 323 in the additional region 220.

[0087] Therefore, the embodiments of this disclosure can provide additional regions that change dynamically over time, and the regions to be expanded can be rendered reasonably based on such additional regions, thereby improving the utilization of computing resources and improving the quality of image rendering.

[0088] Furthermore, the image processing device 110 can determine a predictive expansion strategy based on grid description information; and perform temporal smoothing processing based on the historical expansion strategies and predictive expansion strategies of multiple grids to determine the target expansion strategy for multiple grids.

[0089] As an example, a predictive expansion strategy can be used to determine additional regions to be rendered. The predictive expansion strategy can be grid-dimensional; for example, it can be a predictive expansion strategy per grid. Alternatively, the predictive expansion strategy can also be boundary-dimensional; for example, it can indicate an expansion strategy for a specified boundary of screen region 210.

[0090] In some examples, to avoid screen flickering caused by rapid changes in the additional regions between different frames, the image processing device 110 can perform temporal smoothing processing based on the historical and predicted expansion strategies of multiple grids to determine the target expansion strategy for multiple grids.

[0091] As an example, a history expansion strategy can instruct multiple meshes to expand their rendering strategy outside screen area 210 during the rendering of historical frames. For instance, the image processing device 110 can perform temporal smoothing on the amount of expansion of multiple meshes in the additional area to make the transition between historical frames and the current frame more natural.

[0092] Alternatively or additionally, if the image content of the current frame (e.g., the current rendering frame) changes little compared to historical frames, the image processing device 110 may also reuse the extended rendering content of historical frames in their corresponding additional regions during the rendering of the current frame, in order to further reduce rendering costs and improve the temporal consistency between the current frame and historical frames.

[0093] In some embodiments, the renderer of the image processing device 110 corresponds to limited computing resources. To improve the utilization of computing resources, the image processing device 110 may adjust the target expansion strategy of multiple meshes based on the availability of computing resources; and determine the regional attributes of additional regions to be rendered based on the adjusted target expansion strategy.

[0094] As an example, the availability status of computing resources may include the available budget for computing resources, whether additional rendering extensions are indicated, and so on. Furthermore, the image processing device 110 can calculate the total resource requirements of the additional regions corresponding to multiple meshes, and if such total resource requirements exceed the available budget for computing resources, adjust the target extension strategy of the multiple meshes to reduce the total resource requirements and ensure rendering efficiency.

[0095] As an example, the image processing device 110 can adjust the target expansion strategy of multiple meshes based on the mesh description information described above. For example, the image processing device 110 can retain the additional regions corresponding to higher priority meshes while reducing the expansion amount of the additional regions corresponding to lower priority meshes. Thus, embodiments of this disclosure can dynamically optimize the target expansion strategy to improve the reliability of rendering.

[0096] In order to further improve the rationality of the additional area and avoid wasting the budget of the additional area on the expansion of less important areas, the image processing device 110 can also identify the object information of pixels in multiple grids, so that objects of higher importance can be given priority to obtain sufficient additional area, and the expansion priority of less important screen content such as background areas can be reduced.

[0097] As an example, the image processing device 110 can determine third evaluation information of multiple grids based on the classification information of pixels in multiple grids, the third evaluation information indicating whether the pixels in the multiple grids belong to objects of a preset category; and determine a rendering strategy based on the first evaluation information, the second evaluation information and the third evaluation information.

[0098] Continue to refer to Figure 4 Taking grid 313 as an example, the image processing device 110 can identify one or more pixels in grid 313 and determine whether grid 313 includes objects belonging to a preset category based on object recognition of multiple pixels. As an example, such a preset category can indicate object types with higher importance, such as a person's body, an object to be interacted with by the user, etc. Therefore, the image processing device 110 can determine the expansion priority of grid 313 based on whether grid 313 includes objects belonging to the preset category. For example, the expansion priority of a grid that includes objects such as a person's body can be higher than the expansion priority of a grid that only includes background content.

[0099] As an example, the image processing device 110 can determine a rendering strategy based on one or more of the first evaluation information, second evaluation information, and third evaluation information. The combination of different evaluation information can be implemented through combinations of some embodiments of this disclosure, which will not be elaborated further in the embodiments of this disclosure. Therefore, the embodiments of this disclosure can further improve the rationality of the rendering strategy.

[0100] In step 350, the image processing device 110 generates a rendering frame based on a rendering strategy, the rendering frame covering the screen area and an additional area.

[0101] As an example, the rendering frame can be the currently rendered image frame, which can be used, for example, to display the current scene in a virtual scene. The rendering frame can cover screen area 210 and additional area 220. As an example, during the display of this rendering frame, the image processing device 110 can display only the portion of the rendering frame that covers screen area 210. For example, such as Figure 4 As shown, the rendering frame can cover screen area 210 and additional area 220, and can only show the part covering screen area 210 to the user.

[0102] In step 360, the image processing device 110 performs extrapolation frame processing on the rendered frame to generate an extrapolation frame.

[0103] As an example, the image processing device 110 can use a portion of the rendered frame covering the additional region 220 as a reference to a future real frame to predict the portion of the rendered frame covering the screen region 210, thereby generating such an extrapolated frame. Thus, embodiments of this disclosure, by referencing a future real frame (e.g., a portion of the rendered frame covering the additional region 220), enable the generated extrapolated frame to have higher rendering quality.

[0104] Based on this approach, embodiments of this disclosure can analyze whether multiple grids are about to move off-screen based on their current motion information within the screen area, prioritizing the rendering of grids that are about to move off-screen. Furthermore, embodiments of this disclosure can analyze the smoothness of the motion of multiple grids based on their historical motion information, prioritizing the rendering of grids with less stable motion. Thus, embodiments of this disclosure can dynamically allocate image rendering computational resources through a rendering strategy determined at least based on first and second evaluation information, allowing computational resources to be directed towards additional areas that require more rendering, thereby improving the utilization of computational resources. In addition, embodiments of this disclosure can generate rendering frames based on this rendering strategy, thereby rendering more appropriate additional areas during the rendering frame generation process. Furthermore, embodiments of this disclosure can ensure that the interpolated frames generated based on the rendering frames not only reference the portion of the rendering frame covering the screen area but also reference appropriate additional areas, thereby improving the quality of the interpolated frames and the overall image rendering quality.

[0105] Therefore, the embodiments of this disclosure can dynamically adjust the additional area to be rendered, improve the utilization of computing resources, and improve the quality of image rendering.

[0106] Example devices and equipment:

[0107] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an example apparatus 500 for image rendering according to certain embodiments of the present disclosure is shown. Apparatus 500 may be implemented as or included in an image processing device 110. Various modules / components in apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0108] like Figure 5 As shown, the device 500 includes a partitioning module 510 configured to divide a screen area into multiple grids; a first determining module 520 configured to determine first evaluation information of the multiple grids based on current motion information of the multiple grids, the first evaluation information indicating whether the pixels corresponding to the multiple grids will move outside the screen area; a second determining module 530 configured to determine second evaluation information of the multiple grids based on historical motion information of the multiple grids, the second evaluation information indicating the smoothness of the motion of the multiple grids; a third determining module 540 configured to determine a rendering strategy based at least on the first evaluation information and the second evaluation information, the rendering strategy indicating the regional attributes of an additional region to be rendered, the additional region being outside the screen area; a rendering module 550 configured to generate a rendering frame based on the rendering strategy, the rendering frame covering the screen area and the additional region; and a generating module 560 configured to perform interpolation frame processing on the rendering frame to generate an interpolated frame.

[0109] In some embodiments, the apparatus 500 further includes a fourth determining module configured to: determine motion vectors of a plurality of pixels in a first grid among a plurality of grids; and determine current motion information of the first grid based on the motion vectors of the plurality of pixels.

[0110] In some embodiments, the current motion information indicates: the average motion speed of multiple pixels; and the average motion direction of multiple pixels.

[0111] In some embodiments, the first determining module 520 is further configured to: determine the predicted position information of the multiple grids at a future time based on the current motion information of the multiple grids; and determine the first evaluation information of the multiple grids based on whether the predicted position information exceeds the screen area.

[0112] In some embodiments, the second determining module 530 is further configured to: determine a first difference between the current motion information and historical motion information of a second grid among a plurality of grids; and determine second evaluation information of the second grid based on the first difference.

[0113] In some embodiments, the second determining module 530 is further configured to: determine variance information of multiple motion vectors of multiple pixels in the second grid; and determine second evaluation information of the second grid based on the first difference and variance information.

[0114] In some embodiments, the second determining module 530 is further configured to: determine a second difference between the current depth information and the historical depth information of the second grid; and determine a second evaluation information of the second grid based on the first difference and the second difference.

[0115] In some embodiments, the third determining module 540 is further configured to determine third evaluation information of the multiple grids based on the classification information of pixels in the multiple grids, the third evaluation information indicating whether the pixels in the multiple grids belong to objects of a preset category; and to determine a rendering strategy based on the first evaluation information, the second evaluation information and the third evaluation information.

[0116] In some embodiments, the third determining module 540 is further configured to determine mesh description information for a plurality of meshes based at least on first evaluation information and second evaluation information, wherein the mesh description information indicates at least one of the following: the direction of movement of the mesh, the boundary that the mesh will cross, the movement speed of the mesh, the movement stability of the mesh, and the importance of the objects associated with the mesh; determine a target expansion strategy for the plurality of meshes based on the mesh description information, wherein the target expansion strategy indicates the expansion direction and expansion magnitude of each mesh; and determine the region attributes of an additional region to be rendered based on the target expansion strategy, wherein the region attributes include the position and size of the additional region.

[0117] In some embodiments, the third determining module 540 is further configured to adjust the target expansion strategy of multiple meshes based on the availability of computing resources; and to determine the regional attributes of the additional region to be rendered based on the adjusted target expansion strategy.

[0118] In some embodiments, the third determining module 540 is further configured to determine a predictive expansion strategy based on grid description information; and to perform time smoothing processing based on the historical expansion strategies and predictive expansion strategies of multiple grids to determine the target expansion strategy of multiple grids.

[0119] The modules included in device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the modules in device 500 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0120] Figure 6 A block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 6 The electronic device 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to achieve Figure 1 Image processing device 110 or Figure 5 The device 500.

[0121] like Figure 6 As shown, electronic device 600 is in the form of a general-purpose electronic device. Components of electronic device 600 may include, but are not limited to, at least one processor 610 or processing unit, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processor 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 600.

[0122] Electronic device 600 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 600.

[0123] Electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0124] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 600 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 600 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0125] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 600 can also communicate with one or more external devices (not shown) via communication unit 640 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 600, or with any device that enables electronic device 600 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interfaces (not shown).

[0126] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0127] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0128] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0129] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0131] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for image rendering, characterized in that, The method includes: Divide the screen area into multiple grids; Based on the current motion information of the multiple grids, a first evaluation information of the multiple grids is determined, wherein the first evaluation information indicates whether the pixels corresponding to the multiple grids will move outside the screen area; Based on the historical motion information of the multiple grids, a second evaluation information for the multiple grids is determined, the second evaluation information indicating the smoothness of the motion of the multiple grids; A rendering strategy is determined based at least on the first evaluation information and the second evaluation information, wherein the rendering strategy indicates the regional attributes of an additional region to be rendered, the additional region being outside the screen area; Based on the rendering strategy, a rendering frame is generated, the rendering frame covering the screen area and the additional area; and The rendered frame is subjected to interpolation processing to generate an interpolated frame.

2. The method according to claim 1, characterized in that, The method further includes: For a first grid among the plurality of grids, determine the motion vectors of multiple pixels in the first grid; and The current motion information of the first grid is determined based on the motion vectors of the plurality of pixels.

3. The method according to claim 2, characterized in that, The current motion information indicates: The average motion speed of the plurality of pixels; The average direction of motion of the plurality of pixels.

4. The method according to claim 1, characterized in that, Based on the current motion information of the multiple grids, the first evaluation information of the multiple grids is determined as follows: Based on the current motion information of the multiple grids, the predicted position information of the multiple grids at future times is determined; and The first evaluation information of the plurality of grids is determined based on whether the predicted position information exceeds the screen area.

5. The method according to claim 1, characterized in that, Based on the historical motion information of the multiple grids, the second evaluation information of the multiple grids is determined as follows: For the second grid among the plurality of grids, determine a first difference between the current motion information and the historical motion information of the second grid; and Based on the first difference, the second evaluation information of the second grid is determined.

6. The method according to claim 5, characterized in that, Based on the first difference, the second evaluation information for the second grid is determined to include: Determine the variance information of multiple motion vectors of multiple pixels in the second grid; and Based on the first difference and the variance information, the second evaluation information of the second grid is determined.

7. The method according to claim 5, characterized in that, Based on the first difference, the second evaluation information for the second grid is determined to include: Determine the second difference between the current depth information and the historical depth information of the second grid; and Based on the first difference and the second difference, the second evaluation information of the second grid is determined.

8. The method according to claim 1, characterized in that, The rendering strategy is determined based at least on the first evaluation information and the second evaluation information, including: Based on the classification information of pixels in the plurality of grids, a third evaluation information is determined for the plurality of grids, the third evaluation information indicating whether the pixels in the plurality of grids belong to a preset category of objects; and The rendering strategy is determined based on the first evaluation information, the second evaluation information, and the third evaluation information.

9. The method according to claim 1, characterized in that, The rendering strategy is determined based at least on the first evaluation information and the second evaluation information, including: Based at least on the first evaluation information and the second evaluation information, mesh description information is determined for the plurality of meshes, wherein the mesh description information indicates at least one of the following: the direction of movement of the mesh, the boundary that the mesh will cross, the movement speed of the mesh, the movement stability of the mesh, and the importance of the objects associated with the mesh; Based on the mesh description information, a target expansion strategy for the plurality of meshes is determined, wherein the target expansion strategy indicates the expansion direction and expansion magnitude of each mesh; and Based on the target expansion strategy, the region attributes of the additional region to be rendered are determined, including the position and size of the additional region.

10. The method according to claim 9, characterized in that, Based on the target expansion strategy, the region attributes of the additional region to be rendered are determined as follows: Based on the availability of computing resources, adjust the target expansion strategy of the multiple grids; and Based on the adjusted target expansion strategy, the region attributes of the additional region to be rendered are determined.

11. The method according to claim 9, characterized in that, Based on the mesh description information, the target expansion strategy for the multiple meshes is determined as follows: Based on the grid description information, a prediction expansion strategy is determined; and Based on the historical expansion strategies and the predicted expansion strategies of the multiple grids, time smoothing is performed to determine the target expansion strategy of the multiple grids.

12. An apparatus for image rendering, characterized in that, The device includes: The partitioning module is configured to divide the screen area into multiple grids; The first determining module is configured to determine first evaluation information of the multiple grids based on the current motion information of the multiple grids, wherein the first evaluation information indicates whether the pixels corresponding to the multiple grids will move outside the screen area; The second determining module is configured to determine second evaluation information of the multiple grids based on the historical motion information of the multiple grids, wherein the second evaluation information indicates the smoothness of the motion of the multiple grids; The third determining module is configured to determine a rendering strategy based at least on the first evaluation information and the second evaluation information, wherein the rendering strategy indicates the regional attributes of an additional region to be rendered, the additional region being outside the screen area; A rendering module is configured to generate rendering frames based on the rendering strategy, the rendering frames covering the screen area and the additional area; and The generation module is configured to perform extrapolation frame processing on the rendered frame to generate an extrapolation frame.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processor.

14. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that, The computer-executable instructions can be executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product, said computer program product being tangibly stored in a computer storage medium and comprising computer-executable instructions, characterized in that, The computer-executable instructions, when executed by the device, cause the device to perform the method according to any one of claims 1 to 11.

Images