Rendering method, storage medium, electronic equipment and program product
By dynamically adjusting the rendering mode based on the interaction status and battery level, the problem of 3D rendering under the hardware configuration limitations of electronic devices is solved, achieving improved visual effects and battery life while reducing power consumption.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ASR MICROELECTRONICS CO LTD
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-12
AI Technical Summary
Hardware limitations of electronic devices can lead to insufficient computing power during 3D rendering, resulting in issues such as screen tearing, system lag, and rapid battery drain.
The rendering mode is dynamically adjusted based on the interaction status and battery level of the electronic device, including high frame rate mode, low frame rate mode and low light mode. By adjusting the mesh model and number of vertices of the rendered object, the computational load and power consumption are reduced.
Without compromising the user's visual experience, reduce the power consumption of electronic devices, improve rendering visual effects, and extend battery life.
Smart Images
Figure CN122018664A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image technology, specifically to a rendering method, storage medium, electronic device, and program product. Background Technology
[0002] With the widespread adoption of electronic devices (such as smart wearables), users are demanding higher visual effects from their user interfaces (UIs), such as displaying a dynamically rotating three-dimensional (3D) Earth on a smartwatch face. However, electronic devices are limited by their hardware configurations, resulting in significant bottlenecks in rendering such 3D visual effects. For example, the vertex transformations and lighting calculations required for 3D rendering exceed the computing power of most electronic devices. Furthermore, frequent full-screen refreshes and insufficient video memory bandwidth for texture reading can easily cause screen tearing, system lag, and a degraded visual experience. Moreover, continuous 3D rendering rapidly depletes battery power, shortening the battery life of electronic devices. Therefore, there is an urgent need for a rendering method adapted to electronic devices with limited hardware configurations, which can effectively improve the visual effects of rendering while reducing power consumption. Summary of the Invention
[0003] In view of this, embodiments of the present invention provide a rendering method, a storage medium, an electronic device, and a program product. In a first aspect, the present invention provides a rendering method applied to an electronic device, the electronic device including a touch screen, the method comprising: acquiring the interaction state and battery level of the electronic device; determining the rendering mode corresponding to the object to be rendered based on the interaction state and battery level, wherein different rendering modes include different numbers of vertices used to render the object to be rendered.
[0004] In one possible implementation of the first aspect above, the rendering modes include a high frame rate mode, a low frame rate mode, and a low-light mode. The method further includes: rendering the object to be rendered in a high frame rate mode when the interaction state is a swipe operation on the touchscreen and / or the battery level exceeds a first battery threshold; rendering the object to be rendered in a low frame rate mode when the interaction state is a static display time on the touchscreen exceeding a first time threshold and / or the battery level does not exceed a second battery threshold; and rendering the object to be rendered in a low-light mode when the interaction state is an electronic device in low-light mode and the electronic device is in sleep mode and / or the electronic device detects a forced screen-off. The first battery threshold is greater than or equal to the second battery threshold, and the mesh models of the objects to be rendered are different for each rendering mode, with the mesh model in the low frame rate mode containing fewer vertices than the mesh model in the high frame rate mode.
[0005] In one possible implementation of the first aspect above, a mesh model of the object to be rendered is obtained, wherein the object to be rendered includes a central region and an edge region, the mesh model is a non-uniform mesh, the non-uniform mesh includes multiple vertices, and the number of vertices corresponding to the central region and the edge region are different in the non-uniform mesh of the object to be rendered; the vertices on the non-uniform mesh are pre-stored in an electronic device.
[0006] In one possible implementation of the first aspect described above, the object to be rendered is a sphere.
[0007] In one possible implementation of the first aspect described above, a non-uniform mesh is determined based on the angle between the normals on the sphere and the center of the sphere, and the mesh step size, wherein the mesh step size is the difference between the angles between the normals of two adjacent vertices on the sphere and the center of the sphere.
[0008] In one possible implementation of the first aspect above, a non-uniform mesh is determined based on the angle between the normals on the sphere and the center of the sphere and the mesh step size, including: when 0°≤α≤60°, the mesh step size is 10°≤λ≤15°, where α represents the angle between the normals on the sphere and the center of the sphere, and λ represents the mesh step size; when 60°<α≤90°, the mesh step size is 2°≤λ≤4°.
[0009] In one possible implementation of the first aspect above, based on the current time, a rotation operation is performed on the first coordinate of the vertex to obtain the second coordinate, where the first coordinate is the coordinate of the vertex on the mesh model; the second coordinate is projected onto a two-dimensional space to obtain the third coordinate, which can be displayed on a touch screen.
[0010] In one possible implementation of the first aspect above, the quadrilateral located on the back of the object to be rendered in the mesh model is determined according to the following formula:
[0011] Where n represents the normal vector of any quadrilateral in the grid. Indicates the direction of the line of sight. In In the case where the quadrilateral corresponding to the normal vector n is located on the back of the object to be rendered; delete the vertices contained in the quadrilateral located on the back of the object to be rendered in the mesh model to obtain the first vertex set.
[0012] In one possible implementation of the first aspect above, a background texture image of the object to be rendered is obtained, wherein the background texture image is the background image of the object to be rendered on the touch screen. Obtain the texture image of the object to be rendered, fill the texture image of the object to be rendered into the first vertex set corresponding to the mesh model to obtain the object image of the object to be rendered on the touch screen; obtain the mask image corresponding to the object image, the mask image is used to cover the mesh of the edge area of the object image; use the background texture image as the first image layer, the object image as the second image layer, and the mask image as the third image layer, and render the object to be rendered according to the first image layer, the second image layer, and the third image layer.
[0013] In one possible implementation of the first aspect above, a first instruction is generated based on the first vertex set. The first instruction is used to generate a texture image of the object to be rendered. The first instruction includes: a source image, which refers to a panoramic texture image of the object to be rendered; a source region, which refers to the region where the first vertex set is located; a target region, which refers to the region where the third coordinate of the quadrilateral corresponding to the first vertex set is located; and a transformation matrix, which refers to the matrix that maps the source region to the target region.
[0014] In a second aspect, embodiments of the present invention provide a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to implement any of the rendering methods provided by the first aspect and various possible implementations of the first aspect. Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, one of the processors of the electronic device, for executing the instructions stored in the memory to implement any of the rendering methods provided by the first aspect and various possible implementations of the first aspect.
[0015] Fourthly, embodiments of the present invention provide a program product that includes instructions that, when executed by an electronic device, enable the electronic device to implement any of the rendering methods provided by the first aspect and various possible implementations of the first aspect.
[0016] Compared with the prior art, the embodiments of this application have the following advantages: The method provided in this application determines different rendering modes based on the different interaction states and battery levels of the electronic device, and can classify the rendered visual effects. This reduces the power consumption of the electronic device without degrading the user's visual experience. When the mesh model in low frame rate mode contains fewer vertices than the mesh model in high frame rate mode, rendering the object in low frame rate mode can also reduce the power consumption of the electronic device, while simultaneously reducing the computational load during the rendering process to improve the visual effect. Attached Figure Description
[0017] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0018] Figure 1 This is a flowchart of a rendering method according to an embodiment of this application; Figure 2 This is a schematic diagram of a non-uniform mesh of a sphere according to an embodiment of this application; Figure 3 This is a schematic diagram of an image corresponding to a multi-layer rendering method according to an embodiment of this application; Figure 4 This is a flowchart of another rendering method according to an embodiment of this application; Figure 5 This is a state machine transition logic diagram of a dynamic rendering mode according to an embodiment of this application; Figure 6 This is a structural diagram of an electronic device 10 according to an embodiment of this application. Detailed Implementation
[0019] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] It should be noted that the following description covers various aspects of embodiments within the scope of the appended claims. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.
[0021] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0022] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.
[0023] As mentioned earlier, with the widespread adoption of electronic devices (such as smart wearables), users have placed higher demands on the visual effects of electronic device UIs. For example, displaying a dynamically rotating 3D globe on a smartwatch face. However, due to limitations in the hardware configuration of electronic devices, for instance, the computational power required for 3D rendering (e.g., vertex transformation, lighting calculations) exceeds the computing power of the electronic device. Furthermore, the frequent full-screen refreshes and texture data reading operations during 3D rendering require video memory bandwidth exceeding the video memory bandwidth of the electronic device, leading to problems such as screen tearing and system lag, thus degrading the visual effect. Finally, continuous 3D rendering consumes battery power, shortening the battery life of electronic devices.
[0024] To address the aforementioned issues, existing technical solutions typically employ two approaches: one is to play a pre-rendered sequence of images on the electronic device's display screen. While this method ensures visual quality on the screen, it requires a significant amount of storage space on the electronic device. The other approach involves real-time rendering on the electronic device. However, limited by the device's CPU processing power, real-time rendering often results in low frame rates and stuttering. Both methods struggle to achieve a balance between visual detail, rendering smoothness, and the electronic device's battery life.
[0025] Therefore, there is an urgent need for a rendering method that is compatible with electronic devices with limited hardware configurations, which can reduce the power consumption of electronic devices and improve the visual effects of rendering.
[0026] Based on this, embodiments of this application provide a rendering method applied to an electronic device, which includes a touchscreen (or display screen). This rendering method can acquire the interaction state and battery level of the electronic device, and determine the rendering mode corresponding to the object to be rendered based on the interaction state and battery level. Different rendering modes involve different numbers of vertices used to render the object, interaction states include swiping operations on the touchscreen, and the touchscreen display being static, etc., and rendering modes include high frame rate mode, low frame rate mode, and always-on display (AOD) mode, etc.
[0027] In some embodiments, when the interaction state is a swipe operation on the touchscreen, and / or the battery level exceeds a first battery threshold, the electronic device can render the object to be rendered in a high frame rate mode. When the touchscreen display remains still for more than a first time threshold, and / or the battery level does not exceed a second battery threshold, the electronic device can render the object to be rendered in a low frame rate mode. When the interaction state is that the electronic device is in low-light mode and in sleep mode, and / or the electronic device detects a forced screen-off event, the electronic device can render the object to be rendered in low-light mode. Wherein, the first battery threshold is greater than or equal to the second battery threshold, the mesh model of the object to be rendered differs for each rendering mode, and the mesh model in low frame rate mode contains fewer vertices than the mesh model in high frame rate mode.
[0028] Understandably, in low-light mode, electronic devices display static images on the touchscreen, and the electronic devices only refresh the time, without re-rendering the image.
[0029] It's understandable that determining different rendering modes based on the varying interaction states and battery levels of electronic devices allows for the categorization of rendered visual effects. This, in turn, reduces the power consumption of electronic devices without compromising the user's visual experience. Furthermore, when the mesh model in low frame rate mode contains fewer vertices than that in high frame rate mode, rendering the object in low frame rate mode can also reduce the power consumption of electronic devices and simultaneously decrease the computational load during rendering, thereby improving the visual quality.
[0030] Figure 1 A flowchart of a rendering method is shown according to some embodiments of this application. It can be understood that... Figure 1 The processes shown are all executed by electronic devices. For simplicity, the following description... Figure 1 The execution entity will not be described again in the process shown. For example... Figure 1 As shown, including but not limited to the following processes: S101, Based on the user's viewpoint received by the electronic device, a non-uniform mesh of the object to be rendered is created.
[0031] It's understandable that during the rendering of an object, the electronic device uniformly divides the object into a grid based on its latitude and longitude, and then obtains the vertices of the object from this grid. Consequently, when the electronic device displays the object, the perspective distortion is smaller in the central area and larger in the edge areas, resulting in a perspective distortion on the display where "vertices in the central area are too dense, and vertices in the edge areas are too sparse." Therefore, during rendering, the electronic device needs to use computationally expensive filtering algorithms to eliminate this perspective distortion.
[0032] To reduce the computational cost of rendering on electronic devices, the device can dynamically adjust the mesh density of the object to be rendered based on the user's viewpoint received by the device, thus creating a non-uniform mesh for the object. This can be understood as... Figure 2 As shown, a grid refers to a set of quadrilaterals that an electronic device can use to divide the visible half (front) of an object to be rendered into M rows (latitude direction, where M is a positive integer greater than or equal to 1) × N columns (longitude direction, where N is a positive integer greater than or equal to 1). Different regions of the object to be rendered correspond to the user's perspective, such as the center and edge regions of the object.
[0033] In some embodiments, such as Figure 2 As shown, taking a sphere as an example, the rendering process is based on the user's viewpoint received by the electronic device. This user viewpoint includes the central and edge regions of the sphere. In the central region of the sphere, the electronic device can create a larger mesh, reducing the number of vertices and the computational cost of the device. In the edge regions, the electronic device can create a high-density mesh, improving the smoothness of the sphere's outline, preventing severe texture stretching at the edges, which could lead to deformation and improve the visual effect.
[0034] For example, the electronic device can determine a non-uniform mesh based on the angle between the vertices and the normals to the center of the sphere (denoted by α) and a dynamically set mesh step size (denoted by λ). Specifically, for the central region of the sphere (e.g., 0°≤α<60°), the electronic device can set the mesh step size to 10°≤λ≤15°. This allows for a sparse mesh in the central region, reducing the number of vertices and lowering the computational cost of the electronic device. For the edge region of the sphere (e.g., 60°≤α≤90°), the electronic device can set the mesh step size to 2°≤λ≤4°. This further reduces the computational cost of the electronic device, while allowing for a denser mesh in the edge region, improving the smoothness of the sphere's outline and enhancing the visual appeal. It is understood that reducing the computational cost of the electronic device can reduce its overall efficiency.
[0035] In some embodiments, the electronic device may pre-store the vertices on the non-uniform mesh in the electronic device. For example, the electronic device may pre-store the 3D coordinates (represented by (x0,y0,z0)) of the vertices contained in the non-uniform mesh and the corresponding texture coordinates in the memory of the electronic device as a base mesh model for rendering the sphere by the electronic device.
[0036] It is understood that the aforementioned memory can be non-volatile memory (Flash). In this way, electronic devices can perform read-only operations on the 3D coordinates and texture coordinates of vertices from the non-volatile memory, preventing accidental modification of the electronic device. Furthermore, compared to other types of memory (such as random access memory, electrically erasable programmable read-only memory, etc.), non-volatile memory consumes less power.
[0037] In some embodiments, in order to control the power consumption of the electronic device during the rendering of a sphere, the electronic device can establish a multi-level detail (LOD) model library, which contains multiple reference mesh models, and the non-uniform meshes contained in the multiple reference mesh models have different numbers of vertices.
[0038] For example, this LOD library contains two different baseline mesh models. If the baseline mesh model contains 400-600 vertices, an electronic device can render a high-fidelity sphere based on this baseline mesh model. Based on this, the baseline mesh model can be used as a high-fidelity mesh model (denoted as LOD-High). If the baseline mesh model contains 80-120 vertices, an electronic device can render a sphere with low power based on this baseline mesh model. Based on this, the baseline mesh model can be used as a low-power mesh model (denoted as LOD-Low).
[0039] It is understandable that electronic devices can determine 80-120 vertices based on the skeletal structure of the sphere to render the sphere, or they can render the sphere based on 80-120 vertices from other parts of the sphere; no limitation is imposed here.
[0040] It is understandable that electronic devices can determine different rendering modes using other methods, and no restrictions are imposed here.
[0041] S102: Obtain the interaction status and battery level of the electronic device, and determine the rendering mode corresponding to the object to be rendered based on the interaction status and battery level.
[0042] In some embodiments, the electronic device can obtain its interaction state through a rendering scheduler and its battery level through a power management integrated circuit (PMIC). Based on the interaction state and battery level, the electronic device determines the rendering mode corresponding to the object to be rendered.
[0043] It is understood that the interaction state includes swiping on the touchscreen of an electronic device, the user's finger (or stylus) touching the touchscreen surface, the user's finger (or stylus) leaving the touchscreen surface, and the touchscreen display remaining still. It is understood that the electronic device can determine the interaction state based on the actual situation, and no restrictions are imposed here.
[0044] It is understandable that rendering modes include high frame rate mode, low frame rate mode, and low light mode, and different rendering modes contain different numbers of vertices used to render the object to be rendered.
[0045] In some embodiments, the electronic device detects a swiping operation on the touchscreen, such as a user's finger (or stylus) moving across the touchscreen surface, and / or the electronic device also detects that the power level exceeds a first power threshold (e.g., power > 20%). Based on this, the electronic device can render the sphere according to LOD-High, and the electronic device can treat the LOD-High-based rendering process as a high frame rate mode, that is, the electronic device renders the object to be rendered in a high frame rate mode.
[0046] In some embodiments, the sphere displayed on the touchscreen of the electronic device is in the inertial rotation damping phase (i.e., the electronic device detects that the user's finger or stylus has left the touchscreen surface, but the sphere continues to rotate due to inertia), and / or, the electronic device also detects that the power level exceeds a first power threshold (e.g., power > 20%). Based on this, the electronic device can render the sphere according to LOD-High, and the electronic device can treat the LOD-High-based rendering process as a high frame rate mode, that is, the electronic device renders the object to be rendered through a high frame rate mode.
[0047] In some embodiments, based on the high frame rate mode, the electronic device renders the sphere with a smoothness of 60 frames per second (FPS), meaning that the electronic device can render 60 images per second. Furthermore, the surface of the sphere uses a high-resolution (e.g., 1024×512) texture, and the touchscreen has full-screen anti-aliasing.
[0048] In some embodiments, the electronic device detects that the touchscreen display remains still for more than a first time threshold (e.g., 2 seconds), and / or that the power level does not exceed a second power threshold (e.g., 20%). The electronic device can render the sphere according to LOD-Low, and the electronic device can treat the LOD-Low-based rendering process as a low frame rate mode, that is, the electronic device renders the object to be rendered in a low frame rate mode.
[0049] In some embodiments, based on a low frame rate mode, the electronic device renders the sphere at a smoothness of 15 FPS, meaning it can render 15 images per second. Furthermore, based on the low frame rate mode, the electronic device can reduce the texture sampling rate to decrease the bandwidth per sample. This reduces the power consumption of the electronic device.
[0050] In some embodiments, when the electronic device is in low-light mode and in sleep mode, and / or when the electronic device detects a forced screen-off event, the electronic device can render the object to be rendered in low-light mode. Specifically, the electronic device can stop the rendering process based on the sphere's 3D coordinates and only display a pre-rendered static sphere bitmap. The electronic device can also refresh the position of the static sphere every minute, without limitation.
[0051] In some embodiments, the first power threshold is greater than or equal to the second power threshold.
[0052] In some embodiments, the mesh model of the object to be rendered is different for different rendering modes.
[0053] In some embodiments, the mesh model in low frame rate mode contains fewer vertices than the mesh model in high frame rate mode.
[0054] S103 performs rotation and geometric transformation on the vertices included in the rendering mode, and removes the vertices located on the back of the object to be rendered, thus obtaining the first vertex set.
[0055] It is understandable that an electronic device can rotate the first coordinate of the vertex contained in the rendering mode based on the current time to obtain the second coordinate, where the first coordinate is the coordinate of the vertex on the mesh model.
[0056] Specifically, firstly, the electronic device can obtain the Earth's rotation angle θrot based on the current time t. For example, if the current time is 16:05, then the Earth's rotation angle θrot ≈ 155°. Next, the electronic device can use the Rodrigues rotation formula or the Euler angle matrix to rotate the first coordinates [x0, y0, z0] of any vertex included in the rendering mode by θrot around the Y-axis and then by a fixed tilt angle θtilt around the X-axis, obtaining the second coordinates [x`, y`, z`] of the rotated vertex. Specifically, the electronic device can perform rotation operations on all vertices included in the rendering mode according to the following formula: [x`, y`, z`] = Ry(θrot) Rx(θtitle) [x0, y0, z0] Where [x0, y0, z0] represents the first coordinates of any vertex included in the rendering mode, Rx(θtitle) represents a fixed tilt angle θtilt around the X-axis, Ry(θrot) represents a rotation θrot around the Y-axis, and [x`, y`, z`] represents the second coordinates obtained after rotating the vertex.
[0057] It is understandable that during the rotation operation on any vertex, the electronic device performs trigonometric function calculations (i.e., sin and cos). The electronic device can directly obtain the trigonometric function values through a lookup table (LUT), avoiding the need to calculate the trigonometric function values and thus accelerating the rotation operation.
[0058] In some embodiments, after rotating any vertex, the electronic device projects the rotated vertex into a two-dimensional (2D) space to obtain the third coordinates (xs, ys) of the projected vertex. Specifically, the electronic device can project the rotated vertex into 2D space according to the following formula:
[0059] Where D represents the distance from the virtual camera to the projection plane (i.e., the touchscreen or display screen), x` is the x-axis coordinate of the second coordinate system, y` is the y-axis coordinate of the second coordinate system, and z` is the z-axis coordinate of the second coordinate system. The x-coordinate of the center of the projection plane. The y-axis coordinate of the center of the projection plane, xs and ys are the x-axis coordinates of the third coordinates of the vertex projected onto the 2D space, and ys is the y-axis coordinate of the third coordinates of the vertex projected onto the 2D space.
[0060] It can be understood that by rotating any vertex on a non-uniform grid and projecting the vertex's first coordinate onto its second coordinate in 2D space, a static 3D sphere can be transformed into a dynamic sphere that rotates over time.
[0061] In some embodiments, the electronic device can remove vertices from quadrilaterals located on the back side of the object to be rendered in the mesh model to obtain a first vertex set. This reduces the computational load on the electronic device during the rendering process. Specifically, the electronic device can determine the quadrilaterals located on the back side of the object to be rendered in the mesh model according to the following formula:
[0062] Where n represents the normal vector of any quadrilateral in the grid. Indicates the direction of the line of sight. In In this case, the quadrilateral corresponding to the normal vector n is located on the back of the object to be rendered.
[0063] It is understandable that electronic devices can determine the direction of their gaze based on the position of the virtual camera, which will not be elaborated upon here.
[0064] S104, Generate a first instruction based on the first vertex set. The first instruction is used to generate the texture image of the object to be rendered.
[0065] In some embodiments, based on a first vertex set, the electronic device can generate a bitblock transfer (BLIT) instruction (as an example of a first instruction). Specifically, the BLIT instruction includes: a source image, i.e., a panoramic texture image of the object to be rendered; a source rect, i.e., the region where the first vertex set is located; a destination quad, i.e., the region where the third coordinate of the quadrilateral corresponding to the first vertex set is located; and a transformation matrix, i.e., a matrix that maps the coordinates of the source region to the coordinates of the destination region.
[0066] In some embodiments, a graphics processing unit (GPU) can establish a command buffer to drive a 2D graphics engine, such as a lightweight vector graphics library (VGLite), and render the objects to be rendered in slices using the 2D graphics engine.
[0067] In some embodiments, the GPU of the electronic device reads the BLIT instruction into the command buffer and decodes it to obtain the source image, source region, target region, and transformation matrix contained in the BLIT instruction. Further, the electronic device can divide the target region of the sphere displayed on the touchscreen into rectangular blocks containing multiple pixels. For example, the target region can be divided into 8×8 or 16×16 pixel rectangular blocks. Then, for each rectangular block, the electronic device can calculate the bounding box corresponding to that rectangular block in the source image and read all the original texture within the bounding box at once from the on-chip static random access memory. Next, the electronic device can perform bilinear interpolation on the original texture to generate the target pixels. Thus, with the same data throughput (i.e., all pixels in the target region of the sphere displayed on the touchscreen), reading all the target pixels within the bounding box at once can effectively reduce memory bandwidth and power consumption compared to randomly distributing the target pixels. It can be understood that the source image refers to a bitmap image, such as a starry sky background image.
[0068] S105: Based on the first vertex set, render the object to be rendered using a multi-layer rendering method.
[0069] In some embodiments, such as Figure 3 As shown, the electronic device can render the object to be rendered using a multi-layer rendering method. Specifically, the electronic device can acquire the background texture image of the object to be rendered, which is the background image of the object on the touchscreen; the electronic device can also acquire the texture image of the object to be rendered and fill the texture image of the object to be rendered into the first vertex set corresponding to the mesh model to obtain the object image of the object to be rendered on the touchscreen; the electronic device can also acquire the mask image corresponding to the object image, which is used to cover the mesh of the edge area of the object image. Furthermore, the electronic device can use the background texture image as the first image layer, the object image as the second image layer, and the mask image as the third image layer, and render the object to be rendered based on the first image layer, the second image layer, and the third image layer.
[0070] It is understandable that, for a masking image, an electronic device can overlay a static circular mask image with an alpha channel onto the texture image of the object to be rendered. This static circular mask image includes edge anti-aliasing of the object, lighting and shadow effects, and atmospheric halo effects. By occluding the edges of the object to be rendered in the second image layer with the static circular mask image layer, the polygonal lines of the object in the second image layer are eliminated, achieving a 3D lighting visual effect. Compared to directly rendering the 3D lighting visual effect of the object to be rendered by the electronic device, the above multi-layer rendering method reduces the power consumption of the electronic device.
[0071] It is understandable that electronic devices can use 2D GPUs to render 3D images through the above rendering methods.
[0072] Figure 4 According to some embodiments of this application, a flowchart of another rendering method is shown. It can be understood that... Figure 4 The processes shown are all executed by electronic devices. For simplicity, the following description... Figure 4 The execution entity will not be described again in the process shown. For example... Figure 4 As shown, including but not limited to the following processes: S401, obtain the interaction status and battery level of the electronic device.
[0073] In some embodiments, the electronic device can obtain the interaction status and battery level of the electronic device. The specific execution process is described in detail in S102 and will not be repeated here.
[0074] S402 determines the rendering mode corresponding to the object to be rendered based on the interaction status and battery level. Different rendering modes contain different numbers of vertices used to render the object.
[0075] In some embodiments, the electronic device can determine the rendering mode corresponding to the object to be rendered based on the interaction state and battery level. Different rendering modes involve different numbers of vertices used to render the object. The specific execution process is described in detail in S102 and will not be repeated here.
[0076] It's understandable that determining different rendering modes based on the varying interaction states and battery levels of electronic devices allows for the categorization of rendered visual effects. This, in turn, reduces the power consumption of electronic devices without compromising the user's visual experience. Furthermore, when the mesh model in low frame rate mode contains fewer vertices than that in high frame rate mode, rendering the object in low frame rate mode can also reduce the power consumption of electronic devices and simultaneously decrease the computational load during rendering, thereby improving the visual quality.
[0077] Figure 5 According to some embodiments of this application, a state machine transition logic diagram for a dynamic rendering mode is shown. It can be understood that... Figure 5 The processes shown are all executed by electronic devices. For simplicity, the following description... Figure 5 The execution entity will not be described again when the process is shown.
[0078] In some embodiments, when the rendering mode of the electronic device is high frame rate mode, and the electronic device detects that the touch operation on the touch screen has stopped for more than 2 seconds (s), or the battery level of the electronic device is less than 20%, the electronic device may switch the rendering mode to low frame rate mode.
[0079] In some embodiments, when the rendering mode of the electronic device is high frame rate mode and the electronic device detects a forced screen-off event, the electronic device can switch the rendering mode to low light mode.
[0080] In some embodiments, when the rendering mode of the electronic device is a low frame rate mode and the electronic device detects that the system has entered a sleep state, the electronic device can switch the rendering mode to a low light mode.
[0081] In some embodiments, when the rendering mode of the electronic device is low frame rate mode, and the electronic device detects a touch on the touchscreen or a wrist lift operation, and the battery level of the electronic device is greater than 20%, the electronic device can switch the rendering mode to high frame rate mode.
[0082] In some embodiments, when the rendering mode of the electronic device is low-light mode and the electronic device detects a system wake-up event, the electronic device can switch the rendering mode to low frame rate mode.
[0083] It's understandable that in high frame rate mode, the electronic device's screen smoothness is 60 FPS, meaning it can render 60 images per second. In high frame rate mode, the electronic device's baseline mesh model is a high-fidelity mesh model, i.e., LOD-High. During image rendering, the electronic device uses a rendering method with all effects enabled. These full effects include edge anti-aliasing, lighting and shadows of the rendered objects, atmospheric halo effects, etc., which are not limited here.
[0084] It's understandable that in low frame rate mode, the smoothness of an electronic device's display is 15-20 FPS, meaning the device can render 15-20 images per second. In low frame rate mode, the electronic device uses a high-fidelity mesh model, i.e., LOD-Low. During image rendering, the electronic device disables certain special effects, such as atmospheric halo effects.
[0085] Understandably, in low-light mode, electronic devices display static images on the touchscreen. These static images include optical flow primitives (OFPS). Based on this, the electronic devices only refresh the time and do not re-render the image.
[0086] In some embodiments, this application also provides a computer-readable medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the rendering method described in the above embodiments. In some embodiments, this application also provides an electronic device, the electronic device comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, one of the processors of the electronic device, for executing the instructions stored in the memory to perform the rendering method in the above embodiments. In some embodiments, this application also provides a program product comprising: instructions that, when executed on an electronic device, cause the electronic device to perform the rendering method described in the above embodiments.
[0087] To facilitate better implementation of the above-described solutions in the embodiments of this application, related equipment for implementing the above-described solutions is also provided below.
[0088] For example, Figure 6 A schematic diagram of the structure of an electronic device 10 is shown according to some embodiments of this application. It is understood that the electronic device 10 can be used as an example of the aforementioned electronic device.
[0089] In some embodiments, electronic device 10 may include one or more processors 1004, system control logic 1008 connected to at least one of the processors 1004, system memory 1012 connected to system control logic 1008, nonvolatile memory (NVM) 1016 connected to system control logic 1008, and network interface 1020 connected to system control logic 1008.
[0090] In some embodiments, processor 1004 may include one or more single-core or multi-core processors. In some embodiments, processor 1004 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where electronic device 10 employs an evolved node B (eNB) or radio access network (RAN) controller, processor 1004 may be configured to perform various corresponding embodiments.
[0091] In some embodiments, system control logic 1008 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1004 and / or any suitable device or component communicating with system control logic 1008.
[0092] In some embodiments, system control logic 1008 may include one or more memory controllers to provide an interface to system memory 1012. System memory 1012 may be used to load and store data and / or instructions. In some embodiments, memory 1012 of system 1000 may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).
[0093] The NVM 1016 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the NVM 1016 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a hard disk drive (HDD), a compact disc (CD) drive, and a digital versatile disc (DVD) drive.
[0094] NVM 1016 may include a portion of the storage resources on the device on which electronic device 10 is installed, or it may be accessible by the device, but is not necessarily part of the device. For example, NVM 1016 may be accessed over a network via network interface 1020.
[0095] Specifically, system memory 1012 and NVM 1016 may respectively include: a temporary copy and a permanent copy of instruction 1024. Instruction 1024 may include: when executed by at least one of processors 1004, causing electronic device 10 to perform, as Figure 5 The instructions for the method shown. In some embodiments, instructions 1024, hardware, firmware and / or their software components may additionally / alternatively be located in system control logic 1008, network interface 1020 and / or processor 1004.
[0096] Network interface 1020 may include a transceiver for providing a radio interface to electronic device 10, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 1020 may be integrated into other components of electronic device 10. For example, network interface 1020 may be integrated into at least one of processor 1004, system memory 1012, NVM 1016, and firmware device (not shown) with instructions, which, when at least one of processor 1004 executes the instructions, enable electronic device 10 to implement, as Figure 1 The method shown. The network interface 1020 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1020 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem. In one embodiment, at least one of the processors 1004 may be encapsulated with logic from one or more controllers for system control logic 1008 to form a system encapsulation session initialization protocol (SIP).
[0097] In one embodiment, at least one of the processors 1004 may be integrated on the same die with the logic of one or more controllers for system control logic 1008 to form a system on chip (SOC). In one embodiment, electronic device 10 may further include: input / output (I / O) device 1032.
[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0099] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
[0100] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.
Claims
1. A rendering method applied to an electronic device, the electronic device including a touchscreen, characterized in that, include: Obtain the interaction status and battery level of the electronic device; Based on the interaction state and the battery level, the rendering mode corresponding to the object to be rendered is determined. Different rendering modes have different numbers of vertices used to render the object.
2. The method according to claim 1, characterized in that, The rendering modes include high frame rate mode, low frame rate mode, and low light mode, and the method further includes: When the interaction state is a swipe operation on the touchscreen and / or the battery level exceeds a first battery threshold, the object to be rendered is rendered in the high frame rate mode. If the time the display screen of the touch screen remains still in the interactive state exceeds a first time threshold, and / or the battery level does not exceed a second battery level threshold, the object to be rendered is rendered in the low frame rate mode. When the interaction state is that the electronic device has enabled the low-light mode and the electronic device is in sleep mode, and / or when the electronic device detects a forced screen-off, the object to be rendered is rendered through the low-light mode; Wherein, the first power threshold is greater than or equal to the second power threshold, the mesh model of the object to be rendered is different for different rendering modes, and the mesh model of the low frame rate mode contains fewer vertices than the mesh model of the high frame rate mode.
3. The method according to claim 2, characterized in that, The method further includes: Obtain the mesh model of the object to be rendered, wherein the object to be rendered includes a central region and an edge region, the mesh model is a non-uniform mesh, the non-uniform mesh includes multiple vertices, and in the non-uniform mesh of the object to be rendered, the number of vertices corresponding to the central region and the edge region are different; The vertices on the non-uniform mesh are pre-stored in the electronic device.
4. The method according to claim 3, characterized in that, The method further includes: The object to be rendered is a sphere.
5. The method according to claim 4, characterized in that, The method further includes: The non-uniform grid is determined based on the angle between the normals on the sphere and the center of the sphere and the grid step size, wherein the grid step size is the difference between the angles between the normals of two adjacent vertices on the sphere and the center of the sphere.
6. The method according to claim 5, characterized in that, The step of determining the non-uniform mesh based on the included angle between the normals on the sphere and its center, and the mesh step size, includes: When 0°≤α≤60°, the grid step size is 10°≤λ≤15°, where α represents the angle between the normals on the sphere and the center of the sphere, and λ represents the grid step size. When 60°<α≤90°, the grid step size is 2°≤λ≤4°.
7. The method according to claim 3, characterized in that, The method further includes: Based on the current time, the first coordinate of the vertex is rotated to obtain the second coordinate, where the first coordinate is the coordinate of the vertex on the mesh model; The second coordinate is projected onto a two-dimensional space to obtain a third coordinate, which can be displayed on the touch screen.
8. The method according to claim 7, characterized in that, The method further includes: The quadrilateral located on the back of the object to be rendered in the mesh model will be determined according to the following formula: Where n represents the normal vector of any quadrilateral in the grid. Indicates the direction of the line of sight. In In this case, the quadrilateral corresponding to the normal vector n is located behind the object to be rendered; Delete the vertices contained in the quadrilateral located behind the object to be rendered in the mesh model to obtain the first vertex set.
9. The method according to claim 8, characterized in that, The method further includes: Obtain the background texture image of the object to be rendered, wherein the background texture image is the background image of the object to be rendered on the touch screen; Obtain the texture image of the object to be rendered, and fill the texture image of the object to be rendered into the first vertex set corresponding to the mesh model to obtain the object image of the object to be rendered on the touch screen; Obtain the mask image corresponding to the object image, the mask image being used to cover the grid of the edge region of the object image; The background texture image is used as the first image layer, the object image is used as the second image layer, and the mask image is used as the third image layer. The object to be rendered is then rendered based on the first image layer, the second image layer, and the third image layer.
10. The method according to claim 9, characterized in that, The method further includes: Based on the first vertex set, a first instruction is generated, which is used to generate the texture image of the object to be rendered; The first instruction includes: The source image refers to the panoramic texture image of the object to be rendered; The source region refers to the region where the first set of vertices is located. The target region refers to the region where the third coordinate of the quadrilateral corresponding to the first vertex set is located; The transformation matrix refers to the matrix that maps the source region to the target region.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 10.
12. An electronic device, characterized in that, include: Memory is used to store instructions executed by one or more processors of an electronic device; And a processor, one of the processors of the electronic device, for executing instructions stored in the memory to implement the method of any one of claims 1 to 10.
13. A program product, characterized in that, The program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 10.