Image rendering method and device and electronic equipment
By determining the target scene area and similar color blocks in image rendering, and optimizing the rendering of these areas using variable rate shading technology, the power consumption increase caused by VRS in the prior art is solved, and a more efficient rendering process and lower power consumption are achieved.
Patent Information
- Application Number
- CN202510083440.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-17
- Publication Date
- 2025-05-16
AI Technical Summary
In existing image rendering methods, power consumption increases through variable rate shading (VRS) technology, especially when scene analysis is required for each pixel point of each image.
By acquiring the image frame and determining the target scene area and similar color blocks, variable-rate shading is used to optimize the rendering of the target color blocks corresponding to similar color blocks in subsequent image frames, reducing the shading accuracy to improve rendering efficiency.
Reduces the calculation amount, reduces the load on electronic devices, effectively reduces power consumption, and maintains high quality of visual effects.
Smart Images
Figure CN120014109A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and specifically relates to an image rendering method, device and electronic device. Background Art
[0002] Nowadays, in the process of image rendering, in order to reduce the power consumption required for rendering, image rendering can be performed through variable rate shading (VRS). In this rendering method, the rendering quality can be dynamically adjusted according to the complexity and importance of the scene. Less important areas can use lower shading rates, while more important or detailed areas use higher shading rates.
[0003] However, image rendering through VRS requires scene analysis for every pixel of every image, which generates a large amount of calculation and causes the power consumption of electronic devices to increase. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide an image rendering method, device and electronic device, which can solve the technical problem of high power consumption in existing image rendering methods.
[0005] In a first aspect, an embodiment of the present application provides a method for rendering an image, the method comprising:
[0006] Acquire a first image frame displayed in a first application;
[0007] Determine a target scene region in the first image frame, wherein the target scene region is a region representing the same scene in the first image frame and the second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold;
[0008] Determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold;
[0009] The target color block of the third image frame is rendered in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to a similar color block in the first image frame.
[0010] In a second aspect, an embodiment of the present application provides an image rendering device, the device comprising:
[0011] An acquisition module, used to acquire a first image frame displayed in a first application;
[0012] A first determination module is used to determine a target scene area in the first image frame, wherein the target scene area is an area representing the same scene in the first image frame and the second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold;
[0013] A second determination module is used to determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold;
[0014] The first rendering module is used to render the target color block of the third image frame in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to the similar color block in the first image frame.
[0015] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method provided in the first aspect are implemented.
[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method provided in the first aspect are implemented.
[0017] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method provided in the first aspect.
[0018] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method provided in the first aspect.
[0019] In the image rendering method, device and electronic device of the present application, the first image frame in the application can be first obtained, and the target scene area with the same scene in the first image frame and the second image frame can be determined. In the target scene area, similar color blocks with small differences in adjacent pixel values are further screened out. Then, the target color blocks corresponding to the similar color blocks in the subsequent image frames are optimized and rendered through variable rate shading. Since the color changes in the target color blocks are small and the visual effect does not require high precision, the rendering efficiency can be improved by reducing the shading precision. The color changes in the area outside the target color blocks are large, and the requirements for retaining details are high. Reducing the precision may cause obvious visual distortion, so it is necessary to maintain high-precision rendering. In this way, variable rate shading rendering can be used only for target color blocks with higher stability. Compared with VRS rendering of the entire picture, the amount of calculation can be reduced, the load of the electronic device can be reduced, and thus the power consumption can be effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a flowchart of an image rendering method provided by an embodiment of the present application;
[0021] Figure 2 is a structural schematic diagram of an image rendering device provided by another embodiment of the present application;
[0022] Figure 3 is a structural schematic diagram of an electronic device provided by another embodiment of the present application;
[0023] Figure 4 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0026] In order to solve the above technical problems, the present application provides an image rendering method. The image rendering method provided by the embodiment of the present application is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.
[0027] like Figure 1 As shown, Figure 1 : is a flowchart of a method for rendering an image provided by an embodiment of the present application. The embodiment of the present application provides a method for rendering an image, which may include:
[0028] S101, acquiring a first image frame displayed in a first application;
[0029] In this embodiment, the first application may be an application that requires real-time image rendering, such as a game application or a video playback application. The first image frame may be any image frame that has been rendered and displayed in the first application. For example, the first application may be a game application, and the first image frame may be any game screen in the game application.
[0030] S102, determining a target scene area in the first image frame, wherein the target scene area is an area representing the same scene in the first image frame and the second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold;
[0031] In this embodiment, the target scene area refers to an area in which the content and features change little or remain unchanged in consecutive image frames, and the target scene area generally includes the background, fixed objects, or areas where elements that do not move significantly are located in the image frames. The second image frame is an image frame that is before the first image frame in the first application, and the interval between the second image frame and the first image frame does not exceed the set first threshold.
[0032] By comparing the contents of the corresponding regions in the first image frame and the second image frame, regions with smaller changes in contents and features in the corresponding regions in the two image frames can be identified, and these regions can be determined as target scene regions.
[0033] In some embodiments, determining the target scene area in the first image frame includes:
[0034] In the case where the scene in the first image frame is a static scene, all areas in the first image frame are determined as the target scene areas.
[0035] In this embodiment, a static scene refers to a scene without obvious changes or movements within a period of time, during which the elements and backgrounds in a plurality of consecutive image frames remain substantially unchanged. Therefore, if the scene in the first image frame is determined to be a static scene, then compared with the second image frame, the content and features of all regions in the first image frame change little or remain unchanged, and therefore, all regions in the first image frame can be determined as target scene regions.
[0036] In the above manner, if the scene in the first image frame is a static scene, then all regions in the image frame can be directly regarded as the target scene region. In this way, complex regional analysis and calculation can be avoided, and the determination process of the target scene region is simplified.
[0037] In some other embodiments, in some embodiments, determining the target scene area in the first image frame includes:
[0038] When the scene in the first image frame is a dynamic scene, traverse each pixel row in the first image frame;
[0039] When every first pixel point in a first pixel row satisfies a consistency condition, the first pixel row is determined as the target scene area, wherein the first pixel row is any pixel row in the first image frame.
[0040] In this embodiment, a dynamic scene refers to a scene in which elements or objects change or move over time, such as displacement, deformation, or other dynamic feature changes or movements of characters, objects, or backgrounds.
[0041] When it is determined that the scene in the first image frame is a dynamic scene, each pixel row in the first image frame can be traversed first. During the traversal process, for any first pixel row in the first image frame, it can be determined whether each first pixel point in the first pixel row meets the consistency condition. The consistency condition can be that the color difference or other visual feature differences between the pixels are small. When each first pixel point in the first pixel row meets the above consistency condition, it can be considered that the first pixel row belongs to the target scene area.
[0042] In this way, when the scene in the first image frame is a dynamic scene, the target scene area in the first image frame is a set of pixel rows.
[0043] In addition, when it is determined that the scene in the first image frame is a dynamic scene, each pixel column in the first image frame can also be traversed, and for any first pixel column, if all pixels in the first pixel column meet the consistency condition, the first pixel column is determined as the target scene area. Then, the target scene area in the first image frame is a set of pixel columns.
[0044] In this embodiment, a stable target scene area can be accurately identified in a dynamic scene, and by accurately identifying the target scene area, the variable rate shading technology can be more efficiently applied during the rendering process to render only the target scene area, thereby optimizing performance and reducing power consumption.
[0045] In some embodiments, when each first pixel point in the first pixel row satisfies the consistency condition, before determining the first pixel row as the target scene area, the method further includes:
[0046] For each first pixel point of the first pixel row in the first image frame, determine N second pixel points in N second image frames whose positions correspond to the first pixel point, where N is a positive integer;
[0047] When the difference between the pixel values of any two pixel points between the second pixel point and the first pixel point is less than a second threshold, it is determined that the first pixel point satisfies the consistency condition.
[0048] In this embodiment, in the process of checking whether each first pixel point in the first pixel row satisfies the consistency condition, for each first pixel point in the first pixel row, N second pixel points corresponding to the position of the first pixel point can be determined in N second image frames. For example, N can be 4, and the N second image frames can be four consecutive second image frames before the first image frame.
[0049] Then, the first pixel point and the N second pixel points can be compared. Specifically, the first pixel point and the N second pixel points can be used as target pixel points, and then two target pixel points are arbitrarily selected to calculate the difference in pixel values of the two target pixel points. If all differences are less than the second threshold, then it can be considered that the color difference or other visual feature difference between the first pixel point and the N second pixel points is small. Then the first pixel point meets the consistency condition. If all the first pixel points of a certain pixel row meet the consistency condition, then the pixel row is determined as the target scene area.
[0050] By using the above method, it is possible to effectively identify a stable area from a dynamic scene, and by comparing the first image frame with multiple second image frames, the accuracy of judging dynamic changes is enhanced. By comparing multiple image frames, misjudgment caused by changes in a single frame can be reduced, thereby more reliably identifying the target scene area.
[0051] S103: Determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold.
[0052] In an embodiment of the present application, after determining the target scene area from the first image frame, similar color blocks can be further determined in the determined target scene area. The similar color blocks refer to areas with similar color values in the first image frame. Specifically, the pixel value differences of adjacent pixels in the similar color blocks are all less than a third threshold, so an area with uniform color and less variation can be formed. For example, the third threshold can be 5 or 10.
[0053] In some embodiments, determining at least one similar color block in the target scene area includes:
[0054] For each third pixel point in the target scene area, determining P fourth pixel points in the target scene area according to the pixel value of the third pixel point, where P is 0 or a positive integer;
[0055] Determine a maximum rectangular area formed by at least two target pixel points as a first area corresponding to the third pixel point, wherein the target pixel points include the third pixel point and the fourth pixel point;
[0056] At least one similar color block is determined according to the first area corresponding to each of the third pixel points.
[0057] In this embodiment, each third pixel point can be processed one by one in the target scene area. For each third pixel point, the system determines P fourth pixel points with similar pixel values by pixel value matching. P is 0 or a positive integer. If P is 0, it means that no fourth pixel point that meets the conditions is found.
[0058] After determining P fourth pixel points, the third pixel point and the matched fourth pixel point can be used as a candidate point set, and as many target pixel points as possible are selected from the target pixel points in the candidate point set to construct a rectangular area with the largest possible area, that is, the maximum rectangular area. The maximum rectangular area can only include the third pixel point or the fourth pixel point.
[0059] Since the maximum rectangular area refers to a rectangular area in the target scene area that is composed of the third pixel and the fourth pixel and only contains these pixels and has the largest possible area, then the maximum rectangular area is a continuous area with similar colors, and similar color blocks can be determined based on the maximum rectangular area.
[0060] Specifically, since each third pixel point in the target scene area may correspond to a first area, there may be multiple first areas in the target scene area, and at least one of the multiple first areas may be screened out as a similar color block.
[0061] In this way, in the target scene area, the target pixels can be screened and the largest rectangular area can be determined based on the consistency of pixel values, and similar color blocks can be further effectively determined. In addition, the extracted similar color blocks provide an accurate basis for subsequent variable rate shading, which helps reduce rendering complexity and power consumption.
[0062] In some embodiments, determining at least one similar color block according to the first area corresponding to each of the third pixels includes:
[0063] For any first region, when the first region does not overlap with other first regions, determining the first region as the similar color block;
[0064] In the case that the first region at least partially overlaps with other first regions, the first region with the largest area among the plurality of first regions at least partially overlapped is determined as the similar color block.
[0065] In this embodiment, since each third pixel point in the target scene area may correspond to a first area, there may be multiple first areas in the target scene area.
[0066] For any first region among the plurality of first regions, if the first region does not overlap with any other first region, it is directly determined as a similar color block. If the first region overlaps with the other first regions at least partially, the areas of the overlapping first regions need to be compared.
[0067] The first region with the largest area may be selected from the plurality of first regions that at least partially overlap and determined as a similar color block, and the first region with the smaller area may be discarded from the plurality of first regions that overlap.
[0068] In this way, the area of the selected similar color blocks can be kept as large as possible while avoiding the existence of repeated or redundant areas in the similar color blocks. Maximizing the range of similar color blocks helps improve the accuracy of subsequent variable rate shading and further reduce rendering complexity and power consumption.
[0069] In some embodiments, determining P fourth pixel points in the target scene area according to the pixel value of the third pixel point includes:
[0070] The following operations are performed in a loop until no fourth pixel point adjacent to the third pixel point is obtained, and the fourth pixel points corresponding to all the third pixel points are determined as the P fourth pixel points:
[0071] Determine a fifth pixel point adjacent to the third pixel point in the first direction, and a sixth pixel point adjacent to the third pixel point in the second direction;
[0072] In the case that there is a pixel point satisfying the pixel stability condition between the fifth pixel point and the sixth pixel point, determining the pixel point satisfying the pixel stability condition as the fourth pixel point;
[0073] The fourth pixel point is updated to the third pixel point.
[0074] In this embodiment, in the process of determining P fourth pixel points in the target scene area based on the third pixel point. For each third pixel point, the fifth pixel point adjacent to the third pixel point in the first direction and the sixth pixel point adjacent to the third pixel point in the second direction can be determined. And determine whether the fifth pixel point and the sixth pixel point meet the pixel stability condition. If the adjacent fifth pixel point or sixth pixel point meets the pixel stability condition, then these pixels will be determined as fourth pixels. Next, these fourth pixels are updated to new third pixel points, and the search and judgment of adjacent pixels for the new third pixel point continues. Among them, the first direction can be the right, the second direction can be the bottom, and the pixel stability condition refers to the condition that the pixel value of the pixel point remains stable or consistent within a certain range.
[0075] The above-mentioned search and judgment process will continue to cycle until no adjacent fourth pixel point that meets the pixel stability condition can be found. At this time, the search operation is stopped, and all the fourth pixel points found are determined as the P fourth pixel points corresponding to the third pixel point.
[0076] In this way, by gradually traversing and expanding adjacent pixels, the set of pixels in the target scene area that meet the pixel stability condition can be efficiently determined. This ensures the integrity of the search, avoids missing edge pixels, and accurately obtains all fourth pixels that meet the conditions.
[0077] In some embodiments, when there is a pixel point satisfying the pixel stability condition between the fifth pixel point and the sixth pixel point, before determining the pixel point satisfying the pixel stability condition as the fourth pixel point, the method further includes:
[0078] Determine the maximum pixel value and the minimum pixel value among the pixel values of all third pixels;
[0079] In the case where the absolute value of the first difference corresponding to the seventh pixel point is less than the third threshold value, and the absolute value of the second difference corresponding to the seventh pixel point is less than the third threshold value, it is determined that the seventh pixel point satisfies the pixel stability condition;
[0080] Among them, the seventh pixel point includes the fifth pixel point and the sixth pixel point, the first difference is the difference between the first pixel value of the seventh pixel point and the maximum pixel value, and the second difference is the difference between the first pixel value of the seventh pixel point and the minimum pixel value.
[0081] In this embodiment, in the above-mentioned search and judgment cycle, new third pixel points can be continuously obtained. Then, when judging whether a seventh pixel point meets the pixel stability condition, the maximum pixel value and the minimum pixel value of all the third pixel points can be first calculated.
[0082] Then, for the seventh pixel point adjacent to the third pixel point, a first difference between the pixel value of the seventh pixel point and the maximum pixel value, and a second difference between the pixel value of the seventh pixel point and the minimum pixel value may be calculated respectively.
[0083] If the absolute value of the first difference and the absolute value of the second difference are both smaller than the preset third threshold, it can be considered that the pixel value of the seventh pixel is within a stable range, and it can be considered that the seventh pixel satisfies the pixel stability condition, that is, the seventh pixel is determined as the fourth pixel.
[0084] If the absolute value of the first difference or the absolute value of the second difference is greater than or equal to the preset third threshold, it is considered that the seventh pixel point has a large change and does not meet the pixel stability condition.
[0085] For example, the third threshold value can be 5, the first direction is the right, and the second direction is the bottom. First, the variable i=0 can be initialized, the set of all pixels in the target scene area is defined as {P}, and the set of processed pixels is determined as {P'}, which represents the set of pixels that need to be processed. Then, a third pixel point A can be selected from the set {P}, and then it is checked whether the third pixel point A exists in the set {P'}. If A exists in {P'}, it is considered that the third pixel point A has been processed, and the third pixel point A is ignored; if A does not exist in {P'}, a set {Q} is initialized, and the third pixel point A is saved in the set {P'}, the set {S(i)} and the set {Q}. Among them, the set {Q} is used to record all areas connected to the third pixel point A, and {S(i)} is the set of the third pixel point.
[0086] The fifth pixel point B to the right of the third pixel point A and the sixth pixel point C below can be checked one by one, that is, the maximum pixel value and the minimum pixel value in the set {S(i)} are determined, and then the pixel value of the fifth pixel point B and the pixel value of the sixth pixel point C are compared with the maximum pixel value and the minimum pixel value respectively. If the absolute value of the difference between the pixel value of the fifth pixel point B and the maximum pixel value is less than 5, and the absolute value of the difference between the pixel value of the fifth pixel point B and the minimum pixel value is also less than 5, then it is considered that the fifth pixel value B meets the pixel stability condition, and the fifth pixel value B is added to the set {S(i)} of the third pixel point. Otherwise, the fifth pixel value is ignored. In addition, the sixth pixel point C can be processed according to the method of processing the fifth pixel value B.
[0087] After completing the above processing, it is possible to check whether there are pixels other than the third pixel point A in the set {S(i)}. If not, it means that the current area has been detected. Otherwise, pixels that have not been processed can be selected from {S(i)} to continue expanding the area.
[0088] When a region detection is completed, the rectangular region corresponding to the third pixel point A can be determined based on all the pixels in the set {S(i)}, and the coordinates of the four corners of the rectangular region are recorded in the set {T(i)}, and then the set {T(i)} is checked for overlapping rectangular regions. If there are overlapping regions, the largest rectangular region is retained and the smaller rectangular regions are deleted.
[0089] Then, it can be determined whether the set {P'} is equal to the set {P}. If the two are equal, it means that all pixels in the target scene area have been traversed and the algorithm terminates. If not, the unprocessed pixels continue to be processed.
[0090] Through the multiple difference calculations in the above manner, pixels with large pixel value changes can be excluded, ensuring that the selected fourth pixel has highly consistent pixel stability with the third pixel.
[0091] S104, rendering the target color block of the third image frame in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to the similar color block in the first image frame.
[0092] In the embodiment of the present application, rendering in a variable rate shading (VRS rendering) manner is a graphics rendering technology that can improve rendering efficiency by using different shading precisions in different areas of the image. In particular, the amount of shading calculations is reduced in areas with low visual importance or small changes, while maintaining high-precision rendering in key areas, thereby improving performance and reducing power consumption while ensuring image quality.
[0093] After determining the similar color blocks in the first image frame, VRS rendering can be performed on the target color blocks corresponding to the positions of the similar color blocks in the first image frame in the third image frame during the rendering of the third image frame after the first image frame. Furthermore, non-target color blocks outside the target area in the third image frame are rendered with high precision in a fixed rate shading manner to obtain and display the target image frame.
[0094] In some embodiments, after determining at least one similar color block in the target scene area, the method further includes:
[0095] The non-target color blocks of the third image frame are rendered in a fixed rate shading manner to obtain a target image frame, wherein the non-target color blocks are display areas outside the target color blocks in the third image frame.
[0096] In this embodiment, although the rendering quality of VRS rendering can be dynamically adjusted according to the complexity and importance of the scene, a lower shading rate is used to render less important areas, and a higher shading rate is used to render more important or detailed areas. However, image rendering by VRS requires scene analysis for every pixel of every image, resulting in a large amount of calculation.
[0097] In the above method, the third image frame to be rendered can first be divided into target color blocks and non-target color blocks. Since the positions of the target color blocks and the similar color blocks in the first image frame correspond to each other, it can be considered that the color changes in the target color blocks are small, while the color changes in the non-target color blocks are large.
[0098] Well, since the color changes in the target color block are small and the visual effect does not require high precision, the rendering efficiency can be improved by reducing the shading precision, so VRS rendering can be performed on it. However, the color changes of non-target color blocks with large changes are large, and the requirements for retaining details are high. Reducing the precision may cause obvious visual distortion, thereby affecting the image quality and user experience of the application. Therefore, for non-target color blocks with large dynamic changes, it is necessary to maintain high-precision rendering to ensure the accurate presentation of details. Therefore, the non-target color blocks of the third image frame can be rendered with high precision and at a fixed rate.
[0099] In an embodiment of the present application, the first image frame in the application can be first obtained, and the target scene area with the same scene in the first image frame and the second image frame can be determined. In the target scene area, similar color blocks with small differences in adjacent pixel values are further screened out. Then, the target color blocks corresponding to the similar color blocks in the subsequent image frames are optimized and rendered through variable rate shading. Since the color changes in the target color blocks are small and the visual effect does not require high precision, the rendering efficiency can be improved by reducing the shading precision. The color changes in the area outside the target color blocks are large, and the requirements for retaining details are high. Reducing the precision may cause obvious visual distortion, so it is necessary to maintain high-precision rendering. In this way, variable rate shading rendering can be used only for target color blocks with higher stability. Compared with VRS rendering of the entire picture, the amount of calculation can be reduced, the load on the electronic device can be reduced, and thus the power consumption can be effectively reduced.
[0100] In addition, as an optional embodiment, the image rendering method is applied to an electronic device, which includes a central processing unit (CPU) and a GPU (Graphics Processing Unit). The application can send the rendered image frame to the intermediate layer for dynamic and static scene analysis and target pixel calculation to obtain similar color blocks. Then, the coordinates of the similar color blocks and the shading rate can be set as VRS parameters. For example, the VRS parameters can include the coordinate values of the four vertices of the largest rectangular area (x1, y1), (x2, y2), (x3, y3), (x4, y4), and a shading rate of 2*2.
[0101] The CPU can pass the above VRS parameters to the GPU, and the GPU reads the shading rate of each pixel or block, shades each pixel in the image frame according to the shading rate, and outputs the processed target image frame.
[0102] In addition, as another optional embodiment, the call of the VRS rendering method can also be controlled by a timer. The timer is used to trigger subsequent operations within a set time interval. Specifically, a timer can be set and the timer duration can be set, which can be 30 seconds or one minute.
[0103] Once the timer reaches the timing duration and overflows, the above-mentioned image rendering method is triggered, that is, similar color blocks are determined from the first image frame, and the target color blocks in the image frames subsequent to the first image frame are rendered according to the VRS rendering method.
[0104] In this way, the system can use the timer mechanism to help prevent the rendering method from being unable to adjust in time when the scene changes, ensuring that the rendering method of the above-mentioned image is always valid during the rendering process.
[0105] Figure 2 is a schematic diagram of the structure of an image rendering device provided by another embodiment of the present application, such as Figure 2 As shown, the image rendering device may include:
[0106] An acquisition module 201 is used to acquire a first image frame displayed in a first application;
[0107] A first determining module 202 is used to determine a target scene area in the first image frame, wherein the target scene area is an area representing the same scene in the first image frame and the second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold;
[0108] A second determination module 203 is used to determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold;
[0109] The first rendering module 204 is used to render the target color block of the third image frame in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to the similar color block in the first image frame.
[0110] In the present application, the first image frame in the application can be first obtained, and the target scene area with the same scene in the first image frame and the second image frame can be determined. In the target scene area, similar color blocks with small differences in adjacent pixel values are further screened out. Then, the target color blocks corresponding to the similar color blocks in the subsequent image frames are optimized and rendered through variable rate shading. Since the color changes in the target color blocks are small and the visual effect does not require high precision, the rendering efficiency can be improved by reducing the shading precision. The color changes in the area outside the target color blocks are large, and the requirements for retaining details are high. Reducing the precision may cause obvious visual distortion, so it is necessary to maintain high-precision rendering. In this way, variable rate shading rendering can be used only for target color blocks with higher stability. Compared with VRS rendering of the entire picture, the amount of calculation can be reduced, the load on the electronic device can be reduced, and thus the power consumption can be effectively reduced.
[0111] In another optional example, the first determining module 202 includes:
[0112] a traversal unit, configured to traverse each pixel row in the first image frame when the scene in the first image frame is a dynamic scene;
[0113] A first determination unit is configured to determine a first pixel row as the target scene area if every first pixel point in the first pixel row satisfies a consistency condition, wherein the first pixel row is any pixel row in the first image frame.
[0114] In another optional example, the image rendering device further includes:
[0115] A third determination module is used to determine, for each first pixel point of the first pixel row in the first image frame, N second pixel points in N second image frames whose positions correspond to the first pixel point, where N is a positive integer;
[0116] The fourth determination module is used to determine that the first pixel point satisfies the consistency condition when the difference between the pixel values of any two pixel points between the second pixel point and the first pixel point is less than a second threshold.
[0117] In another optional example, the first determining module 202 includes:
[0118] The second determining unit is configured to determine all regions in the first image frame as the target scene regions when the scene in the first image frame is a static scene.
[0119] In another optional example, the second determining module 203 includes:
[0120] A third determining unit, configured to determine, for each third pixel point in the target scene area, P fourth pixel points in the target scene area according to the pixel value of the third pixel point, where P is 0 or a positive integer;
[0121] a fourth determining unit, configured to determine a maximum rectangular area formed by at least two target pixel points as a first area corresponding to the third pixel point, wherein the target pixel points include the third pixel point and the fourth pixel point;
[0122] The fifth determining unit is used to determine at least one of the similar color blocks according to the first area corresponding to each of the third pixel points.
[0123] In another optional example, the fifth determining unit is specifically configured to:
[0124] For any first region, when the first region does not overlap with other first regions, determining the first region as the similar color block;
[0125] In the case that the first region at least partially overlaps with other first regions, the first region with the largest area among the plurality of first regions at least partially overlapped is determined as the similar color block.
[0126] In another optional example, the third determining unit is specifically configured to:
[0127] The following operations are performed in a loop until no fourth pixel point adjacent to the third pixel point is obtained, and the fourth pixel points corresponding to all the third pixel points are determined as the P fourth pixel points:
[0128] Determine a fifth pixel point adjacent to the third pixel point in the first direction, and a sixth pixel point adjacent to the third pixel point in the second direction;
[0129] In the case that there is a pixel point satisfying the pixel stability condition between the fifth pixel point and the sixth pixel point, determining the pixel point satisfying the pixel stability condition as the fourth pixel point;
[0130] The fourth pixel point is updated to the third pixel point.
[0131] In another optional example, the image rendering device further includes:
[0132] A fifth determining module, used to determine a maximum pixel value and a minimum pixel value among the pixel values of all third pixel points;
[0133] a sixth determining unit, configured to determine that the seventh pixel satisfies the pixel stability condition when an absolute value of a first difference corresponding to the seventh pixel is less than a third threshold and an absolute value of a second difference corresponding to the seventh pixel is less than the third threshold;
[0134] Among them, the seventh pixel point includes the fifth pixel point and the sixth pixel point, the first difference is the difference between the first pixel value of the seventh pixel point and the maximum pixel value, and the second difference is the difference between the first pixel value of the seventh pixel point and the minimum pixel value.
[0135] In another optional example, the image rendering device further includes:
[0136] The second rendering module is used to render the non-target color block of the third image frame in a fixed rate shading manner to obtain a target image frame, wherein the non-target color block is a display area outside the target color block in the third image frame.
[0137] The rendering device of the image in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.
[0138] The image rendering device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0139] The image rendering device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0140] Alternatively, if Figure 3 As shown, an embodiment of the present application also provides an electronic device 100, including a processor 110, a memory 119, and a program or instruction stored in the memory 119 and executable on the processor 110. When the program or instruction is executed by the processor 110, each process of the above-mentioned image rendering method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0141] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
[0142] Please refer to Figure 4, Figure 4 The hardware structure diagram of an electronic device for implementing an embodiment of the present application is shown in FIG. The electronic device 100 includes, but is not limited to, a radio frequency unit 121, a network module 122, an audio output unit 123, an input unit 124, a sensor 125, a display unit 126, a user input unit 127, an interface unit 128, a memory 129, and a processor 120.
[0143] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 120 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0144] The processor 120 is used to obtain a first image frame displayed in the first application;
[0145] The processor 120 is configured to determine a target scene region in the first image frame, wherein the target scene region is a region representing the same scene in the first image frame and the second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold;
[0146] The processor 120 is configured to determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold;
[0147] Processor 120 is used to render the target color block of the third image frame in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to the similar color block in the first image frame.
[0148] In the present application, the first image frame in the application can be first obtained, and the target scene area with the same scene in the first image frame and the second image frame can be determined. In the target scene area, similar color blocks with small differences in adjacent pixel values are further screened out. Then, the target color blocks corresponding to the similar color blocks in the subsequent image frames are optimized and rendered through variable rate shading. Since the color changes in the target color blocks are small and the visual effect does not require high precision, the rendering efficiency can be improved by reducing the shading precision. The color changes in the area outside the target color blocks are large, and the requirements for retaining details are high. Reducing the precision may cause obvious visual distortion, so it is necessary to maintain high-precision rendering. In this way, variable rate shading rendering can be used only for target color blocks with higher stability. Compared with VRS rendering of the entire picture, the amount of calculation can be reduced, the load on the electronic device can be reduced, and thus the power consumption can be effectively reduced.
[0149] In another optional example, the processor 120 is further configured to:
[0150] When the scene in the first image frame is a dynamic scene, traverse each pixel row in the first image frame;
[0151] When every first pixel point in a first pixel row satisfies a consistency condition, the first pixel row is determined as the target scene area, wherein the first pixel row is any pixel row in the first image frame.
[0152] In another optional example, the processor 120 is further configured to:
[0153] For each first pixel point of the first pixel row in the first image frame, determine N second pixel points in N second image frames whose positions correspond to the first pixel point, where N is a positive integer;
[0154] When the difference between the pixel values of any two pixel points between the second pixel point and the first pixel point is less than a second threshold, it is determined that the first pixel point satisfies the consistency condition.
[0155] In another optional example, the processor 120 is further configured to:
[0156] In the case where the scene in the first image frame is a static scene, all areas in the first image frame are determined as the target scene areas.
[0157] In another optional example, the processor 120 is further configured to:
[0158] For each third pixel point in the target scene area, determining P fourth pixel points in the target scene area according to the pixel value of the third pixel point, where P is 0 or a positive integer;
[0159] Determine a maximum rectangular area formed by at least two target pixel points as a first area corresponding to the third pixel point, wherein the target pixel points include the third pixel point and the fourth pixel point;
[0160] At least one similar color block is determined according to the first area corresponding to each of the third pixel points.
[0161] In another optional example, the processor 120 is further configured to:
[0162] For any first region, when the first region does not overlap with other first regions, determining the first region as the similar color block;
[0163] In the case that the first region at least partially overlaps with other first regions, the first region with the largest area among the plurality of first regions at least partially overlapped is determined as the similar color block.
[0164] In another optional example, the processor 120 is further configured to:
[0165] The following operations are performed in a loop until no fourth pixel point adjacent to the third pixel point is obtained, and the fourth pixel points corresponding to all the third pixel points are determined as the P fourth pixel points:
[0166] Determine a fifth pixel point adjacent to the third pixel point in the first direction, and a sixth pixel point adjacent to the third pixel point in the second direction;
[0167] In the case that there is a pixel point satisfying the pixel stability condition between the fifth pixel point and the sixth pixel point, determining the pixel point satisfying the pixel stability condition as the fourth pixel point;
[0168] The fourth pixel point is updated to the third pixel point.
[0169] In another optional example, the processor 120 is further configured to:
[0170] Determine the maximum pixel value and the minimum pixel value among the pixel values of all third pixels;
[0171] In the case where the absolute value of the first difference corresponding to the seventh pixel point is less than the third threshold value, and the absolute value of the second difference corresponding to the seventh pixel point is less than the third threshold value, it is determined that the seventh pixel point satisfies the pixel stability condition;
[0172] Among them, the seventh pixel point includes the fifth pixel point and the sixth pixel point, the first difference is the difference between the first pixel value of the seventh pixel point and the maximum pixel value, and the second difference is the difference between the first pixel value of the seventh pixel point and the minimum pixel value.
[0173] In another optional example, the processor 120 is further configured to:
[0174] The non-target color blocks of the third image frame are rendered in a fixed rate shading manner to obtain a target image frame, wherein the non-target color blocks are display areas outside the target color blocks in the third image frame.
[0175] It should be understood that in the embodiment of the present application, the input unit 124 may include a graphics processor (Graphics Processing Unit, GPU) 1241 and a microphone 1242, and the graphics processor 1241 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 126 may include a display panel 1261, and the display panel 1261 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 127 includes a touch panel 1271 and at least one of other input devices 1272. The touch panel 1271 is also called a touch screen. The touch panel 1271 may include two parts: a touch detection device and a touch controller. Other input devices 1272 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0176] The memory 129 can be used to store software programs and various data. The memory 129 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 129 may include a volatile memory or a non-volatile memory, or the memory 129 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 129 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0177] The processor 120 may include one or more processing units; optionally, the processor 120 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 120.
[0178] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image rendering method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0179] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0180] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image rendering method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0181] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0182] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image rendering method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0183] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0185] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image rendering method, characterized in that: include: Acquire a first image frame displayed in a first application; Determine a target scene region in the first image frame, wherein the target scene region is a region representing the same scene in the first image frame and the second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold; Determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold; The target color block of the third image frame is rendered in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to a similar color block in the first image frame.
2. The method according to claim 1, characterized in that The determining of the target scene area in the first image frame includes: When the scene in the first image frame is a dynamic scene, traverse each pixel row in the first image frame; When every first pixel point in a first pixel row satisfies a consistency condition, the first pixel row is determined as the target scene area, wherein the first pixel row is any pixel row in the first image frame.
3. The method according to claim 2, characterized in that In the case where each first pixel point in the first pixel row satisfies the consistency condition, before determining the first pixel row as the target scene area, the method further includes: For each first pixel point of the first pixel row in the first image frame, determine N second pixel points in N second image frames whose positions correspond to the first pixel point, where N is a positive integer; When the difference between the pixel values of any two pixel points between the second pixel point and the first pixel point is less than a second threshold, it is determined that the first pixel point satisfies the consistency condition.
4. The method according to claim 1, characterized in that: The determining of the target scene area in the first image frame includes: In the case where the scene in the first image frame is a static scene, all areas in the first image frame are determined as the target scene areas.
5. The method according to claim 1, characterized in that Determining at least one similar color block in the target scene area includes: For each third pixel point in the target scene area, determining P fourth pixel points in the target scene area according to the pixel value of the third pixel point, where P is 0 or a positive integer; Determine a maximum rectangular area formed by at least two target pixel points as a first area corresponding to the third pixel point, wherein the target pixel points include the third pixel point and the fourth pixel point; At least one similar color block is determined according to the first area corresponding to each of the third pixel points.
6. The method according to claim 5, characterized in that The determining at least one similar color block according to the first area corresponding to each of the third pixel points includes: For any first region, when the first region does not overlap with other first regions, determining the first region as the similar color block; In the case that the first region at least partially overlaps with other first regions, the first region with the largest area among the plurality of first regions at least partially overlapped is determined as the similar color block.
7. The method according to claim 5, characterized in that The step of determining P fourth pixel points in the target scene area according to the pixel value of the third pixel point comprises: The following operations are performed in a loop until no fourth pixel point adjacent to the third pixel point is obtained, and the fourth pixel points corresponding to all the third pixel points are determined as the P fourth pixel points: Determine a fifth pixel point adjacent to the third pixel point in the first direction, and a sixth pixel point adjacent to the third pixel point in the second direction; In the case that there is a pixel point satisfying the pixel stability condition between the fifth pixel point and the sixth pixel point, determining the pixel point satisfying the pixel stability condition as the fourth pixel point; The fourth pixel point is updated to the third pixel point.
8. The method according to claim 7, characterized in that In the case where there is a pixel point satisfying the pixel stability condition between the fifth pixel point and the sixth pixel point, before determining the pixel point satisfying the pixel stability condition as the fourth pixel point, the method further includes: Determine the maximum pixel value and the minimum pixel value among the pixel values of all third pixels; In the case where the absolute value of the first difference corresponding to the seventh pixel point is less than the third threshold, and the absolute value of the second difference corresponding to the seventh pixel point is less than the third threshold, it is determined that the seventh pixel point satisfies the pixel stability condition; Among them, the seventh pixel point includes the fifth pixel point and the sixth pixel point, the first difference is the difference between the first pixel value of the seventh pixel point and the maximum pixel value, and the second difference is the difference between the first pixel value of the seventh pixel point and the minimum pixel value.
9. The method according to any one of claims 1 to 8, characterized in that: After determining at least one similar color block in the target scene area, the method further includes: The non-target color blocks of the third image frame are rendered in a fixed rate shading manner to obtain a target image frame, wherein the non-target color blocks are display areas outside the target color blocks in the third image frame.
10. An image rendering device, characterized in that: include: An acquisition module, used to acquire a first image frame displayed in a first application; A first determination module is configured to determine a target scene region in the first image frame, wherein the target scene region is a region representing the same scene in the first image frame and a second image frame, the second image frame is an image frame before the first image frame in the first application, and the number of image frames between the second image frame and the first image frame is less than a first threshold; A second determination module is used to determine at least one similar color block in the target scene area, wherein a difference in pixel values of adjacent pixels in the similar color block is less than a third threshold; The first rendering module is used to render the target color block of the third image frame in a variable rate shading manner, wherein the third image frame is an image frame after the first image frame in the first application, and the position of the target color block in the third image frame corresponds to the similar color block in the first image frame.
11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image rendering method according to any one of claims 1 to 9 are implemented.