System and method for power management by regulating display refresh rate

Abstract

An artificial intelligence (AI) based method for power management by regulating a display refresh rate of a user equipment (UE). The method includes obtaining velocity vectors associated with a content displayed with a first display refresh rate. Further, the method includes estimating a jitter level perceived by a user related to the displayed content observed by a user. Further, the method includes determining a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. Furthermore, the method includes applying the second display refresh rate instead of the first display refresh rate for the content displayed on the UE thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

Description

SYSTEM AND METHOD FOR POWER MANAGEMENT BY REGULATING DISPLAY REFRESH RATE

[0001] The present disclosure generally relates to power management, and more particularly relates to a system and a method for power management by regulating a display refresh rate in a user equipment (UE).

[0002] In the modern landscape of user equipment (UE) such as smartphones, tablets, and other electronic devices, efficient power management is a critical aspect of device design and functionality. A significant portion of the UE power consumption is attributed to its display, particularly the display refresh rate of the UE. The display refresh rate is the frequency at which the display updates its content per second and is measured in Hertz (Hz). In the UE, managing the refresh rate optimally may significantly influence the power consumption and battery life of the UE. However, current technologies exhibit several technical shortcomings in the UE for regulating the display refresh rate.

[0003] According to one embodiment of the present disclosure, a method for power management by regulating a display refresh rate of a user equipment (UE) is disclosed. The method may include obtaining velocity vectors associated with a content displayed on a display of the UE with a first display refresh rate. The method includes estimating a jitter level perceived by a user related to the displayed content by measuring variations in the displayed content resulting in an unstable appearance of the displayed content. The method may include determining a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. The method may include applying the second display refresh rate instead of the first display refresh rate for the content displayed on the UE thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

[0004] According to one embodiment of the present disclosure, a system for power management by regulating a display refresh rate of a user equipment is disclosed. The system may include a memory and at least one processor in communication with the memory. The at least one processor may be configured to obtain velocity vectors associated with a content displayed on a display of the UE with a first display refresh rate. The at least one processor may be configured to estimate a jitter level perceived by a user related to the displayed content by measuring variations in the displayed content, resulting in an unstable appearance of the displayed content. The at least one processor may be configured to determine a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. The at least one processor may be configured to apply the second display refresh rate instead of the first display refresh rate for the content displayed on the UE thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

[0005] According to one embodiment of the present disclosure, a method for power management by regulating a display refresh rate of a user equipment (UE) is disclosed. The method includes generating velocity vectors associated with a content displayed on a display of the UE with a first display refresh rate. Further, the method includes estimating a jitter level in the displayed content by measuring variations in the displayed content, resulting in an unstable appearance displayed content. Furthermore, the method includes determining a second display refresh rate lesser than a predefined threshold (T) wherein the predefined threshold (T) indicates a predefined limit beyond which a decrease in the display refresh rate would lead to a loss in quality of the content displayed, such that the jitter level observed by the user remains unchanged. Furthermore, the method includes applying the second display refresh rate for the content displayed on the UE thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

[0006] According to one embodiment of the present disclosure, a system for power management by regulating a display refresh rate of a user equipment (UE) is disclosed. The system includes a memory and at least one processor in communication with the memory. The at least one processor is configured to generate velocity vectors associated with a content displayed on a display of the UE with a first display refresh rate. Further, the at least one processor is configured to estimate a jitter level in the displayed content by measuring variations in the displayed content, thereby resulting in an unstable appearance of the displayed content. Furthermore, the at least one processor is configured to determine a second display refresh rate lesser than a predefined threshold (T), wherein the predefined threshold (T) indicates a predefined limit beyond which a decrease in the display refresh rate would lead to a loss in quality of the content displayed, such that the jitter level observed by the user remains unchanged. Furthermore, the at least one processor is configured to apply the second display refresh rate for the content displayed on the UE thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

[0007] To further clarify the advantages and features of the present disclosure, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which are illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail in the accompanying drawings.

[0008] These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0009] Figure 1 illustrates a schematic block diagram depicting an environment for the implementation of a system for power management by regulating a display refresh rate of a user equipment (UE), according to an embodiment of the present disclosure;

[0010] Figure 2a illustrates a schematic block diagram of modules components of the system for power management by regulating the display refresh rate of the UE, according to an embodiment of the present disclosure;

[0011] Figure 2b illustrates a schematic process flow for power management by regulating the display refresh rate of the UE via the modules of the system, according to an embodiment of the present disclosure;

[0012] Figure 3 illustrates a block diagram associated with a generating module of the system, according to an embodiment of the present disclosure;

[0013] Figure 4a illustrates an exemplary scrolling scenario on two identical UE's for the jitter level, according to an embodiment of the present disclosure;

[0014] Figure 4b illustrates an exemplary scrolling scenario on two identical UE's for the jitter level, according to an embodiment of the present disclosure;

[0015] Figure 5 illustrates a process flow associated with a determining module of the system, according to an embodiment of the present disclosure;

[0016] Figure 6 illustrates an exemplary block diagram of the determining module of the system, according to an embodiment of the present disclosure;

[0017] Figure 7 illustrates a process flow associated with an implementing module of the system, according to an embodiment of the present disclosure;

[0018] Figure 8a illustrates an exemplary block diagram of the implementing module of the system, according to an embodiment of the present disclosure;

[0019] Figure 8b illustrates an exemplary process flow of the implementing module to be modelled as a temporal update scheduler, according to an embodiment of the present disclosure;

[0020] Figure 9 illustrates an exemplary process flow comprising a method for power management by regulating the display refresh rate of the UE, according to an embodiment of the present disclosure;

[0021] Figure 10 illustrates another flowchart depicting a method at run-time for power management by regulating the display refresh rate of the UE, according to an embodiment of the present disclosure; and

[0022] Figure 11 illustrates an exemplary use-case scenario for the display refresh rate dynamically adjusting to content velocity changes, according to an embodiment of the present disclosure.

[0023] Further, skilled artisans will appreciate that elements in the drawings are illustrated for simplicity and may not have necessarily been drawn to scale. For example, the flow charts illustrate the method in terms of the most prominent steps involved to help to improve understanding of aspects of the present disclosure. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0024] For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the various embodiments and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated system, and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur to one skilled in the art to which the invention relates.

[0025] It will be understood by those skilled in the art that the foregoing general description and the following detailed description are explanatory of the invention and are not intended to be restrictive thereof.

[0026] Reference throughout this specification to "an aspect," "another aspect" or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrase "in an embodiment," "in another embodiment" and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0027] The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or sub-systems or elements or structures or components proceeded by "comprises... a" does not, without more constraints, preclude the existence of other devices or other sub-systems or other elements or other structures or other components or additional devices or additional sub-systems or additional elements or additional structures or additional components.

[0028] Throughout the disclosure, the expression "at least one of a, b or c" indicates "a", "b", "c", "a and b", "a and c", "b and c", "a, b, and c", or variations thereof.

[0029] It should be understood that blocks in each flowchart and combinations of flowcharts may be performed by one or more computer programs including computer-executable instructions. The one or more computer programs may all be stored in a single memory or may be divided and stored in different memories.

[0030] It is to be understood that the singular forms, e.g., "a", "an", and "the", include the plural forms as well, unless the context clearly indicates otherwise. Thus, for example, the term "a component surface" may also include one or more of such surfaces.

[0031] Any function or operation described in the present document may be performed by a single processor or a combination of processors. The single processor or the combination of processors may include circuitry that performs processing, such as an application processor (AP), a communication processor (CP), a graphical processing unit (GPU), a neural processing unit (NPU), a microprocessor unit (MPU), a system on chip (SoC), or an integrated chip (IC).

[0032] The default setting for many UEs with adaptive refresh rates is to operate at the highest possible display refresh rate. This is intended to ensure that all content, regardless of its nature, is displayed smoothly. However, operating at the highest display refresh rate is inherently inefficient because it does not account for the varying demands of different types of content. Instead, it relies on a fixed policy to change the display refresh rate, which lacks the flexibility to adjust the display refresh rate dynamically based on the real-time content analysis being played on the UE.

[0033] One of the major technical problems in the commercial technologies is the failure to differentiate between various types of content. The content displayed on the UE may vary widely, from high-motion video games to static text or images. Thus, each type of content may have different requirements for refresh rates. For instance, full pixel movement content includes high-speed video or gaming scenes where the entire screen's content on the UE changes rapidly. Thus, a high refresh rate is crucial to avoid motion blur and ensure smooth playback. For another instance, partial pixel movement content such as scrolling through web pages or social media feeds, where only portions of the screen of the UE are updated at a time. Thus, such scenarios may not necessarily require the highest refresh rate. In another instance, 3D content movement involving depth and perspective changes, such as augmented reality (AR) or virtual reality (VR) applications, may require specific refresh rate adjustments to maintain a seamless experience. The existing technologies do not account for these variations, leading to suboptimal refresh rate settings that either waste power or degrade the user experience.

[0034] Furthermore, the higher display refresh rates consume more power because the display has to update more frequently. When the UE maintains the higher display refresh rate irrespective of the content type, it results in unnecessary power consumption. This is particularly problematic for battery operated UEs, where power efficiency is paramount. The inability to adjust the display refresh rates dynamically means that the UE's battery life is often shorter than it could be if more sophisticated refresh rate management were implemented.

[0035] Furthermore, the commercial technologies fail to balance the need for low jitter levels with power-saving measures. The higher display refresh rates in the UE with the existing technologies are maintained to minimize jitter levels, but such techniques come at the cost of increased power consumption. Conversely, lowering the display refresh rate without considering its impact on the jitter level may lead to choppy or unstable visuals, detracting from the user experience.

[0036] Furthermore, another significant technical problem in the commercial technologies is the lack of real-time adjustment capabilities. The commercial technologies do not dynamically compute and adjust to the optimal refresh rate at specific time instances based on the content being displayed. Instead, the existing technologies rely on predetermined settings that do not adapt to changes in content motion. Such a static approach is inefficient and does not cater to the dynamic nature of user interactions and content variations.

[0037] The technical challenges in managing display refresh rates for user equipment are multi-faceted and significant. Hence, there is a need for a new solution for optimizing the display refresh rate by modelling the perception of content type.

[0038] Figure 1 illustrates a schematic block diagram depicting an environment for the implementation of a system 100 for power management by regulating a display refresh rate of a user equipment (UE) 102, according to an embodiment of the present disclosure.

[0039] In an embodiment, referring to Figure 1, the system 100 may be implemented in the UE 102 as an application installed in the UE 102 and running on an operating system (OS) of the UE 102 that generally defines a first active user environment. The OS typically presents or displays the application through a graphical user interface ("GUI") of the OS. In a non-limiting example, the UE 102 may be a laptop computer, a desktop computer, a Personal Computer (PC), a notebook, a smartphone, a tablet, a smartwatch, or any device capable of displaying electronic media.

[0040] A user may interact with the UE 102, generating continuous interaction information during the interaction. For instance, the user may touch a display or a screen of the UE 102 at various coordinates, such as by swiping in different directions at specific time instances. This interaction information can be generated through gestures, gyroscope readings, and accelerometer readings. The system 100 may be configured to provide an output by regulating a display refresh rate of the UE 102 based on the user interaction such that power consumption may be managed.

[0041] In an example scenario, the user may be engaged in a high-intensity action game on the UE 102. As the user navigates through the action game, the user may frequently swipe and tap on the screen of the UE 102 to control the character's movements and actions. The game may involve rapid movements in various directions such as vertical jumps, horizontal runs, and diagonal dodges. The system 100 analyzes the user's interactions, such as swiping gestures and taps, as well as data from sensors of the UE 102 like the gyroscope and accelerometer. Additionally, a slow-motion camera associated with the UE 102 may record the movements within the game to train an artificial intelligence (AI) model to generate velocity data. For instance, when the user swipes left to dodge an obstacle in the game, the system 100 may measure the speed and direction of the swipe and the resulting character movement on the screen.

[0042] Consequently, the system 100 using the AI model may learn velocity vectors for sets of pixels during these interactions. For example, when the user's character moves rapidly across the screen of the UE 102, the pixels change quickly. Thus, the system 100, using neuromorphic sensing, maps the changes in the sets of pixels to an optimal display refresh rate. Therefore, during intense game scenes, the display refresh rate may be increased to ensure smooth motion and visual clarity, while during slower moments, the display refresh rate of the UE 102 may be reduced to save power.

[0043] Furthermore, in the example, the system 100 may include the AI models continuously monitoring and predicting the optimal time and interval to switch the display refresh rate to manage power consumption without compromising the gaming experience. For instance, if the user pauses the game or enters a menu screen, the AI model of the system 100 may be configured to lower the display refresh rate to conserve the battery of the UE 102. Conversely, during a high-speed chase, the AI model of the system 100 may ensure that the display refresh rate is at its peak to maintain a seamless visual experience.

[0044] Further, the display refresh rate may be determined and implemented using modules of the system 100 as explained in forthcoming paragraphs of Figure 2-6.

[0045] Figure 2a illustrates a schematic block diagram of modules components of the system 100 for power management by regulating the display refresh rate of the UE 102, according to an embodiment of the present disclosure.

[0046] The UE 102 may include but is not limited to, a processor 202, memory 204, modules 206, and data 208. The modules 206 and the memory 204 may be coupled to the processor 202.

[0047] The processor 202 can be a single processing unit or several units, all of which could include multiple computing units. The processor 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor 202 is adapted to fetch and execute computer-readable instructions and data stored in the memory 204.

[0048] The memory 204 may include any non-transitory computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random-access memory (DRAM), and / or non-volatile memory, such as read-only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory 204 may alternatively be referred to as the database in the present disclosure, within the scope of the invention.

[0049] The modules 206, amongst other things, include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement data types. The modules 206 may also be implemented as, signal processor(s), state machine(s), logic circuitries, and / or any other device or component that manipulates signals based on operational instructions.

[0050] Further, the modules 206 can be implemented in hardware, instructions executed by a processing unit, or by a combination thereof. The processor 202 can comprise a computer, a processor, a state machine, a logic array, or any other suitable devices capable of processing instructions. The processing unit can be a general-purpose processor (e.g., processor 202) which executes instructions to cause the general-purpose processor to perform the required tasks or, the processing unit can be dedicated to performing the required functions. In an embodiment of the present disclosure, the modules 206 may be machine-readable instructions (software) which, when executed by the processor 202 / processing unit, perform any of the described functionalities / methods, as discussed throughout the present disclosure.

[0051] In an embodiment, the modules 206 may include a generating module 210, a determining module 212, and an implementing module 214. The generating module 210, the determining module 212, and the implementing module 214 may be in communication with each other. The data 208 serves, amongst other things, as a repository for storing data processed, received, and generated by one or more of the modules 206. Figure 3, Figure 4, and Figure 5 provide a detailed description of each of the modules 206.

[0052] Referring to Figure 1 and Figure 2a, in an embodiment, the generating module 210 may be configured to generate the velocity vectors associated with a content displayed on the display of the UE 102, while the UE 102 maintains a first display refresh rate. The velocity vectors may be generated based on a scene classification and a set of user input variables. In an example, the velocity vectors may refer to a current spatial-temporal velocity of pixels in the display of the UE 102.

[0053] In an embodiment, the generating module 210 may be configured to estimate a jitter level in the displayed content observed by the user based on the velocity vectors while the UE 102 maintains the first display refresh rate. The jitter level may correspond to variations in the displayed content, thereby resulting in an unstable appearance. According to an embodiment of the present disclosure, the generating module 210 may be configured to estimate a jitter level perceived by a user related to the displayed content using an Artificial Intelligence (AI) model.

[0054] In an embodiment, the determining module 212 may be configured to determine a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. For example the determining module 212 determines the second display refresh rate that maintains the quality of the content within a predefined threshold (T), based on the velocity vectors. Consequently, the jitter level perceived by the user while interacting with the UE 102 remains unchanged. The second display refresh rate may be associated with the quality of the content displayed on the UE 102. Further, the predefined threshold (T) may correspond to a predefined limit of the UE 102, beyond which a decrease in the display refresh rate would lead to a loss in the quality of the content displayed. That is, power consumption can be managed by adjusting the display refresh rate, and since the adjusted display refresh rate maintains the estimated jitter level, the displayed content can remain visually stable. According to an embodiment of the present disclosure, the determining module 212 may determine a second display refresh rate using the AI model.

[0055] In an embodiment, the implementing module 214 may be configured to apply the second display refresh rate for the content displayed on the UE 102. Thereby, power consumption is managed in the UE 102 while the content displayed remains visually stable due to regulation in the display refresh rate.

[0056] Figure 2b illustrates a schematic process flow 200 for power management by regulating the display refresh rate of the UE 102 via the modules of the system 100, according to an embodiment of the present disclosure.

[0057] At step 220, the process flow 200 may include the system 100 generating, via the generating module 210, the velocity vectors for the pixels on the display of the UE 102 based on various inputs. These inputs may include the content classification, e.g., identifying whether the content is a video, game, or static image. Further, the inputs may include the continuous interaction information such as user touches and gestures, device sensor data from accelerometer and gyroscope, and super slow-motion cameras. These inputs may help the system 100 determine the movement and speed of the pixels in the display of the UE 102.

[0058] In an example scenario, while watching a high-speed car race on the UE 102, say on a smartphone. The generating module 210 may analyze the video content to classify it as a fast-moving scene. The generating module 210 may also track gesture of the user (watching the content), if the user is scrolling or zooming in and out on the display of the UE 102. The device's sensors may detect any shaking or tilting of the smartphone. All these inputs may be used to generate the velocity vectors, which correspond to the speed and direction of the pixels moving on the screen (display) of the UE 102.

[0059] At step 224, the process flow 200 may include the system 100 capturing the velocity vectors at current time (Tc). The velocity vectors may provide a snapshot of the pixel movements at that specific moment or time.

[0060] In an example scenario, while watching the car race, at time Tc, the velocity vectors indicate the rapid movement of cars across the screen, showing their current speed and direction. A detailed explanation of the generating module 210 is provided in Figure 3.

[0061] At step 226, the process flow 200 may include the system 100 using the velocity vectors to determine an optimal display refresh rate. The determination of optimal refresh rate may include mapping the velocity vectors to an appropriate refresh rate based on human perception of motion. In an example, neuromorphic sensing techniques, which mimic the neural pathways of the human brain, may be used to learn and predict the optimal refresh rate for the given content. A detailed explanation of the neuromorphic sensing techniques is explained in Figure 4 via the determining module 212.

[0062] In an example scenario, the system 100 analyzes the velocity vectors of the racing cars and uses neuromorphic sensing to determine that a higher refresh rate. May be for e.g., 120Hz is needed to ensure smooth and clear motion perception during the race.

[0063] At step 228, the process flow 200 may include the system 100 determining, via the determining module 212, the optimal refresh rate for the current time (Tc) based on the previous step. In an advantageous aspect, the optimal refresh rate may ensure the best visual experience while maintaining the energy efficiency of the UE 102.

[0064] In an example scenario, the system 100 determines that for the current scene of the car race, a refresh rate of 120Hz may be optimal to prevent any blurring or jittering (while displaying) of the fast-moving cars.

[0065] At step 230, the process flow 200 may include the system 100 implementing, via the implementing module 214, the determined optimal refresh rate to the display of the UE 102. In an advantageous aspect, the implementation ensures that the visual content is rendered smoothly and clearly on the UE 102.

[0066] In an example scenario, the system 100 adjusts the display refresh rate of the UE 102 to 120Hz while the user watches the car race, providing a smooth and immersive viewing experience.

[0067] At step 232, the process flow 200 may include the system 100 instead of adjusting in real-time or instantly, scheduling the adjustment of the display refresh rate for the next time (Tn).

[0068] At step 234, the process flow 200 may include the system 100 determines if the current time (Tc) equals the scheduled update time (Tn).

[0069] In continuation with step 234, at step 236, the process flow 200 may include the system 100 updating the current refresh rate to the newly determined optimal refresh rate, in response to matching the current time with the scheduled update time.

[0070] In an example scenario, the system 100 at the scheduled instance Tn, if the system 100 determines a new optimal refresh rate say, 90Hz for a slower scene, the system 100 may update the display refresh rate of the UE 102 accordingly.

[0071] In continuation with step 234, at step 238, the process flow 200 may include the system 100 maintaining the current refresh rate until the next scheduled check in response to determining that Tcis not equal to Tn.

[0072] In an example scenario, the system 100 continues to operate the display of the UE 102 at the current refresh rate of 120Hz until the next check (Tn).

[0073] A detailed explanation of each of the generating module 210, the determining module 212, and the implementing module 214 is provided in the forthcoming paragraphs.

[0074] Figure 3 illustrates a block diagram associated with the generating module 210 of the system 100, according to an embodiment of the present disclosure.

[0075] In an embodiment, the generating module 210 may include a deep neural network (DNN model) (the AI model), trained to generate the velocity vectors associated with the content displayed on the display of the UE 102.

[0076] The generating module 210 may be configured to collect detailed data related to interactions of the user with the UE 102 and the visual content being displayed on the UE 102. Thus, at step 302, the generating module 210 may be configured to classify a scene being displayed on the UE 102. The scene corresponds to the displayed content, thus, the scene classification in a non-limiting example may refer to identifying whether the scene is one of,

[0077] Completely or fully changing 2-dimensional (2D) scene i.e., vertical scrolls such that all the pixels in the set of pixels move with the same velocity.

[0078] Partially changing 2D scene, i.e., swipe or horizontal scrolls, where a first set of pixels moves while another set of pixels may be stationary or moving, or 2D content, where a subset of the set of pixels moves based on user inputs.

[0079] 3D scene: For 3D games where different pixels at different depths move with different velocities based on user input.

[0080] In an advantageous aspect, the scene classification may help in understanding the context of user interactions.

[0081] At step 304, the generating module 210 may be configured to determine the set of user input variables, using a slow-motion camera, to capture the user's interactions with the UE 102. In a non-limiting example, the set of user input variables may include, start and end coordinates i.e., the initial and final positions of the user's touch or gesture on the screen of the UE 102 and speed of interaction i.e., the rate at which the user performs the interaction with the UE 102 e.g., a swipe speed. In an example scenario, if the user swipes from coordinate (x1, y1) to (x2, y2) in 0.5 seconds, such interaction may be captured in slow motion to record precise details.

[0082] Further, at step 306, the generating module 210 may be configured to determine variations in the user interactions by capturing multiple instances based on recording a plurality of instances of the user interactions at N discrete levels. Each of the N discrete levels may correspond to a different variation or type of the user interaction. For instance, the N discrete levels may vary by speed, direction, pressure, or other user interaction characteristics. In an example scenario, recording the plurality of instances of the user interaction may include slow swipes, medium-speed swipes, and fast swipes, all captured separately to cover a range of user behaviours.

[0083] In an embodiment, at step 308, the velocity vectors are generated using the DNN model. The DNN model may be trained based on the relationships between the set of user input variables and the resulting motion of the set of pixels.

[0084] In an example, during a training phase of the DNN model, the slow-motion camera may capture high-frame-rate video of the user interactions. From the captured videos of the training phase, multiple frames may be extracted for each of the user interaction instances. In an example scenario, if the user interaction lasts for 1 second and the slow-motion camera captures at 120 frames per second (fps), 120 frames may be extracted for that single user interaction. Thus, the frames may correspond to a detailed sequence of the user interaction over time.

[0085] Further, each of the extracted frames may be labelled with relevant information about the user interaction that the frame may represent. For instance, the labelling in a non-limiting example may include the type of interaction e.g., swipe, tap; start and end coordinates of the user interaction, and the speed of the interaction. In an example scenario, if a swipe starts at coordinate (x1, y1) and ends at (x2, y2) in 0.5 seconds, then each of the extracted frames from that swipe may be labelled with this data along with the timestamp within the user interaction.

[0086] Furthermore, key motion features are extracted from the labelled frames. In a non-limiting example, the features may correspond to the difference in pixel positions between consecutive frames and the time intervals between the consecutive frames. In an example scenario, if an object moves from (x1, y1) to (x2, y2) over three frames, the position change and the time taken for the object movement are recorded as features.

[0087] Consequently, the DNN model is trained to understand and predict the relationship between the user interactions and the resulting visual motion. In an example, the labelled frames and the extracted features may be used as training data for the DNN model. In an advantageous aspect, the training data includes labelled frames with detailed metadata about the user interaction, with each of the frames linked to the user interaction's context and quantification of the motion observed between each of the frames, producing features that describe the movement of the set of pixels over time, constitutes a rich dataset for training the DNN model.

[0088] Thus, the DNN model is trained to learn the relationships between the user interactions (i.e., the set of user input variables) and the visual motion captured in the frames (i.e., the output variables). In an example scenario, the DNN model may learn patterns such as "a fast swipe results in rapid horizontal movement" by analysing the position changes and time intervals of the set of pixels corresponding to various swipes. Therefore, in an advantageous aspect, the trained DNN model, may predict pixel motion and generate the velocity vectors for new user interactions (during the inference phase), optimizing the display refresh rates accordingly.

[0089] Therefore, at step 310, the generating module 210 may be configured to generate the velocity vectors that describe the motion of pixels based on the trained DNN model. In an example, the trained DNN model may be used to analyse the user interactions (during the inference phase) and the content displayed on the display of the UE 102 with the first display refresh rate. The first display refresh rate may refer to the initial or current refresh rate at which the display of the UE 102 is operating while generating the velocity vectors. This first display refresh rate is the starting point or the baseline refresh rate before any adjustments or regulations are implemented based on the velocity vectors and subsequent analysis. Furthermore, the first display refresh rate is the rate at which the display of the UE 102 updates the displayed content per second. For example, if the first display refresh rate is 60 Hz, it means the display refreshes 60 times per second.

[0090] Furthermore, the generated velocity vectors may indicate the direction and speed of pixel movement on the display of the UE 102. In an example scenario, for a new swipe (the user interaction during the inference phase), the trained DNN model may predict how fast and in which direction the set of pixels should move, thus creating the velocity vector for the display to follow.

[0091] The table 1 provides an exemplary scenario for the current velocities of the set of pixels, as illustrated below:

[0092]

[0093] For example, the table 1 indicates that 20% of the set of pixels are accelerating, while 60% of the set of pixels are decelerating, and the remaining pixels are stationary. The elapsed and remaining time for each set of pixels may vary. The velocity vector changes continuously at each time instant based on the acceleration and deceleration values and any updated input information.Further, at step 312, the generating module 210 may be configured to estimate the jitter level in the displayed content observed by the user based on the velocity vectors at the first display refresh rate. The jitter level may correspond to the variations or inconsistencies in the smoothness of the displayed content as perceived by the user. The jitter level may cause the displayed content to appear unstable or jerky, detracting from a smooth visual experience. For instance, high jitter levels may result in noticeable stuttering or flickering in the displayed content, which may be distracting or uncomfortable for the user.

[0094] In an example, based on analysing the direction and speed of pixel movements (the velocity vectors), the generating module 210 may be configured to estimate the consistency of the displayed content being rendered on the display of the UE 102. In one aspect, the estimation of the jitter level is crucial because if the adjusted display refresh rate does not align with the content's motion requirements, it may lead to an irregular frame display, resulting in a jittery or unstable appearance.

[0095] Figure 4a illustrates an exemplary scrolling scenario on two identical UE's 102a and 102b for the jitter level, according to an embodiment of the present disclosure.

[0096] As illustrated in the Figure 4a, a first UE 102a and a second UE 102b present two screenshots of the content, illustrating the scrolling scenario. In Figure 4a, the first UE 102a may be operating at the refresh rate of 120Hz, while the second UE 102b may be operating at the refresh rate of 60Hz. In an exemplary scenario, the first UE 102a and the second UE 102b may display identical content and may receive the same swipe input to initiate the scroll.

[0097] The screenshots demonstrate that upon scrolling as the velocity of the content increases, the first UE 102a running at 120Hz may produce more frames per second compared to the second UE 102b running at 60Hz. This results in a smaller gap between consecutive frames on the 120Hz device, leading to less noticeable jitter in the content on the display.

[0098] However, as the content is moving rapidly, the jitter caused by the gap becomes imperceptible to the human brain. As a result, the difference in smoothness between 120Hz and 60Hz is not distinguishable to the human eye under high-velocity conditions. Thus, while the first UE 102a running at 120Hz technically offers a higher refresh rate and smoother frame transitions, the practical perceptual difference is minimal when the content moves quickly.

[0099] Figure 4b illustrates an exemplary scrolling scenario on two identical UE's 102a and 102b for the jitter level, according to an embodiment of the present disclosure.

[0100] Referring to Figure 4b, the first UE 102a may be operating at the refresh rate of 120Hz, while the second UE 102b may be operating at 60Hz. In an exemplary scenario, the first UE 102a and the second UE 102b may display identical content and receive the same swipe input to initiate the scroll.

[0101] As illustrated, the Figure 4b illustrates the content velocity slows down and settles. As the content slows down, the first UE 102a (120Hz) handles the speed reduction more effectively, resulting in a lower jitter level. Consequently, there is a smaller gap between consecutive frames, making the text more legible even while it is still moving.

[0102] As illustrated, the second UE 102b (60Hz) also reduces jitter as the content slows down, but not as effectively as the first UE 102a. Thus, resulting in larger gaps between frames, and a more noticeable jitter level. Due to the lower velocity of the content, the jitter level on the second UE 102b may become highly perceivable, and users may easily notice and differentiate the stutter, leading to a less smooth visual experience compared to the first UE 102a. Thus, the higher refresh rate of the first UE 102a may provide a smoother and clearer display during slower movement, enhancing the user experience by minimizing perceivable jitter and stutter.

[0103] Figure 5 illustrates a process flow associated with the determining module 212 of the system 100, according to an embodiment of the present disclosure.

[0104] In an embodiment, at step 502, the determining module 212 may be configured to obtain a perception stimulus matrix. The perception stimulus matrix may be pre-generated using a neuromorphic device. In a non-limiting example, the neuromorphic device may mimic human perception of content quality such that the perception stimulus matrix includes correlation among a stimulus value, at least one refresh rate, and the velocity vectors associated with the content.

[0105] In an embodiment, the perception stimulus matrix provides a mapping of a plurality of the display refresh rates, user interactions, and content types affecting perceived quality. In an advantageous aspect, the perception stimulus matrix is crucial for dynamically adjusting the display refresh rate to optimize power consumption while maintaining an optimal viewing experience.

[0106] In an example, during a training phase for the generation of the perception stimulus matrix, a captures the content displayed on the UE 102 under various conditions. Further, the user interactions may be simulated e.g., touch inputs, and gestures across the plurality of display refresh rates ranging from 1 Hz to M Hz. The user interactions during the training phase may cover a range of scenarios to ensure that a comprehensive training dataset is created. Furthermore, the neuromorphic device which mimics human visual and neural processing to estimate changes in the display affect perceived content quality, may be configured to generate a perception stimulus value (ΔL) for each of the plurality of display refresh rates and user interaction scenarios. The perception stimulus value (ΔL) as generated by the neuromorphic device may indicate the perceived change in frame luminance value and quality as experienced by the human eye and brain.

[0107] In the example, the perception stimulus value (ΔL) corresponds to the change in luminance and overall visual quality perceived by the user. Consequently, the perception stimulus value (ΔL) for each combination of the display refresh rate and user input scenario based on the camera feed data may be generated. In the example, the frame luminance value may correspond to the distribution of brightness levels within each frame of the displayed content. In the example, the frame luminance value may be computed based on a mean and standard deviation of luminance values.

[0108] Consequently, the perception stimulus values (ΔL), the frame luminance values, and the velocity vectors may be correlated using the DNN model to learn the relationship between these inputs. Therefore, the DNN model understands changes in the display refresh rate and user interactions affecting the perceived quality of the displayed content.

[0109] Consequently, the perception stimulus matrix is created wherein each entry represents the correlation between a specific display refresh rate, a set of velocity vectors, and the corresponding perception stimulus value (ΔL).

[0110] In an example scenario, the UE 102 may be displaying a fast-paced action movie. The camera captures the content at various refresh rates e.g., 30 Hz, 60 Hz, 90 Hz, while the user swipes and interacts with the display of the UE 102. The neuromorphic device may measure the user perception changes in the action movie's brightness and motion at each of the plurality of display refresh rates. For a swipe gesture at 60 Hz, the neuromorphic may generate a ΔL indicating a high-quality perception due to smooth motion. Further, a brightly lit action scene may have a high mean frame luminance value with a low standard deviation, indicating uniform brightness. Thus, the DNN model learns that at 60 Hz, the high ΔL correlates with fast-moving velocity vectors and stable luminance values. Similarly, at 30 Hz, the DNN model may learn that a lower ΔL due to perceived motion blur or stutter. Consequently, an entry in the perception stimulus matrix for the 60 Hz refresh rate and specific velocity vectors may depict a high ΔL, suggesting good perceived quality and for 30 Hz and the same velocity vectors, the perception stimulus matrix may depict a lower ΔL, thus indicating reduced quality.

[0111] At step 504, the determining module 212 may be configured to generate a reference perception stimulus value (ΔLRRR). RRR is a reference refresh rate. The reference perception stimulus value (ΔLRRR) may correspond to a change in luminance perceived at the first display refresh rate. The determining module 212 may be configured to generate the reference perception stimulus value (ΔLRRR) for each of the velocity vectors at the first display refresh rate (baseline refresh rate).

[0112] Furthermore, the determining module 212 may be configured to generate an Optimal Perception Stimulus Value (ΔLORR) for the velocity vectors but corresponding to each of the plurality of display refresh rates. ORR is an optimal refresh rate. The ΔLORRmay correspond to a change in luminance perceived at various other refresh rates.

[0113] At step 506, the determining module 212 may be configured to compare the reference perception stimulus value (ΔLRRR) and the optimal perception stimulus value (ΔLORR) with the predefined perception stimulus matrix. Consequently, the determining module 212 may be configured to compute the perception stimulus value (ΔL) that best matches the observed values, thus, indicating an effect of changes in the display refresh rate on the perceived quality of the displayed content.

[0114] At step 508, the determining module 212 may be configured to determine if the perception stimulus value (ΔL) is less than the predefined threshold (T) based on the comparison, thus, confirming an acceptable quality of the displayed content. In an example, the predefined threshold (T) may correspond to a pre-stored loss of perceivable quality to ensure that the reduction in the display refresh rate does not degrade the visual quality of the displayed content.

[0115] At step 510, the determining module 212 may be configured to determine the second display refresh rate using the DNN model. In an example, the second display refresh rate may correspond to a minimum display refresh rate at which the velocity vector results in a perceived quality difference less than the predefined threshold (T). In the example, the DNN model is trained to determine the second display refresh rate based on the training imparted using the mean and standard deviation of the frame luminance value and signals from the neuromorphic device indicating cumulative display effects, as described in above paragraphs. Thus, during the inference phase, the DNN model generates ΔL for the velocity vector (as generated based on scene classification) and outputs the second display refresh rate (the optimal refresh rate (ORR)) based on the following equation (1):

[0116] ...(1)

[0117] Thus, if ΔLORRis greater than ΔLRRR, exemplifying deceleration of moving content thus, requiring a higher refresh rate. Conversely, a lower ΔLORRexemplifies acceleration of moving content, allowing a reduced refresh rate.

[0118] In an example scenario, the user interacts with a high-motion video displayed as the content on the UE 102. The current display refresh rate or the first display refresh rate may be 90 Hz. Consequently, based on the neuromorphic device, the system 100 may pre-generate the perception stimulus matrix correlating the plurality of the display refresh rates, the velocity vectors, and the perceived quality changes. Further, the system 100 may be configured to determine ΔLRRRfor the high-motion video content at 90 Hz. Similarly, the system 100 may be configured to determine ΔLORRfor the same content at the plurality of display refresh rates e.g., 60 Hz, 75 Hz. Furthermore, based on the perception stimulus matrix the ΔLORRand ΔLRRRmay be compared to identify the perception stimulus value (ΔL) that best matches the observed quality of the high-motion video. Additionally, the system 100 may be configured to ensure the perception stimulus value (ΔL) is less than the predefined threshold (T), thus, maintaining acceptable quality. Consequently, the DNN model, trained on historical data including the frame luminance value and the outputs of the neuromorphic device, may determine the second display refresh rate. In the example, if the DNN model determines that 75 Hz maintains the quality of the high-motion video within the threshold (T), then 75 Hz may be selected as the second display refresh rate for the UE 102. In an advantageous aspect, the determination of the second display refresh rate thus ensures that the UE 102 optimizes power consumption by dynamically adjusting the display refresh rate while maintaining a seamless user experience.

[0119] Figure 6 illustrates an exemplary block diagram of the determining module 212 of the system 100, according to an embodiment of the present disclosure.

[0120] In an embodiment, the determining module 212 may include the DNN model 608 (the AI model), trained to determine the second refresh rate based on the perception stimulus value (ΔL) which is lesser than the predefined threshold (T).

[0121] At block 602, the determining module 212 may be configured to receive training phase inputs for training the DNN model 608. In an example, the DNN model 608 may be configured to receive the current refresh rate. In the example, the system 100 may be configured to record the current refresh rate ranging from 1Hz to M Hz during the training phase. In an example, the system 100 may also capture slow-motion videos of the content displayed on the UE 102 under different conditions and refresh rates during the training phase of the DNN 608. Furthermore, in an example, the DNN 608 may also receive the neuromorphic stimulus signals during the training phase.

[0122] Consequently, the DNN model 608 may be trained using the current refresh rate, slow-motion captures, and neuromorphic stimulus signals. In an example, the training phase includes capturing various user interactions such as but not limited to swipes, touches and content types such as but not limited to videos and games to create a comprehensive dataset meant for training. Furthermore, the neuromorphic device may generate the perception stimulus value (ΔL) for each combination of display refresh rates and user interaction scenarios, representing the perceived change in frame luminance and quality.

[0123] Consequently, the DNN model 608 may learn the correlation between refresh rates, user interactions, velocity vectors, and the perception stimulus values to create a perception stimulus matrix.

[0124] At block 604, the DNN model 608 may receive the generated velocity vectors and the standard deviation of frame luminance during the runtime phase.

[0125] Consequently, at block 610, the DNN model 608 may use the generated velocity vectors and the standard deviation (real-time data) along with the pre-trained perception stimulus matrix to determine the optimal refresh rate or alternatively referred to as the second refresh rate within the scope of the present disclosure.

[0126] Thus, the DNN model 608 determines the optimal (second) display refresh rate that maintains perceived quality within the predefined threshold (T).

[0127] In an example scenario, the user may be watching a high-motion video (content) on the UE 102 with the current display refresh rate of the UE as 90Hz. The system 100 may capture the video content and user interactions, generating the velocity vectors and frame luminance data. Further, the neuromorphic device may pre-generate the perception stimulus values for different refresh rates. The DNN model 608 may then determine ΔLRRRfor the video at 90Hz and ΔLORRfor other refresh rates such as e.g., 60Hz, 75Hz. Furthermore, based on comparing these values, the DNN model 608 determines the optimal refresh rate for the UE 102, which maintains quality within the threshold T. In the example, if 75Hz of refresh rate maintains acceptable quality, it may be selected as the second refresh rate. Therefore, the UE 102 may adjust to 75Hz, thus, optimizing power consumption of the UE 102 while maintaining smooth video playback.

[0128] Figure 7 illustrates a process flow associated with the implementing module 214 of the system 100, according to an embodiment of the present disclosure.

[0129] In an embodiment, at step 702, the implementing module 214 may be configured to select an optimal time and interval to apply the second display refresh rate for the content displayed on the UE 102 by modelling a plurality of display power consumption parameters.

[0130] In an embodiment, the implementing module 214 may be modelled as a temporal update scheduler which is responsible for efficiently regulating the display refresh rate to balance the power consumption and visual quality in the UE 102. The implementing module 214 may be implemented using the DNN model based on a sequence model, such as a Recurrent Neural Network (RNN).

[0131] In an embodiment, as discussed in the previous paragraphs the second display refresh rate is determined for every frame of the content. However, frequent updates in the display refresh rate may increase power consumption in the UE 102 due to the overhead of switching the display refresh rate. Thus, the implementing module 214 may receive real-time data related to the power consumption of the display of the UE 102. The real-time data related to the power consumption of the display may be computed based on the frame luminance value.

[0132] In an example, the implementing module 214 may be configured to obtain a sequence of previous display refresh rates and the velocity vectors to understand the content's motion dynamics over time.

[0133] Further, the implementing module 214 may be configured to model a cost function based on power consumption parameters. The power consumption parameters may include a refresh power cost (PR) which corresponds to the power consumed per refresh rate, which depends on the current brightness and the mean of the frame luminance value of the display. The power consumption parameters may further include a refresh rate switch Power Cost (PS) which is the power overhead associated with switching the display refresh rate from the first display refresh rate to the second display refresh rate.

[0134] Further, the cost function may consider, a cumulative power savings (PowerD) referring to the total power difference obtained by switching to the second display refresh rate over a set of future timestamps. Furthermore, the cost function may further include a cumulative switching cost (PowerS) referring to the total power cost of switching the display refresh rate multiple times.

[0135] In an example, the implementing module 214 may be configured to compute the cumulative power savings (PowerD) and the cumulative switching cost (PowerS) using the following equations (2) and (3).

[0136] ... (2) and

[0137] ...(3)

[0138] Thus, the Cost function is computed as illustrated in equation (4) below:

[0139] ...(4)

[0140] The final cost function is computed as illustrated in equation (5) below becomes,

[0141] ...(5)

[0142] In an example, the PowerD being negative may indicate that the overall switching from the first display refresh rate to the second display refresh rate leads to power savings.

[0143] In another example, the PowerD being positive may indicate that the switching from the first display refresh rate to the second display refresh rate results in power loss.

[0144] In an embodiment, the RNN (the implementing module 214) may be trained to predict a sequence of second display refresh rates based on past data. In an advantageous aspect, the prediction helps in understanding the content's acceleration and deceleration patterns.

[0145] In an example, the RNN (the implementing module 214) may be configured to receive a plurality of second display refresh rates from the past data, the frame luminance value, and the real-time power consumption values to learn the optimal intervals for updating the display of the UE 102 with the determined second display refresh rate. In the example, the RNN may assign weights to the plurality of second display refresh rates from the past data within a sliding time window to predict optimal timestamps for regulating the display refresh rates.

[0146] Consequently, the RNN (the implementing module 214) may be configured to predict the second display refresh rates for the future. In the example, the RNN predicts a set of N future second display refresh rates (PRRf) based on the sequence of plurality of second display refresh rates from the past data.

[0147] The RNN (the implementing module 214) may be configured to compute a set of N future timestamps (Tf) for applying the predicted second display refresh rates. In an advantageous aspect, the N future timestamps (Tf) may ensure that the refresh rate updates occur at optimal intervals to minimize power consumption in the UE 102.

[0148] Consequently, at step 704, the RNN (the implementing module 214) may be configured to apply the second display refresh rate for the content displayed on the UE 102. In an example, the implementing module 214 may be configured to select the first timestamp in the computed set (Tf1) as the optimal time to apply the determined second display refresh rate. In an advantageous aspect, the selection of the first timestamp ensures that the update of the second display refresh rate aligns with the content's motion dynamics and power management criteria of the UE 102.

[0149] Figure 8a illustrates an exemplary block diagram of the implementing module 214 of the system, according to an embodiment of the present disclosure.

[0150] Figure 8b illustrates an exemplary process flow 800 of the implementing module 214 to be modelled as the temporal update scheduler 802, according to an embodiment of the present disclosure.

[0151] Referring to Figure 8a and 8b collectively, in an embodiment, the temporal update scheduler 802 is configured to efficiently regulate the display refresh rate for balancing the power consumption and visual quality in the UE 102.

[0152] The temporal update scheduler 802 may be configured to receive the current timestamp (Tc), inputs from the determining module 212, refresh power cost of the display (PR) based on current brightness and the mean luminance value, and the panel refresh rate switch power cost (PS) for switching from one refresh rate to another. The RNN 804 within the temporal update scheduler 802 processes these inputs to generate the set of N predicted optimal refresh rate values (PRRf) for future timestamps. The PRRfmay be based on a sequence model that considers the previous few samples of data. The number of future samples (N) and the maximum time limit (f) for predictions may be determined by the RNN 804 based on the variance of previous optimal refresh rate (ORR) values.

[0153] In an example scenario, if the content velocity is steadily increasing or decreasing, the RNN 804 may predict finer and longer sets of future refresh rate values. Conversely, the irregular content velocity patterns may result in predictions with shorter intervals and fewer samples by the RNN 804.

[0154] Further, the DNN 806 in the temporal update scheduler 802 may be configured to receive the predicted optimal refresh rate values (PRRf) and calculate the set of N future timestamps (Tf) for applying the optimal refresh rates, thereby advantageously scheduling the refresh rate updates at optimal intervals to balance power consumption and content motion dynamics in the UE 102.

[0155] Further, the temporal update scheduler 802, at block 808, may be configured to compute the cost function to maximize power savings, using equations (2) - (5) as mentioned in Figure 7 and omitted herein for the sake of brevity. The cost function combines the PowerD and the PowerS to evaluate the net power savings versus the switching costs.

[0156] Consequently, at block 810, if the computed cost function is minimized, the temporal update scheduler 802 may be configured to update the future timestamps (Tf) accordingly. Thus, this ensures that the refresh rate changes are applied efficiently, maintaining content quality and minimizing power usage in the UE 102.

[0157] Alternatively, at block 812, if the cost function is not minimized, the DNN 806 back propagates to refine the set of future timestamps (Tf). Thus, this iterative process continues until the optimal set of future timestamps is determined, ensuring efficient application of refresh rates based on real-time data.

[0158] Figure 9 illustrates an exemplary process flow comprising a method 700 for power management by regulating the display refresh rate of the UE 102, according to an embodiment of the present disclosure. The method 900 may be a computer-implemented method executed, for example, by the UE 102 and the modules 206. For the sake of brevity, constructional and operational features of the system 100 that are already explained in the description of Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, and Figure 8 are not explained in detail in the description of Figure 9.

[0159] At step 902, the method 900 may include obtaining the velocity vectors associated with the content displayed on the display of the UE 102 with the first display refresh rate. For example the method 900 may include obtaining the velocity vectors associated with the content displayed on the display of the UE 102 with the first display refresh rate, based on the scene classification and the set of user input variables. The velocity vectors may indicate the current spatial-temporal velocity of pixels in the display of the UE 102.

[0160] At step 904, the method 900 may include estimating the jitter level in the displayed content perceived by the user based on the velocity vectors at the first display refresh rate. In an example, the jitter level indicates variations in the displayed content, thereby resulting in an unstable appearance.

[0161] At step 906, the method 900 may include determining the second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. For example the method 900 may include determining the second display refresh rate that maintains the quality of the content within the predefined threshold (T), based on the velocity vectors, such that the jitter level observed by the user remains unchanged. In an example, the second display refresh rate is associated with the quality of the content displayed. In an example, the predefined threshold (T) may indicate the predefined limit beyond which a decrease in the display refresh rate would lead to a loss in the quality of the content displayed. According to an embodiment of the present disclosure, the method 900 may include determining a second display refresh rate using an AI model.

[0162] At step 908, the method 900 may include applying the second display refresh rate for the content displayed on the UE 102 thereby managing the power consumption in the UE 102 while the content displayed remains visually stable due to regulation in the display refresh rate.

[0163] Figure 10 illustrates another flowchart depicting a method 1000 at run-time for power management by regulating the display refresh rate of the UE, according to an embodiment of the present disclosure.

[0164] While the following discussed steps in Figures 10 are shown and described in a particular sequence, the steps may occur in variations to the sequence in accordance with various embodiments. Further, a detailed description related to the various steps of Figure 10 is already covered in the description related to Figures 1-8 and is omitted herein for the sake of brevity.

[0165] At block 1002, an application is initiated, and the scene classification is performed. The current refresh rate is set to the default value and a set of user input variables are determined. In an example, the velocity vectors may refer to a current spatial-temporal velocity of pixels in the display of the UE 102.

[0166] At block 1004, continuous user interaction information in form of the set of user variables may be provided to the generating module 210.

[0167] At block 1006, the velocity vectors may be generated based on the scene classification and the set of user input variables.

[0168] At block 1008, the velocity vectors at current time (Tc) may be generated.

[0169] At block 1010, the DNN model or refresh rate preceptor model may receive the generated velocity vectors and the standard deviation (real-time data) along with the pre-trained perception stimulus matrix to determine the optimal refresh rate or alternatively referred to as the second refresh rate within the scope of the present disclosure.

[0170] At block 1012, the optimal refresh rate or alternatively referred to as the second refresh rate may be computed.

[0171] In a sub-step of block 1012, at block 1012-a, it may be determined if the optimal refresh rate is more than the current refresh rate.

[0172] At block 1014, in a scenario with the optimal refresh rate is not more than the current refresh rate, the system determines whether refresh may be performed based on the optimal refresh rate.

[0173] At block 1016, the temporal update scheduler receives the current timestamp (Tc), inputs from the determining module 212, refresh power cost of the display (PR) based on current brightness and the mean luminance value, and the panel refresh rate switch power cost (PS) for switching from one refresh rate to another.

[0174] At block 1018, the temporal update scheduler calculates the set of N future timestamps (Tf) for applying the optimal refresh rates.

[0175] At block 1020, the system 100 determines if an update of the refresh rate is required based on the Tcequaling to Tn.

[0176] At block 1022, the system 100 maintaining the current refresh rate until the next scheduled check in response to determining that Tcis not equal to Tn.

[0177] At block 1024, the system 100 updating the current refresh rate of the display of the UE to the newly determined optimal refresh rate, in response to matching the current time with the scheduled update time.

[0178] At block 1026, thus, this iterative process continues until the optimal set of future timestamps is determined, ensuring efficient application of refresh rates based on real-time data.

[0179] Figure 11 illustrates an exemplary use-case scenario for the display refresh rate dynamically adjusting to content velocity changes, according to an embodiment of the present disclosure.

[0180] Referring to Figure 11, depicts the display's current refresh rate (CRR) 1102 is adjusted based on the content velocity during a single touch and swipe scenario. The optimal refresh rate (ORR) is determined at each timestamp 1104 according to the frame velocity but is applied to the display only at the timestamp specified based on the temporal update scheduler.

[0181] In an example scenario, the display refresh rate of the UE 102 may be initially set at 120 Hz. Now, at timestamp 1 to 3 the ORR may be computed based on the changing content velocity. For instance, at timestamp 3, the ORR is calculated to be 80 Hz. Furthermore, the temporal update scheduler schedules the ORR (80 Hz) to be applied at timestamp 4.

[0182] Thus, as the time timestamp 4 approaches, the ORR has been recalculated to 70 Hz due to further changes in content velocity. The display refresh rate is then set to the latest ORR value of 70 Hz at timestamp 4, disregarding the previously scheduled 80 Hz. Thus, this sequence illustrates the dynamic adjustment of the display refresh rate to content velocity changes, ensuring that the most recent ORR is applied for optimal performance in the UE 102.

[0183] The present disclosure provides various advantages:

[0184] The present disclosure provides enhanced power efficiency by dynamically adjusting the refresh rate, the UE may conserve battery power, extending the UE's operational life between charges.

[0185] The present disclosure provides improved user experience by maintaining optimal jitter levels to ensure that users enjoy a smooth and stable visual experience, regardless of the content type.

[0186] The present disclosure provides content-sensitive adjustments i.e., the ability to differentiate between various types of content, thus allowing for tailored display refresh rate adjustments, further optimizing power usage.

[0187] The present disclosure provides power management of the UE thus aids in improving thermal state of the UE and enhancing sustainable performance.

[0188] The method according to an embodiment of the present disclosure describes in detail a system to choose and set an optimal set of display refresh rates to best suit the rate of change of movement of display frame content without causing discernible difference to end user perception, along with an optimal time and schedule in terms of frequency to switch the refresh rate.

[0189] According to one embodiment of the present disclosure, a method for power management by regulating a display refresh rate of a user equipment (UE) may include obtaining velocity vectors associated with a content displayed on a display of the UE with a first display refresh rate. The method may include estimating a jitter level perceived by a user related to the displayed content by measuring variations in the displayed content resulting in an unstable appearance of the displayed content. The method may include determining a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. The method may include applying the second display refresh rate instead of the first display refresh rate for the content displayed on the UE (102) thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

[0190] According to one embodiment of the present disclosure, the method may include capturing a scene classification and a set of user input variables indicating user interactions using a slow-motion camera capture, wherein the user interactions include at least one start and end coordinates and a speed of interaction. The method may include recording a plurality of instances of the user interactions at N discrete levels, wherein each of the discrete levels indicates a distinct variation in the user interactions. The method may include extracting a plurality of frames from the slow-motion camera capture for each instance of the plurality of instances of the user interactions. The method may include labelling the extracted plurality of frames with the corresponding user interaction. The method may include extracting features from the labelled frames, the features indicating motion over time and including at least, position changes between labelled frames and time intervals between labelled frames. The method may include training a deep neural network (DNN) with the scene classification and the extracted features, wherein the DNN is trained to model relationship between the user interactions and the slow-motion camera capture. The method may include generating the velocity vectors associated with the content displayed on the UE using the trained DNN, wherein the velocity vectors indicate a current spatial-temporal velocity of pixels in the display of the UE (102).

[0191] According to one embodiment of the present disclosure, the method may include obtaining a perception stimulus matrix pre-generated using a neuromorphic device, wherein the neuromorphic device mimics human perception of content quality and the perception stimulus matrix includes correlation among a stimulus value, at least one refresh rate, and the velocity vectors. The method may include generating a reference perception stimulus value (ΔLRRR) for each of the corresponding velocity vectors at the first display refresh rate, wherein the reference perception stimulus value (ΔLRRR) indicates change in luminance perceived by the user due to motion. The method may include generating an optimal perception stimulus value (ΔLORR) for each of the corresponding velocity vectors at a plurality of display refresh rates different from the first display refresh rate. The method may include comparing the reference perception stimulus value (ΔLRRR) and the optimal perception stimulus value (ΔLORR) with the predefined perception stimulus matrix. The method may include determining if a perception stimulus value (ΔL) is less than the predefined threshold (T) based on the comparison. The method may include determining the second display refresh rate based on the perception stimulus value (ΔL) which is lesser than a predefined threshold (T), such that the second display refresh rate satisfies a predefined perception criteria, wherein the predefined perception criteria indicate factors for an optimal viewing experience. The predefined threshold (T) indicates a predefined limit beyond which a decrease in the display refresh rate would lead to a loss in quality of the content displayed such that the jitter level observed by the user remains unchanged.

[0192] According to one embodiment of the present disclosure, the method may include when the second display refresh rate is more than the first refresh rate then increasing the display refresh rate in response to deceleration of the content displayed.

[0193] According to one embodiment of the present disclosure, the method may include when the second display refresh rate is less than the first refresh rate then decreasing the display refresh rate in response to acceleration of the content displayed.

[0194] According to one embodiment of the present disclosure, the method may include generating the perception stimulus value (ΔL) from the neuromorphic device against each of the plurality of display refresh rates and the set of user input variables, based on a camera feed, wherein the perception stimulus indicates a signal or stimulus representing a human eye and brain's perception of a quality of the content during a training phase. The method may include obtaining a frame luminance value indicative of distribution of brightness levels within a frame of the content displayed. The method may include learning a correlation between the generated perception stimulus value (ΔL) by the neuromorphic device and the corresponding velocity vectors. The method may include obtaining the predefined perception stimulus matrix based on the correlation learned.

[0195] According to one embodiment of the present disclosure, the method may include selecting an optimal time and interval to applying the second display refresh rate for the content displayed on the UE by modelling a plurality of display power consumption parameters.

[0196] According to one embodiment of the present disclosure, the set of user input variables may include start and end coordinates of user inputs and a speed of swipe.

[0197] According to one embodiment of the present disclosure, the content may include one or more of a frame, an object in the frame, and a background in the frame.

[0198] According to one embodiment of the present disclosure, a system for power management by regulating a display refresh rate of a user equipment (UE) may include a memory and at least one processor in communication with the memory. The at least one processor may be configured to obtain velocity vectors associated with a content displayed on a display of the UE (102) with a first display refresh rate. The at least one processor may be configured to estimate a jitter level perceived by a user related to the displayed content by measuring variations in the displayed content, resulting in an unstable appearance of the displayed content. The at least one processor may be configured to determine a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level. The at least one processor may be configured to apply the second display refresh rate instead of the first display refresh rate for the content displayed on the UE (102) thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.

[0199] According to one embodiment of the present disclosure, the at least one processor may be configured to capture a scene classification and a set of user input variables indicating user interactions using a slow-motion camera capture, wherein the user interactions include at least one start and end coordinates and a speed of interaction. The at least one processor may be configured to record a plurality of instances of the user interactions at N discrete levels, wherein each of the discrete levels indicates a distinct variation in the user interactions. The at least one processor may be configured to extract a plurality of frames from the slow-motion camera capture for each instance of the plurality of instances of the user interactions. The at least one processor may be configured to label the extracted plurality of frames with the corresponding user interaction. The at least one processor may be configured to extract features from the labelled frames, the features indicating motion over time and including at least, position changes between labelled frames and time intervals between labelled frames. The at least one processor may be configured to train a deep neural network (DNN) with the scene classification and the extracted features, wherein the DNN is trained to model relationship between the user interactions and the slow-motion camera capture. The at least one processor may be configured to generate the velocity vectors associated with the content displayed on the UE (102) using the trained DNN, wherein the velocity vectors indicate a current spatial-temporal velocity of pixels in the display of the UE (102).

[0200] According to one embodiment of the present disclosure, the at least one processor may be configured to obtain a perception stimulus matrix pre-generated using a neuromorphic device, wherein the neuromorphic device mimics human perception of content quality and the perception stimulus matrix includes correlation among a stimulus value, at least one refresh rate, and the velocity vectors. The at least one processor may be configured to generate a reference perception stimulus value (ΔLRRR) for each of the corresponding velocity vectors at the first display refresh rate, wherein the reference perception stimulus value (ΔLRRR) indicates change in luminance perceived by the user due to motion. The at least one processor may be configured to generate an optimal perception stimulus value (ΔLORR) for each of the corresponding velocity vectors at a plurality of display refresh rates different from the first display refresh rate. The at least one processor may be configured to compare the reference perception stimulus value (ΔLRRR) and the optimal perception stimulus value (ΔLORR) with the predefined perception stimulus matrix. The at least one processor may be configured to determine if a perception stimulus value (ΔL) is less than the predefined threshold (T) based on the comparison. The at least one processor may be configured to determine the second display refresh rate based on the perception stimulus value (ΔL) which is lesser than a predefined threshold (T), such that the second display refresh rate satisfies a predefined perception criteria, wherein the predefined perception criteria indicate factors for an optimal viewing experience.

[0201] According to one embodiment of the present disclosure, the at least one processor may be configured to, when the second display refresh rate is more than the first refresh rate, increase the display refresh rate in response to deceleration of the content displayed.

[0202] According to one embodiment of the present disclosure, the at least one processor may be configured to, when the second display refresh rate is less than the first refresh rate, decrease the display refresh rate in response to acceleration of the content displayed.

[0203] According to one embodiment of the present disclosure, the at least one processor may be configured to generate the perception stimulus value (ΔL) from the neuromorphic device against each of the plurality of display refresh rates and the set of user input variables, based on a camera feed, wherein the perception stimulus indicates a signal or stimulus representing a human eye and brain's perception of a quality of the content during a training phase. The at least one processor may be configured to obtain a frame luminance value indicative of distribution of brightness levels within a frame of the content displayed. The at least one processor may be configured to learn a correlation between the generated perception stimulus value (ΔL) by the neuromorphic device and the corresponding velocity vectors. The at least one processor may be configured to obtain the predefined perception stimulus matrix based on the correlation learned.

[0204] According to one embodiment of the present disclosure, a method for power management by regulating a display refresh rate of a user equipment (UE) may include measuring a rate of motion of a content displayed on a device operating at a first display refresh rate; estimating by an AI model, a jitter level perceived by the user due to the motion of the content at the first display refresh rate; determining by the AI model, a second display refresh rate at which the jitter level perceived by the user remains unchanged, wherein the determined second display refresh rate is the lowest possible value as compared to the first display refresh rate; and applying the second display refresh rate for the operation of the device.

[0205] According to one embodiment of the present disclosure, the AI model is trained using a neuromorphic or bionic device / sensor that generates signals simulating perceivable jitter by the user for different motions of content at different refresh rates.

[0206] According to one embodiment of the present disclosure, the method may include transforming one or more of user interactions, gestures and device sensor data to a display content velocity and acceleration vector in time and space that can be used as a quantitative input for modelling motion (velocity and acceleration) of content on the display.

[0207] According to one embodiment of the present disclosure, the method may include choosing an optimal time to apply the second display refresh rate for the operation of the device by modelling display power consumption properties and refresh rate change power cost.

[0208] According to one embodiment of the present disclosure, a method for power management by regulating a display refresh rate of a user equipment (UE) may include measuring a rate of motion of a content displayed on a device operating at a first display refresh rate; estimating by an AI model, a jitter perceivable by the user for the measured rate of motion; determining a change required in the first display refresh rate as a measure of the jitter perceivable by the user; and applying the required change onto the first refresh rate to render the subsequent content on the device.

[0209] While specific language has been used to describe the disclosure, any limitations arising on account of the same are not intended. As would be apparent to a person in the art, various working modifications may be made to the method in order to implement the inventive concept as taught herein.

[0210] The drawings and the forgoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of processes described herein may be changed and are not limited to the manner described herein.

Claims

1.A method (900) for power management by regulating a display refresh rate of a user equipment (UE) (102), the method (900) comprising:obtaining (902) velocity vectors associated with a content displayed on a display of the UE (102) with a first display refresh rate;estimating (904) a jitter level perceived by a user related to the displayed content by measuring variations in the displayed content resulting in an unstable appearance of the displayed content;determining (906) a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level; andapplying (908) the second display refresh rate instead of the first display refresh rate for the content displayed on the UE (102) thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.2.The method (900) of claim 1, wherein obtaining the velocity vectors associated with the content displayed on the UE (102) comprises:capturing a scene classification and a set of user input variables indicating user interactions using a slow-motion camera capture, wherein the user interactions include at least one start and end coordinates and a speed of interaction;recording a plurality of instances of the user interactions at N discrete levels, wherein each of the discrete levels indicates a distinct variation in the user interactions;extracting a plurality of frames from the slow-motion camera capture for each instance of the plurality of instances of the user interactions;labelling the extracted plurality of frames with the corresponding user interaction;extracting features from the labelled frames, the features indicating motion over time and including at least, position changes between labelled frames and time intervals between labelled frames;training a deep neural network (DNN) with the scene classification and the extracted features, wherein the DNN is trained to model relationship between the user interactions and the slow-motion camera capture; andgenerating the velocity vectors associated with the content displayed on the UE using the trained DNN, wherein the velocity vectors indicate a current spatial-temporal velocity of pixels in the display of the UE.3.The method (900) of claim 1 or claim 2, wherein determining the second display refresh rate comprises:obtaining a perception stimulus matrix pre-generated using a neuromorphic device, wherein the neuromorphic device mimics human perception of content quality and the perception stimulus matrix includes correlation among a stimulus value, at least one refresh rate, and the velocity vectors;generating a reference perception stimulus value (ΔLRRR) for each of the corresponding velocity vectors at the first display refresh rate, wherein the reference perception stimulus value (ΔLRRR) indicates change in luminance perceived by the user due to motion;generating an optimal perception stimulus value (ΔLORR) for each of the corresponding velocity vectors at a plurality of display refresh rates different from the first display refresh rate;comparing the reference perception stimulus value (ΔLRRR) and the optimal perception stimulus value (ΔLORR) with the predefined perception stimulus matrix;determining if a perception stimulus value (ΔL) is less than the predefined threshold (T) based on the comparison; anddetermining the second display refresh rate based on the perception stimulus value (ΔL) which is lesser than a predefined threshold (T), such that the second display refresh rate satisfies a predefined perception criteria, wherein the predefined perception criteria indicate factors for an optimal viewing experience.4.The method (900) of claim 3, further comprising:when the second display refresh rate is more than the first refresh rate, increasing the display refresh rate in response to deceleration of the content displayed.5.The method (900) of claim 3, further comprising:when the second display refresh rate is less than the first refresh rate, decreasing the display refresh rate in response to acceleration of the content displayed.6.The method (900) of claim 3, wherein obtaining the predefined perception stimulus matrix comprises:generating the perception stimulus value (ΔL) from the neuromorphic device against each of the plurality of display refresh rates and the set of user input variables, based on a camera feed, wherein the perception stimulus indicates a signal or stimulus representing a human eye and brain's perception of a quality of the content during a training phase;obtaining a frame luminance value indicative of distribution of brightness levels within a frame of the content displayed;learning a correlation between the generated perception stimulus value (ΔL) by the neuromorphic device and the corresponding velocity vectors; andobtaining the predefined perception stimulus matrix based on the correlation learned.7.The method (900) of any one of claims 1 to 6, wherein applying the second display refresh rate comprises:selecting an optimal time and interval to applying the second display refresh rate for the content displayed on the UE by modelling a plurality of display power consumption parameters.8.The method (900) of claim 2, wherein the set of user input variables includes start and end coordinates of user inputs and a speed of swipe.9.The method (900) of any one of claims 1 to 8, wherein the content includes one or more of a frame, an object in the frame, and a background in the frame.10.A system (100) for power management by regulating a display refresh rate of a user equipment (UE) (102), the system (100) comprising:a memory (204); andat least one processor (202) in communication with the memory (204), the at least one processor (202) configured to:obtain velocity vectors associated with a content displayed on a display of the UE (102) with a first display refresh rate;estimate a jitter level perceived by a user related to the displayed content by measuring variations in the displayed content, resulting in an unstable appearance of the displayed content;determine a second display refresh rate as the smallest display refresh rate that maintains the estimated jitter level; andapply the second display refresh rate instead of the first display refresh rate for the content displayed on the UE (102) thereby managing power consumption while the content displayed remains visually stable due to regulation in the display refresh rate.11.The system (100) of claim 10, wherein to obtain the velocity vectors associated with the content displayed on the UE (102), the at least one processor (202) is configured to:capture a scene classification and a set of user input variables indicating user interactions using a slow-motion camera capture, wherein the user interactions include at least one start and end coordinates and a speed of interaction;record a plurality of instances of the user interactions at N discrete levels, wherein each of the discrete levels indicates a distinct variation in the user interactions;extract a plurality of frames from the slow-motion camera capture for each instance of the plurality of instances of the user interactions;label the extracted plurality of frames with the corresponding user interaction;extract features from the labelled frames, the features indicating motion over time and including at least, position changes between labelled frames and time intervals between labelled frames;train a deep neural network (DNN) with the scene classification and the extracted features, wherein the DNN is trained to model relationship between the user interactions and the slow-motion camera capture; andgenerate the velocity vectors associated with the content displayed on the UE (102) using the trained DNN, wherein the velocity vectors indicate a current spatial-temporal velocity of pixels in the display of the UE (102).12.The system (100) of claim 10 or claim 11, wherein to determine the second display refresh rate, the at least one processor (202) is configured to:obtain a perception stimulus matrix pre-generated using a neuromorphic device, wherein the neuromorphic device mimics human perception of content quality and the perception stimulus matrix includes correlation among a stimulus value, at least one refresh rate, and the velocity vectors;generate a reference perception stimulus value (ΔLRRR) for each of the corresponding velocity vectors at the first display refresh rate, wherein the reference perception stimulus value (ΔLRRR) indicates change in luminance perceived by the user due to motion;generate an optimal perception stimulus value (ΔLORR) for each of the corresponding velocity vectors at a plurality of display refresh rates different from the first display refresh rate;compare the reference perception stimulus value (ΔLRRR) and the optimal perception stimulus value (ΔLORR) with the predefined perception stimulus matrix;determine if a perception stimulus value (ΔL) is less than the predefined threshold (T) based on the comparison; anddetermine the second display refresh rate based on the perception stimulus value (ΔL) which is lesser than a predefined threshold (T), such that the second display refresh rate satisfies a predefined perception criteria, wherein the predefined perception criteria indicate factors for an optimal viewing experience.13.The system (100) of claim 12, wherein the at least one processor (202) is further configured to, when the second display refresh rate is more than the first refresh rate, increase the display refresh rate in response to deceleration of the content displayed.14.The system (100) of claim 12, wherein the at least one processor (202) is configured to, when the second display refresh rate is less than the first refresh rate, decrease the display refresh rate in response to acceleration of the content displayed.15.The system (100) of claim 12, wherein to obtain the predefined perception stimulus matrix, the at least one processor (202) is configured to:generate the perception stimulus value (ΔL) from the neuromorphic device against each of the plurality of display refresh rates and the set of user input variables, based on a camera feed, wherein the perception stimulus indicates a signal or stimulus representing a human eye and brain's perception of a quality of the content during a training phase;obtain a frame luminance value indicative of distribution of brightness levels within a frame of the content displayed;learn a correlation between the generated perception stimulus value (ΔL) by the neuromorphic device and the corresponding velocity vectors; andobtain the predefined perception stimulus matrix based on the correlation learned.

Images