Display control method and electronic device

Abstract

The application discloses a display control method and an electronic device. The method comprises the following steps: obtaining region coordinate data of a region where a picture changes in a display picture to be output, in a process of decoding the display picture to be output by using a picture decoder of a display device in a mode in which the display device displays a plurality of display regions at different refresh rates; controlling a refresh rate of the display device when the display device outputs the display picture according to the region coordinate data before the display device outputs the display picture; and the refresh rate of the region where the picture changes when the display picture is output in the display device is different from the refresh rate of other regions in the display device.

Description

Technical Field

[0001] This application relates to the field of display control technology, and in particular to a display control method and electronic device. Background Technology

[0002] Multi-frequency display (MFD) is a power-saving display technology that has been widely used in display devices. Its basic principle is to enable different display areas of the display device to work at different refresh rates. For example, the area of ​​the screen that does not change frequently is set to a low refresh rate, and the area that changes frequently is set to a high refresh rate, thereby reducing the overall power consumption of the display device.

[0003] Current MFD schemes typically rely on the operating system's (OS) display update region reporting mechanism. The OS calculates and reports the display update region by scanning pixel changes in the frame buffer, and the display device's timing controller (TCON) performs partition refresh control based on this display update region.

[0004] However, in this MFD scheme, the operating system needs to continuously and frequently calculate and report the updated display area during video playback, especially for high frame rate videos, which will lead to a significant increase in device power consumption. Summary of the Invention

[0005] In view of the above, this application provides a display control method and an electronic device, as follows: A method for controlling a display, comprising: When the display device is in a mode where multiple display areas are displayed at different refresh rates, the coordinate data of the areas in the display screen that have changed are obtained by using the display device's screen decoder to decode the display screen to be output. Before the display image is output to the display device, the refresh rate of the display device when outputting the display image is controlled according to the area coordinate data; The refresh rate of the area where the screen changes when the display image is output in the display device is different from the refresh rate of other areas in the display device.

[0006] Optionally, the above method involves obtaining the coordinate data of the area in the display screen to be output where the screen change occurs, including: Obtain the frame difference coordinates of each frame relative to the previous frame in a series of consecutive frames to be output by the display device; The frame difference coordinates are merged to obtain the region coordinate data corresponding to the consecutive multiple frames.

[0007] Optionally, the above method may further include: In response to the display device's screen output scenario, the number of screen frames corresponding to the area coordinate data is adjusted.

[0008] Optionally, the above method involves controlling the refresh rate of the display device when outputting the display image based on the region coordinate data, including: The region coordinate data is provided to the controller of the display device, which controls the display device to refresh the first region corresponding to the region coordinate data at a first refresh rate when the display screen is output, and controls the display device to refresh the second region at a second refresh rate when the display screen is output; The first region is different from the second region; the first refresh rate is greater than the second refresh rate.

[0009] Optionally, the above method involves controlling the refresh rate of the display device when outputting the display image based on the region coordinate data, including: Determine the relative positional relationship between the area coordinate data and the historical coordinate data; wherein, the historical coordinate data is the coordinate data used in the previous refresh rate control. If there is new coordinate data in the area coordinate data relative to the historical coordinate data, before the display screen is output in the display device, the controller of the display device is triggered to exit the mode of displaying multiple display areas at different refresh rates and re-enter the mode of displaying multiple display areas at different refresh rates according to the area coordinate data, so that the controller refreshes the first area corresponding to the area coordinate data at the first refresh rate when the display screen is output. If there is no new coordinate data for the area coordinate data relative to the historical coordinate data, the area coordinate data is provided to the controller so that the controller controls the display device to refresh the second area at a second refresh rate when the display screen is output. The first region is different from the second region; the first refresh rate is greater than the second refresh rate.

[0010] Optionally, before controlling the refresh rate of the display device when outputting the display image based on the area coordinate data, the above method further includes: The region coordinate data is transformed based on at least the desktop resolution of the display device, the screen resolution of the display screen, and the window coordinate data of the screen playback window of the display screen.

[0011] Optionally, the above method may involve performing coordinate transformation on the regional coordinate data, including: The scaling factor is obtained based on the window coordinate data of the playback window and the screen resolution of the displayed screen; Based on the scaling factor, the screen resolution, and the window coordinate data, the offset coordinate data of the displayed screen corresponding to the screen playback window is obtained; The region coordinate data is transformed based on the offset coordinate data, the scaling factor, and the window coordinate data.

[0012] Optionally, before controlling the refresh rate of the display device when outputting the display image based on the area coordinate data, the above method further includes: Using the center of the region corresponding to the region coordinate data as a reference point, the region coordinate data is expanded outward by at least one pixel value.

[0013] Optionally, in the above method, upon detecting an event that invokes or loads the screen decoder, the display device enters a mode in which the multiple display areas display at different refresh rates.

[0014] An electronic device, comprising: Display devices; The processor is configured to, during the process of decoding the display image to be output using the display device's image decoder when the display device is in a mode where multiple display areas are displayed at different refresh rates, obtain coordinate data of the areas where image changes occur in the display image to be output; and control the refresh rate of the display device when outputting the display image based on the area coordinate data before outputting the display image to the display device. The refresh rate of the area where the screen changes when the display image is output in the display device is different from the refresh rate of other areas in the display device.

[0015] As can be seen from the above technical solutions, the display control method and electronic device provided in this application reuse the area coordinate data obtained during the decoding process of the display screen by the screen decoder, and accordingly control the refresh rate of the display device when outputting the display screen before the display screen is output, so that the refresh rate of the screen change area is different from the refresh rate of other areas. Therefore, this application eliminates the need for the operating system to calculate the display update area after the display screen is output; instead, it predicts the area coordinate data and completes the refresh rate control in advance before the display screen is output, which can effectively reduce device power consumption and control latency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a display control method provided in an embodiment of this application; Figure 2 A partial flowchart of a display control method provided in an embodiment of this application; Figure 3 This is another part of a flowchart of a display control method provided in an embodiment of this application; Figure 4 This is another part of a flowchart illustrating a display control method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a display control device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 7 Example diagram of implementing MFD on a laptop desktop; Figure 8 This is an example diagram showing the accumulation of frame difference coordinates in a scenario applicable to laptops in this application; Figure 9 This is an example diagram of coordinate mapping in the scenario where this application applies to laptops; Figure 10 This is a flowchart illustrating the overall implementation of this application in the context of laptop computers. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] refer to Figure 1 This is a flowchart illustrating the implementation of a display control method provided in this application embodiment. This method can be applied to electronic devices with display devices, such as laptops or servers with screens. The technical solution in this embodiment is mainly used to reduce power consumption when the display device is in a mode where multiple display areas are displayed at different refresh rates.

[0020] Specifically, the method in this embodiment may include the following steps: Step 101: When the display device is in a mode where multiple display areas are displayed at different refresh rates, the coordinate data of the areas in the display screen to be output are obtained by using the display device's screen decoder to decode the display screen to be output.

[0021] Among them, display devices refer to the display components of electronic devices that have the function of displaying images. For example, display devices may include, but are not limited to, liquid crystal display panels (LCD panels) and organic light emitting diode panels (OLED panels).

[0022] It should be noted that a display device can have multiple display areas displayed at different refresh rates, known as MFD mode. A display device being in MFD mode means that this mode has been activated or enabled. In MFD mode, the screen of the display device is divided into at least two display areas, where the first area refreshes at a first refresh rate and the second area refreshes at a second refresh rate, with the first refresh rate differing from the second refresh rate.

[0023] Specifically, a change in the displayed image refers to a change in pixel values ​​between the current decoded frame and the reference frame (the previous frame). The area in the displayed image where a change occurs can be called the image change area, or the first area, which is the area the user focuses on most. The area in the displayed image where no image change occurs can be called the image still area, or the second area. Based on this, the first refresh rate is 120Hz, and the second refresh rate is 30Hz. The first area is refreshed at a high frequency of 120Hz, while the other areas in the second area are refreshed at a low frequency of 30Hz. By using a high refresh rate for the image change area and a low refresh rate for the image still area, the user's viewing experience in the image change area can be maintained while reducing the power consumption of the display device.

[0024] In one implementation, a screen decoder refers to a codec module used to decode displayed images, such as video streams, to reconstruct the video image. A screen decoder can be implemented in software (e.g., a decoding program running on a processor), in hardware (e.g., a dedicated decoding chip), or in a combination of both. In scenarios where a display device outputs a displayed image, the screen decoder decodes the image to be displayed.

[0025] Region coordinate data refers to the coordinate representation of the area in the displayed image where changes occur. Represented in the screen coordinate system, region coordinate data can specifically be a rectangular area defined by four parameters: the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the area, and the height of the area. Region coordinate data is obtained from intermediate data generated by the screen decoder during inter-frame decoding and can be considered as data inherent to the screen decoder.

[0026] For example, taking a video playback scenario as an example, the video decoder is decoding a 1920×1080 resolution video stream. During the decoding process, the processor in the electronic device obtains the area of ​​image change relative to the reference frame. The coordinate data of this area of ​​image change is generated by the video decoder by default or by the processor in the electronic device based on intermediate data calculated internally by the video decoder. In this embodiment, the area coordinate data is obtained during the decoding process of the video decoder. For example, the area of ​​image change within the video frame is obtained as a rectangle with the upper left corner coordinates (400, 400), a width of 800, and a height of 600.

[0027] Step 102: Before outputting the display image to the display device, control the refresh rate of the display device when outputting the display image based on the area coordinate data.

[0028] The period before the displayed image is output to the display device refers to the point in time before the image is actually displayed on the display panel. It should be noted that due to factors such as inconsistent decoding time for each frame by the image decoder, the need to buffer preceding and following prediction frames, and audio-visual synchronization requirements, the image decoder has a decoding buffer during the decoding process. Therefore, there is a time window between obtaining the area coordinate data and the actual output of the displayed image for that frame. In this embodiment, this time window can be used to execute the refresh rate control process in advance.

[0029] It should be noted that the refresh rate of the area where the image changes when the display is output is different from the refresh rate of other areas of the display device.

[0030] In one specific implementation, the refresh rate of the display device when outputting the display screen according to the area coordinate data in step 102 can be as follows: the screen change area defined by the area coordinate data is taken as the high-frequency refresh area, i.e., the first area, and the area is controlled to refresh at a higher first refresh rate, such as 120Hz; other areas in the display device other than the high-frequency refresh area are taken as the low-frequency refresh area, i.e., the second area, and the low-frequency refresh area is controlled to refresh at a lower second refresh rate, such as 30Hz.

[0031] For example, after obtaining the area coordinate data (400, 400, 800, 600) in the screen coordinate system, the area coordinate data is sent to the TCON. Before the display screen is output, the TCON controls the screen change area in the frame to correspond to 120Hz and controls other areas to correspond to 30Hz. When the frame screen is output, the screen change area in the frame screen is refreshed at 120Hz, and other areas in the frame screen are refreshed at 30Hz. Thus, it is not necessary to wait for the frame screen to be output before the operating system calculates the display update area. This not only reduces device power consumption, but also reduces the control latency of MFD by knowing the area coordinate data of the screen change area in advance.

[0032] As can be seen from the above technical solutions, the display control method provided in this application reuses the area coordinate data obtained during the decoding process of the display screen by the screen decoder, and controls the refresh rate of the display device when outputting the display screen based on this data, so that the refresh rate of the changing area of ​​the screen is different from the refresh rate of other areas. Therefore, this application eliminates the need for the operating system to calculate the display update area after the display screen is output; instead, it predicts the area coordinate data and completes the refresh rate control before the display screen is output, which can effectively reduce device power consumption and control latency.

[0033] based on Figure 1 In a specific implementation of the technical solution shown, step 101, when obtaining the coordinate data of the area where the screen change occurs in the display screen to be output in the display device, can be achieved in the following way: Figure 2 As shown: Step 201: Obtain the frame difference coordinates of each frame relative to the previous frame in the continuous multi-frame images to be output by the display device.

[0034] In this context, the continuous multi-frame images to be output in the display device refer to the continuous multi-frame video images that the image decoder is currently decoding but have not yet been output. Frame difference coordinates refer to the coordinate data of the area in each decoded frame that has undergone image changes relative to its previous reference frame, located in the image coordinate system. Specifically, frame difference coordinates are intermediate data generated by the image decoder during inter-frame prediction decoding. For example, in video codec standards (such as H.264, H.265, etc.), the image decoder performs motion estimation and residual calculation on each macroblock or coding tree unit (CTU) to determine whether the block has changed and to what extent. The motion vectors and residual information generated in this process themselves carry the positional information (specifically pixel coordinates) of the image change area. Therefore, frame difference coordinates are information directly obtained during the decoding process, without the need for additional pixel traversal or frame buffer comparison.

[0035] In one implementation, the specific methods for obtaining frame difference coordinates in this embodiment include, but are not limited to: The reconstructed area data structure in the image decoder is used to record the coordinates of all regions that have changed relative to the reference frame in the current decoded frame. The difference coordinates of each frame are directly read from this data structure to obtain the frame difference coordinates. The decoding mode of each macroblock in the monitoring screen decoder is determined, and the coordinates of all macroblocks decoded in non-skip mode are extracted and used as the frame difference coordinates of that frame. Deploy hook functions in the motion vector calculation code segment and / or prediction residual calculation code segment of the image decoder, and extract the difference coordinates of each frame by calling the hook function to obtain the frame difference coordinates; Frame difference coordinates are extracted from the decoding process of the image decoder using codec analysis tools.

[0036] For example, for a certain video, the difference coordinates of each decoded frame relative to the previous reference frame in 5 consecutive frames are as follows: the difference region of frame 1 is (100, 100, 300, 200), the difference region of frame 2 is (105, 98, 310, 205), the difference region of frame 3 is (95, 102, 305, 198), the difference region of frame 4 is (400, 400, 800, 600) (i.e., a new moving object appears in the frame), and the difference region of frame 5 is (400, 405, 802, 598).

[0037] Step 202: Perform coordinate merging on the frame difference coordinates to obtain the area coordinate data corresponding to multiple consecutive frames.

[0038] In one implementation, the coordinate merging process in this embodiment refers to merging the independent frame difference coordinates of multiple frames (obtained directly from the image decoder) according to a preset merging rule to generate a merged region coordinate that can cover all the changed areas corresponding to these frame difference coordinates, i.e., region coordinate data.

[0039] It should be noted that since the frame difference coordinates of a single frame may be small or scattered, directly using the frame difference coordinates of a single frame for MFD control may cause TCON to switch frequently. However, by merging multiple frames, a stable area of ​​image change with actual power saving benefits can be obtained.

[0040] Specifically, in this embodiment, the number of frames corresponding to multiple consecutive frames can be predetermined based on the requirements of MFD control and the power consumption of the display device. For example, if the user has high requirements for MFD control, i.e., needs precise MFD control, then the number of frames can be set to a smaller value. This avoids merging too many frame difference coordinates from multiple consecutive frames, thus ensuring precise control for each frame as much as possible. Conversely, if the user has high requirements for the display device's power consumption, i.e., needs to reduce power consumption, then the number of frames can be set to a larger value. This avoids excessively frequent MFD switching (changing the area coordinate data provided to the TCON), thus reducing the power consumption of the display device.

[0041] Alternatively, in this embodiment, the number of frames corresponding to multiple consecutive frames can be dynamically changed. Specifically, the number of frames in the coordinate merging process can be determined based on the proximity of the frame difference coordinates between adjacent frames. For example, if the difference regions of adjacent frames 1 to 3 are close, while the difference regions of adjacent frames 1 to 3 and frames 4 to 5 are far apart, then the frame difference coordinates of frames 1 to 3 are merged, and the frame difference coordinates of frames 4 to 5 are merged.

[0042] In one specific implementation, the coordinate merging process in this embodiment can employ a logical OR merging operation. This involves taking the union boundary of multiple rectangular regions formed by the frame difference coordinates of each frame to generate a minimum bounding rectangle that can cover all the rectangular regions to be merged, i.e., the frame change area. Specifically, the top-left x-coordinate of the merged minimum bounding rectangle is the minimum value among the top-left x-coordinates of all the rectangular regions to be merged; the top-left y-coordinate is the minimum value among the top-left y-coordinates of all the rectangular regions to be merged; the width of the minimum bounding rectangle is the maximum value among the bottom-right x-coordinates of all the rectangular regions to be merged minus the minimum value among the top-left x-coordinates; and the height of the minimum bounding rectangle is the maximum value among the bottom-right y-coordinates of all the rectangular regions to be merged minus the minimum value among the top-left y-coordinates.

[0043] For example, merging the frame difference coordinates of frames 1 to 5 can include: the difference regions of frames 1 to 3 are close together, and after merging, a bounding rectangle covering the difference regions of these three frames is obtained (95, 98, 310, 205); the difference regions of frames 4 and 5 are merged to obtain a bounding rectangle covering the change regions of these two frames (400, 400, 802, 602). Since the change regions of frames 1 to 3 and frames 4 to 5 are far apart, two sets of region coordinate data can be formed separately, or they can be further merged into a larger minimum bounding rectangle as needed.

[0044] In another implementation, the coordinate merging process in this embodiment can employ other merging algorithms such as weighted average or cluster merging, and this application does not limit this to any particular method.

[0045] As can be seen, by merging the frame difference coordinates of multiple consecutive frames in this embodiment, more stable and more power-saving regional coordinate data can be obtained, avoiding frequent exits and re-entries of MFD mode due to the small change area of ​​a single frame or frequent micro-movements in position, thereby reducing device power consumption.

[0046] Based on the above implementation scheme, this embodiment can also adjust the number of frame data corresponding to the area coordinate data in response to the screen output scene of the display device.

[0047] The output scenarios can include various types of applications such as video playback and office document processing. Different types of output scenarios have different changes in the frame difference coordinates between adjacent frames; that is, the changes in the change areas of a single frame differ between adjacent frames. Consequently, the number of consecutive frames involved in the coordinate merging process can vary.

[0048] For example, in video playback scenarios, since the changing area of ​​a single frame is usually quite drastic and continuously changing, a smaller number of frames (e.g., 2 or 3 frames) can be set so that the area coordinate data can follow the changes in the screen in a timely manner, avoiding stuttering caused by excessive deviation between the MFD area (the area of ​​screen change corresponding to the coordinate data provided to TCON) and the actual changing area.

[0049] For example, in office document scenarios (such as text editing, spreadsheet processing, etc.), since the screen update area is usually small and changes slowly, a larger number of screen frames (such as 5 or 10 frames) can be set to obtain a larger stable MFD area and further improve power saving benefits.

[0050] In one specific implementation, the identification of the screen output scene can be determined by detecting information such as the current foreground process type, window update frequency, or the output frame rate of the screen decoder. For example, if the current foreground process is detected to be a video player, it is determined to be a video playback scene, and the number of frames is set to 3 to improve display quality; if the current foreground process is detected to be a word processing software, it is determined to be an office document scene, and the number of frames is set to 8 to reduce device power consumption.

[0051] As can be seen, this embodiment can dynamically adjust the number of frame frames participating in coordinate merging processing according to the screen output scene, so as to balance the stability of the MFD area and the tracking sensitivity of the changing area in different scenes, thereby achieving a balance between power saving and display quality.

[0052] based on Figure 1In one specific implementation of the technical solution shown, in step 102, when controlling the refresh rate of the display device when outputting the display screen based on the area coordinate data, the area coordinate data can be provided to the controller of the display device. The controller controls the display device to refresh the first area corresponding to the area coordinate data at a first refresh rate when outputting the display screen, and controls the display device to refresh the second area at a second refresh rate when outputting the display screen.

[0053] The first region differs from the second region. The first region has a higher refresh rate than the second region.

[0054] It should be noted that the first and second regions can each be multiple non-contiguous sub-regions. For example, if there are two separate moving objects in the display screen, the region coordinate data can contain two independent rectangular regions, both of which are refreshed at the first refresh rate, while the remaining regions are refreshed at the second refresh rate.

[0055] In this context, the controller refers to the hardware component in a display device used to control the refresh timing of the display panel. An example of this is the timing controller (TCON). The timing controller is located on the driver circuit board of the display device and is responsible for receiving area coordinate data from the main control unit, converting it into the gate drive signals and source drive signals required by the display panel, so as to control different areas of the display panel to work according to their respective refresh rates, thereby achieving precise control of the refresh timing of each pixel.

[0056] In one implementation, this embodiment uses an I2C (inter-integrated circuit) bus to encode the region coordinate data into register instructions recognizable by the timing controller and send them to the timing controller. The I2C bus is a serial communication bus suitable for low-speed control instruction transmission between chips.

[0057] In another implementation, this embodiment can transmit the area coordinate data to the timing controller via a serial peripheral interface (SPI), a universal asynchronous receiver / transmitter (UART), or other means.

[0058] As can be seen, by directly providing the regional coordinate data to the timing controller and having it perform partition refresh control in this embodiment, hardware-level execution of refresh rate control can be achieved without the intervention of the operating system, further reducing power consumption and latency.

[0059] based on Figure 1In a specific implementation of the technical solution shown, step 102, when controlling the refresh rate of the display device to output the display image based on the area coordinate data, can be achieved in the following way: Figure 3 As shown: Step 301: Determine the relative positional relationship between the regional coordinate data and the historical coordinate data.

[0060] The historical coordinate data refers to the coordinate data used in the previous refresh rate control.

[0061] Specifically, historical coordinate data refers to the area coordinate data used during the previous execution of MFD control. Historical coordinate data is stored in the memory or registers of the display device.

[0062] The relative positional relationship between regional coordinate data and historical coordinate data is used to determine whether the current regional coordinate data has changed relative to historical coordinate data and the state of the change, such as zooming in or moving (new coordinate data), or zooming out (no new coordinate data). Specifically, the relative positional relationship can include, but is not limited to: regional coordinate data being exactly the same as historical coordinate data; regional coordinate data having a newly added coordinate area relative to historical coordinate data; regional coordinate data having a shift in coordinate values ​​relative to historical coordinate data; and regional coordinate data having a smaller area relative to historical coordinate data.

[0063] In one implementation, the method for determining the relative positional relationship in this embodiment can be as follows: calculate the overlap between the rectangular area (the area of ​​screen change) defined by the regional coordinate data and the rectangular area defined by the historical coordinate data. If the two rectangular areas completely overlap, it is determined that there is no new coordinate data; if the two rectangular areas partially overlap or do not overlap at all, it is determined that new coordinate data exists.

[0064] Step 302: When there is new coordinate data in the area coordinate data compared with the historical coordinate data, before outputting the display screen in the display device, the controller of the display device is triggered to exit the mode of displaying multiple display areas at different refresh rates and re-enter the mode of displaying multiple display areas at different refresh rates according to the area coordinate data, so that the controller refreshes the first area corresponding to the area coordinate data at the first refresh rate when the display screen is output.

[0065] In this case, "new coordinate data relative to historical coordinate data" means that the rectangular area defined by the new regional coordinate data exceeds the rectangular area defined by the historical coordinate data, or that the coordinate values ​​of the regional coordinate data have shifted relative to the historical coordinate data (e.g., the shift exceeds a preset threshold). In this situation, the original MFD area can no longer cover the area of ​​screen change during high refresh rate operation of the current TCON, and it is necessary to force exit the current MFD mode and then re-enter MFD mode according to the new regional coordinate data.

[0066] It should be noted that, since the time point at which the region coordinate data is obtained in this embodiment is earlier than the actual time point of the frame's output (based on the time difference of the decoding buffer), the operation of triggering the exit and re-entry of MFD mode can be completed before the frame's output, without causing display delay. This is in stark contrast to the multi-frame (e.g., more than 2 frames) delay caused by relying on the operating system to report the updated display region, only to learn of the changed region after the screen is displayed, and then exiting and re-entering MFD mode.

[0067] Step 303: If there is no new coordinate data for the area coordinate data relative to the historical coordinate data, the area coordinate data is provided to the controller so that the controller controls the display device to refresh the second area at the second refresh rate when the display screen is output.

[0068] The first region is different from the second region; the first refresh rate is greater than the second refresh rate.

[0069] It should be noted that "no new coordinate data" means that the area coordinate data is the same as the historical coordinate data, or that the rectangular area defined by the new area coordinate data falls entirely within the rectangular area defined by the historical coordinate data. In this case, the current MFD area remains valid, and there is no need to exit and re-enter MFD mode; simply continue to maintain partition refresh control using the current area coordinates.

[0070] For example, taking historical coordinate data as (95, 98, 310, 205) and current area coordinate data as (400, 400, 802, 602), since the two rectangular areas are far apart, it is determined that new coordinate data exists. Therefore, before the screen output, the timing controller is triggered to exit the current MFD mode (i.e., exit the MFD configuration with (95, 98, 310, 205) as the high refresh rate area), and then re-enter the MFD mode with (400, 400, 802, 602) as the new high refresh rate area. Since both the exit and re-entry operations are completed before the screen output, the user cannot perceive the switching delay at all.

[0071] As can be seen, this embodiment determines whether to exit and re-enter MFD mode by comparing regional coordinate data with historical coordinate data. This can ensure that the MFD area of ​​TCON accurately covers the changing areas of the screen while avoiding unnecessary mode switching, thus further improving display smoothness.

[0072] based on Figure 1 In one specific implementation of the technical solution shown, before controlling the refresh rate of the display device when outputting the display screen according to the area coordinate data in step 102, the area coordinate data can be transformed according to at least the desktop resolution of the display device, the screen resolution of the display screen, and the window coordinate data of the screen playback window of the display screen.

[0073] Desktop resolution refers to the total number of pixels displayed on the screen in desktop mode, including both horizontal and vertical pixels. For example, a desktop resolution of 3200×2000. Screen resolution refers to the original resolution of the displayed content, i.e., the pixel size of the displayed image output by the screen decoder. For example, a screen resolution of 1920×1080. Window coordinate data refers to the position and size information of the playback window on the desktop of the display device, including the horizontal and vertical coordinates of the top-left corner of the window, and the width and height of the window. For example, the top-left corner coordinates of the window are (600, 600), the window width is 1800, and the window height is 1500.

[0074] In one implementation, the coordinate transformation in this embodiment refers to converting the region coordinate data obtained during the decoding process of the image decoder into region coordinate data in the screen coordinate system of the display device. Since the region coordinate data obtained during the decoding process is relative to the displayed image itself (i.e., the video coordinate system), and the display device needs to display the image at a specific window position on the screen, and the window size may differ from the original resolution of the displayed image (image resolution) (due to scaling), coordinate transformation is necessary.

[0075] Specifically, the implementation method for coordinate transformation of regional coordinate data in this embodiment can be referred to... Figure 4 As shown: Step 401: Obtain the scaling factor based on the window coordinates of the playback window and the screen resolution of the display screen.

[0076] The scaling factor can be calculated as follows: calculate the ratio of the window size of the playback window to the screen resolution of the display screen in the horizontal direction and the ratio in the vertical direction, and take the smaller value of the two as the scaling factor, as shown in formula (1).

[0077] Scaling factor = min(window width / screen width, window height / screen height) (1)

[0078] For example, taking a window with coordinates (1800, 1500) and a screen resolution of 1920×1080 as an example, the scaling factor is min(1800 / 1920, 1500 / 1080), that is, min(0.9375, 1.3889) = 0.9375.

[0079] It should be noted that the use of a smaller value (min) in this embodiment can ensure that the display image is fully displayed in the playback window while maintaining the original aspect ratio, thus avoiding the display image being stretched and distorted.

[0080] Step 402: Based on the scaling factor, screen resolution, and window coordinate data, obtain the offset coordinate data of the displayed screen in the screen playback window.

[0081] In one implementation, the offset coordinate data in this embodiment can be calculated as follows: First, calculate the actual display size of the screen within the playback window based on the scaling factor. For example, the video display width is the screen width × scaling factor, and the video display height is the screen height × scaling factor. Then, calculate the black border offset within the playback window caused by maintaining the aspect ratio, which is the offset coordinate data. For example, the horizontal axis offset is (window width - video display width) / 2, and the vertical axis offset is (window height - video display height) / 2.

[0082] Among them, the offset coordinate data can reflect the offset state of the effective display area of ​​the screen relative to the upper left corner of the screen playback window.

[0083] Step 403: Perform coordinate transformation on the region coordinate data based on the offset coordinate data, scaling factor, and window coordinate data.

[0084] In one implementation, the specific calculation method for coordinate transformation in this embodiment may include: First, multiply the x-coordinate of the top left corner of the region coordinate data by the scaling factor, add the x-axis offset, and then add the x-coordinate of the top left corner of the playback window to obtain the x-coordinate of the top left corner in the screen coordinate system. Then, multiply the y-coordinate of the top left corner of the region coordinate data by the scaling factor, add the y-axis offset, and then add the y-coordinate of the top left corner of the playback window to obtain the y-coordinate of the top left corner in the screen coordinate system. At the same time, the width of the region coordinate data is multiplied by the scaling factor to obtain the region width in the screen coordinate system; and the height of the region coordinate data is multiplied by the scaling factor to obtain the region height in the screen coordinate system.

[0085] For example, taking a desktop resolution of 3200×2000, a window with a top-left corner (600, 600), a width of 1800, a height of 1500, a video resolution of 1920×1080, and the area coordinates (i.e., the display update area) (400, 400, 800, 600) as an example, with a scaling factor of 0.9375, a video display size of 1800 width and 1012.5 height, and a black border offset of 0 on the horizontal axis and (1500-1012.5) / 2=243.75 on the vertical axis; the coordinates mapped to the display window are (375, 618.75, 750, 562.5); and the coordinates mapped to the desktop are (975, 1218.75, 750, 562.5), which are the area coordinates in the screen coordinate system.

[0086] As can be seen, this embodiment, through precise coordinate transformation, can accurately map the coordinates of regional changes to the screen coordinate system, ensuring that the TCON can perform partition refresh control at the correct screen position. Simultaneously, by eliminating black border areas during coordinate mapping, only the changing areas within the actual display area are used as high-frequency refresh targets, further narrowing the high-frequency refresh range and thus improving power saving benefits.

[0087] based on Figure 1 In one specific implementation of the technical solution shown, before controlling the refresh rate of the display device when outputting the display screen according to the regional coordinate data in step 102, the regional coordinate data can be expanded by at least one pixel value with the center of the region corresponding to the regional coordinate data as the reference point.

[0088] In this embodiment, the outward expansion refers to extending the boundary of the rectangular area defined by the region coordinate data outward by several pixels, so that the final MFD region (i.e., the area of ​​image change) is slightly larger than the actual area of ​​image change corresponding to the region coordinate data obtained by the image decoder. The purpose of outward expansion is to prevent the MFD region from failing to completely cover the actual area of ​​image change due to factors such as coordinate rounding errors, loss of scaling calculation accuracy, or slight deviations between the boundary of the area of ​​image change detected by the image decoder and the actual visual boundary of image change, thereby causing image tearing or stuttering and improving the accuracy of MFD control.

[0089] In one implementation, the specific method of the outward expansion processing in this embodiment can be as follows: the horizontal coordinate of the upper left corner of the region coordinate data is shifted to the left by N units (pixels), the vertical coordinate of the upper left corner is shifted upward by N units, and the region width and region height are each expanded outward by 2N units. That is, while keeping the region center unchanged, each of the four sides of the rectangular region is expanded outward by N units.

[0090] Where N is a positive integer greater than or equal to 1.

[0091] For example, the area coordinate data under the desktop coordinates includes (975, 1218.75, 750, 562.5). After rounding the coordinate values ​​in the area coordinate data and shifting the top left corner coordinate by 1 pixel to the left and 1 pixel up, and expanding the width and height by 2 pixels each, the expanded area coordinates are (974, 1217, 752, 565).

[0092] Specifically, in this embodiment, the expanded pixel value can be dynamically adjusted based on the physical characteristics of the display device (such as pixel density and response time) or the output accuracy of the current screen decoder. For example, for a high pixel density screen with 4K resolution, a larger expansion value (such as N=3 or 4) can be set to ensure coverage of all possible micro-variable areas.

[0093] As can be seen, by expanding the regional coordinate data in this embodiment, the problem of incomplete MFD region coverage caused by calculation accuracy or detection error can be effectively prevented, thus ensuring the display quality in MFD mode.

[0094] In one implementation, the display device can enter MFD mode (a mode in which multiple display areas are displayed at different refresh rates) by detecting an event that calls or loads the screen decoder.

[0095] Specifically, when the operating system or application requests the creation of an instance of the screen decoder, it sends a screen decoder creation event, which is used to invoke or load the screen decoder. The MFD control module deployed in the processor of the electronic device can learn that the display screen is about to start outputting by hooking this event, thereby activating the MFD mode.

[0096] In another implementation, the display device can enter MFD mode in any one or more combinations of the following methods: Detect whether the current foreground process is a browser process or a media player process; Check if the current window class name matches the characteristics of a video playback window, for example, if the window class name is "VideoWindow" or "Chrome_WidgetWin_1"; Detect whether the operating system receives a system power request to prevent the display device from hibernating (i.e., a video playback anti-hibernation request). Listen for basic media session events related to video playback; Obtain the current system-level media playback status information through the global system media control interface, for example, by obtaining the playback status through GlobalSystemMediaTransportControlsSessionManager.

[0097] The above methods do not involve the operating system reading pixels or performing mathematical operations on the display device screen content, so the power consumption is extremely low.

[0098] As can be seen, in this embodiment, the MFD mode is triggered by detecting decoder call or loading events. This allows multiple display areas to be activated with different refresh rates before video playback begins, enabling advance preparation of the MFD mode and further avoiding mode switching delays.

[0099] refer to Figure 5 This is a schematic diagram of a display control device provided in an embodiment of this application. This device can be deployed in the processor of an electronic device, such as an MFD control module. The technical solution in this embodiment is mainly used to reduce the power consumption of the display device when it is in a mode where multiple display areas are displayed at different refresh rates.

[0100] Specifically, the apparatus in this embodiment may include the following units: The area acquisition unit 501 is used to obtain the coordinate data of the area in the display screen where the screen changes when the display device is in a mode where multiple display areas are displayed at different refresh rates, by using the screen decoder of the display device to decode the display screen to be output. The display control unit 502 is used to control the refresh rate of the display device when outputting the display image based on the area coordinate data before the display image is output to the display device. The refresh rate of the area where the screen changes when the display image is output in the display device is different from the refresh rate of other areas in the display device.

[0101] As can be seen from the above technical solutions, the display control device provided in this application reuses the area coordinate data obtained during the decoding process of the display screen by the screen decoder, and controls the refresh rate of the display device when outputting the display screen based on this data, so that the refresh rate of the changing area of ​​the screen is different from the refresh rate of other areas. Therefore, this application eliminates the need for the operating system to calculate the display update area after the display screen is output; instead, it predicts the area coordinate data and completes the refresh rate control before the display screen is output, which can effectively reduce device power consumption and control latency.

[0102] In one implementation, the region acquisition unit 501 is specifically used to: obtain the frame difference coordinates of each frame relative to the previous frame in a series of consecutive frames to be output by the display device; and perform coordinate merging processing on the frame difference coordinates to obtain the region coordinate data corresponding to the series of consecutive frames.

[0103] Furthermore, the region acquisition unit 501 is also used to: adjust the number of frame frames corresponding to the region coordinate data in response to the screen output scene of the display device.

[0104] In one implementation, the display control unit 502 is specifically used to: provide the area coordinate data to the controller of the display device, and the controller controls the display device to refresh the first area corresponding to the area coordinate data at a first refresh rate when the display screen is output, and controls the display device to refresh the second area at a second refresh rate when the display screen is output; wherein, the first area is different from the second area; and the first refresh rate is greater than the second refresh rate.

[0105] In one implementation, the display control unit 502 is specifically configured to: determine the relative positional relationship between the area coordinate data and historical coordinate data; wherein the historical coordinate data is the coordinate data used in the previous refresh rate control; when there is new coordinate data in the area coordinate data relative to the historical coordinate data, before outputting the display screen in the display device, trigger the controller of the display device to exit the mode of displaying multiple display areas at different refresh rates and re-enter the mode of displaying multiple display areas at different refresh rates according to the area coordinate data, so that the controller refreshes the first area corresponding to the area coordinate data at a first refresh rate when the display screen is output; when there is no new coordinate data in the area coordinate data relative to the historical coordinate data, provide the area coordinate data to the controller, so that the controller controls the display device to refresh the second area at a second refresh rate when the display screen is output; wherein the first area is different from the second area; the first refresh rate is greater than the second refresh rate.

[0106] In one implementation, before controlling the refresh rate of the display device when outputting the display screen based on the area coordinate data, the display control unit 502 is further configured to: perform coordinate transformation on the area coordinate data based at least on the desktop resolution of the display device, the screen resolution of the display screen, and the window coordinate data of the screen playback window of the display screen.

[0107] Specifically, when performing coordinate transformation on the area coordinate data, the display control unit 502 is used to: obtain a scaling factor based on the window coordinate data of the playback window and the screen resolution of the display screen; obtain the offset coordinate data of the display screen corresponding to the playback window based on the scaling factor, the screen resolution, and the window coordinate data; and perform coordinate transformation on the area coordinate data based on the offset coordinate data, the scaling factor, and the window coordinate data.

[0108] In one implementation, before controlling the refresh rate of the display device when outputting the display screen based on the area coordinate data, the display control unit 502 is further configured to: expand the area coordinate data by at least one pixel value, using the center of the area corresponding to the area coordinate data as a reference point.

[0109] In the event that the screen decoder is invoked or loaded, the display device enters a mode in which the multiple display areas are displayed at different refresh rates.

[0110] It should be noted that the specific implementation of each unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0111] refer to Figure 6 The present application provides a schematic diagram of the structure of an electronic device, which may include the following structures: Display device 601 is used to display images, examples of which include, but are not limited to, liquid crystal display panels and organic light-emitting diode display panels.

[0112] The processor 602, coupled to the display device 601 (e.g., connected via a display driver circuit or timing controller), is used to obtain the coordinate data of the areas in the display screen where screen changes occur during the decoding process of the display screen to be output using the display device's screen decoder when the display device 601 is in a mode where multiple display areas are displayed at different refresh rates; and to control the refresh rate of the display device 601 when outputting the display screen based on the coordinate data before the display screen is output. Wherein, when the display device 601 outputs the display image, the refresh rate of the area where the image changes is different from the refresh rate of other areas in the display device 601.

[0113] As can be seen from the above technical solution, the electronic device provided in this application reuses the area coordinate data obtained during the decoding process of the display screen by the screen decoder, and controls the refresh rate of the display device when outputting the display screen based on this data, so that the refresh rate of the changing area of ​​the screen is different from the refresh rate of other areas. Therefore, this application eliminates the need for the operating system to calculate the display update area after the display screen is output; instead, it predicts the area coordinate data and completes the refresh rate control before the display screen is output, which can effectively reduce device power consumption and control latency.

[0114] Taking a laptop computer as an example, the technical solution of this application is illustrated below: Current power-saving solutions for common panel MFDs rely on the operating system's Display Update Region. Existing processor manufacturers' MFD mechanisms also use this approach. One manufacturer's MFD achieves the MFD effect by directly controlling the TCON using I2C bus instructions, such as... Figure 7 As shown.

[0115] Currently, there are two problems: 1. MFD requires a stable region to function properly. In office software scenarios (such as Office), the display update region in the display device is generally stable, and the update frame rate is low, ensuring that it can enter MFD state most of the time, which can save power. However, in video playback scenarios, the display update region changes drastically. Once the changed region exceeds the range of the previously set MFD region, TCON needs to force exit MFD mode and re-enter. If the MFD region is not changed, persistent screen stuttering will occur; if exiting MFD mode, it is necessary to wait for the region to stabilize and recalculate before re-entering MFD mode. Since the changed region is prepared for display before the display update region changes, the changed region needs to be merged by an algorithm, and sending the update coordinates to TCON via the I2C bus incurs time overhead. Therefore, exiting MFD requires a display delay of approximately two frames or more.

[0116] 2. In the Video Playback scenario, the system relies on the operating system to report the Region. This Region is calculated by the operating system, which leads to increased power consumption for high frame rate videos.

[0117] To address the above problems, the technical solution proposed in this application is as follows: An optimization technique based on a video codec (MCC) is proposed to improve the power-saving benefits of MFD (Multi-Function Display). This technique eliminates the need for the operating system to report display update areas, allowing for advance prediction of display update areas and timely MFD switching, thereby eliminating display latency during switching. Furthermore, it avoids the operating system repeatedly calculating display update areas, reducing device power consumption. Specifically, the following solutions are included: 1. Utilize decoder caching. Due to the varying decoding time for each frame, the need to buffer preceding and following prediction frames, and the requirement for audio-video synchronization, frame difference coordinates are retained during decoding. These frame difference coordinates are intermediate data generated by the decoder by default, requiring no additional computational load. They are accumulated in groups, such as... Figure 8 As shown in the figure, the final difference coordinates are obtained.

[0118] 2. Map the decoder's display update area to the area displayed on the screen. The video update area needs to be mapped to the location where the windowed video is playing on the desktop, such as... Figure 9 As shown in the diagram, the calculation process for coordinate mapping is as follows: First, desktop resolution: 3200×2000; window position and size: top left corner (600, 600), width 1800, height 1500; original video resolution: 1920×1080; video display update area: top left corner (400, 400), width 800, height 600; based on this: (1) Scaling factor: min(1800 / 1920, 1500 / 1080) = min(0.9375, 1.3889) = 0.9375; (2) Video display area dimensions: width 1920×0.9375=1800, height 1080×0.9375=1012.5; (3) Offset of the video display area within the window (top left corner): offset_x=(1800-1800) / 2=0, offset_y=(1500-1012.5) / 2=243.75; (4) Display the updated area mapped to coordinates within the window: x_win = 0 + 400 × 0.9375 = 375; y_win=243.75+400×0.9375=618.75; w_win = 800 × 0.9375 = 750; h_win = 600 × 0.9375 = 562.5; (5) Map to desktop coordinates (including the top left corner of the window): x_desk = 600 + 375 = 975; y_desk = 600 + 618.75 = 1218.75; w_desk=750, h_desk=562.5; (6) The updated area rectangle on the desktop is (975, 1218.75, 750, 562.5), and to ensure accuracy, it is expanded to (974, 1217, 750, 563).

[0119] 3. When the final difference coordinates need to be updated in the decoder, notify TCON in advance to switch MFD.

[0120] refer to Figure 10 The diagram shown is a flowchart of the overall implementation of this application, which specifically includes the following steps: Step 1: The application opens the video playback application and enters the video playback scene; Step 2: Determine that an instance of the codec has been created.

[0121] Step 3: Read the frame difference coordinates from the video decoder and obtain the area coordinate data of the changed area of ​​the screen accordingly.

[0122] Step 4: Map the region coordinate data to the screen coordinate system.

[0123] Step 5: Notify TCON in advance via I2C bus to perform MFD switching.

[0124] Step 6: Before the actual output of the display screen, TCON controls the corresponding area to refresh at a high frequency and controls other areas to refresh at a low frequency based on the area coordinate data.

[0125] In summary, the technical solution of this application has the following advantages: 1. Predict the display update area in advance and notify TCON to update the MFD area before displaying, eliminating the delay during switching.

[0126] 2. The decoder can calculate the display update area on its own, without the need for operating system calculations, thus reducing device power consumption.

[0127] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0128] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0130] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for controlling a display, comprising: When the display device is in a mode where multiple display areas are displayed at different refresh rates, the coordinate data of the areas in the display screen that have changed are obtained by using the display device's screen decoder to decode the display screen to be output. Before the display image is output to the display device, the refresh rate of the display device when outputting the display image is controlled according to the area coordinate data; The refresh rate of the area where the screen changes when the display image is output in the display device is different from the refresh rate of other areas in the display device.

2. The method according to claim 1, obtaining the coordinate data of the area where the screen change occurs in the display screen to be output in the display device, includes: Obtain the frame difference coordinates of each frame relative to the previous frame in a series of consecutive frames to be output by the display device; The frame difference coordinates are merged to obtain the region coordinate data corresponding to the consecutive multiple frames.

3. The method according to claim 2, further comprising: In response to the display device's screen output scenario, the number of screen frames corresponding to the area coordinate data is adjusted.

4. The method according to claim 1 or 2, wherein controlling the refresh rate of the display device when outputting the display image based on the region coordinate data includes: The region coordinate data is provided to the controller of the display device, which controls the display device to refresh the first region corresponding to the region coordinate data at a first refresh rate when the display screen is output, and controls the display device to refresh the second region at a second refresh rate when the display screen is output; The first region is different from the second region; the first refresh rate is greater than the second refresh rate.

5. The method according to claim 1 or 2, wherein controlling the refresh rate of the display device when outputting the display image based on the region coordinate data includes: Determine the relative positional relationship between the area coordinate data and the historical coordinate data; wherein, the historical coordinate data is the coordinate data used in the previous refresh rate control. If there is new coordinate data in the area coordinate data relative to the historical coordinate data, before the display screen is output in the display device, the controller of the display device is triggered to exit the mode of displaying multiple display areas at different refresh rates and re-enter the mode of displaying multiple display areas at different refresh rates according to the area coordinate data, so that the controller refreshes the first area corresponding to the area coordinate data at the first refresh rate when the display screen is output. If there is no new coordinate data for the area coordinate data relative to the historical coordinate data, the area coordinate data is provided to the controller so that the controller controls the display device to refresh the second area at a second refresh rate when the display screen is output. The first region is different from the second region; the first refresh rate is greater than the second refresh rate.

6. The method according to claim 1, 2, or 3, before controlling the refresh rate of the display device when outputting the display image based on the area coordinate data, the method further includes: The region coordinate data is transformed based on at least the desktop resolution of the display device, the screen resolution of the display screen, and the window coordinate data of the screen playback window of the display screen.

7. The method according to claim 6, wherein coordinate transformation is performed on the regional coordinate data, comprising: The scaling factor is obtained based on the window coordinate data of the playback window and the screen resolution of the displayed screen; Based on the scaling factor, the screen resolution, and the window coordinate data, the offset coordinate data of the displayed screen corresponding to the screen playback window is obtained; The region coordinate data is transformed based on the offset coordinate data, the scaling factor, and the window coordinate data.

8. The method according to claim 1 or 2, before controlling the refresh rate of the display device when outputting the display image based on the area coordinate data, the method further includes: Using the center of the region corresponding to the region coordinate data as a reference point, the region coordinate data is expanded outward by at least one pixel value.

9. The method according to claim 1, wherein upon detecting an event of calling or loading the screen decoder, the display device enters a mode in which multiple display areas are displayed at different refresh rates.

10. An electronic device, comprising: Display devices; The processor is configured to obtain coordinate data of the area in the display screen where the screen changes during the decoding process of the display screen to be output using the screen decoder of the display device when the display device is in a mode where multiple display areas are displayed at different refresh rates. Before the display image is output to the display device, the refresh rate of the display device when outputting the display image is controlled according to the area coordinate data; The refresh rate of the area where the screen changes when the display image is output in the display device is different from the refresh rate of other areas in the display device.

Images