A multi-screen refresh system, method and storage medium

Abstract

This application provides a multi-screen refresh system, method, and storage medium. The multi-screen refresh system includes N display refresh links. Each display refresh link includes at least an image drawing module, an image compositing module, and a display module. The image drawing module draws preset image data and sends the preset image data to the image compositing module. The image compositing module, in response to an image refresh signal, performs image compositing processing on the preset image data to obtain image display data and transmits the image display data to the display module for display. This application enables different display modules to perform image rendering at the optimal refresh rate, ensuring the smoothness of each display module, improving the user experience, and reducing power consumption.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a multi-screen refresh system, method and storage medium. Background Technology

[0002] With the increasing popularity of new energy vehicles, the intelligence and entertainment features of cockpits are becoming more and more apparent, and the number of screens in the cockpit is also increasing. In addition to the instrument panel and central control screen, there are also large screens such as ceiling-mounted screens and rear-seat screens.

[0003] Different screens may have different resolutions and refresh rates. Apart from the instrument cluster screen, the other screens are entertainment screens, rendered by the central control unit. The central control unit typically runs on Android and uses SurfaceFlinger for screen rendering. However, the native SurfaceFlinger uses serial rendering for multiple screens, resulting in lower refresh rates and potentially causing noticeable stuttering and other user experience issues.

[0004] Furthermore, Android's native rendering mechanism prioritizes the main screen's refresh rate, and refreshes other secondary screens according to the main screen's refresh rate. This results in inconsistent refresh rates, which can lead to stuttering and higher power consumption on screens with different refresh rates. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a multi-screen refresh system, method, and storage medium that can satisfy different displays using the optimal refresh rate for rendering.

[0006] Specifically, this application provides a multi-screen refresh system, characterized in that it includes at least N display screen refresh links, where N≥2.

[0007] Each display refresh link includes at least an image rendering module, an image compositing module, and a display module.

[0008] The image drawing module is used to draw preset image data and send the preset image data to the image synthesis module.

[0009] The image synthesis module is used to perform image synthesis processing on the preset image data in response to the image refresh signal to obtain image display data, and transmit the image display data to the display screen module for display.

[0010] The multi-screen refresh system is equipped with an image rendering module and an image compositing module on each display refresh link, enabling different display modules to render images at the optimal refresh rate, ensuring the smoothness of each display module, improving the user experience, and reducing power consumption.

[0011] The image drawing module includes a first image drawing unit and a second image drawing unit.

[0012] The first image drawing unit is used to send an image drawing signal to the second drawing unit.

[0013] The second image drawing unit is used to draw preset image data in response to the image drawing signal, and send the preset image data to the image synthesis module.

[0014] Separating the module responsible for signal input from the module responsible for image rendering can prevent the module responsible for image rendering from being blocked when handling complex rendering tasks, further ensuring the smoothness and responsiveness of the display module and improving the user experience.

[0015] The display refresh link also includes a display driver module and an image transmission module.

[0016] The display screen driver module is used to transmit the preset image signal sent by the display screen module to the image transmission module, and is also used to transmit image display data to the display screen module.

[0017] The image transmission module is used to convert the preset image signal into an image refresh signal and transmit the image refresh signal to the image synthesis module. It is also used to transmit the image display data from the image synthesis module to the display screen driver module.

[0018] The preset image signal is a pulse signal, and the image refresh signal is a VSYNC signal. Converting the pulse signal emitted by the display module into a VSYNC signal and transmitting it to the upper layer can precisely control the display refresh time, ensuring accurate data transmission and normal operation of the display module.

[0019] The image display data is transmitted to the display module sequentially through the image transmission module and the display driver module, which improves the image display efficiency and enhances the system's response speed and stability.

[0020] The display refresh link also includes an application module.

[0021] The application module runs on the display module, and both the first image drawing unit and the second image drawing unit are located in the application module.

[0022] The first image drawing unit and the second image drawing unit are actually two internal threads in the application module. They work together to improve the smoothness and responsiveness of the display module interface, thereby achieving flexible interactive effects for the display module and improving system performance and user experience.

[0023] The first image drawing unit, the second image drawing unit, and the image synthesis module in each display refresh link are sequentially connected to and accessed by the same image transmission module. The other end of the image transmission module is connected to a display driving module, and the other end of the display driving module is connected to the display module in each display refresh link.

[0024] Through the above connection methods, the entire display refresh chain realizes a complete image processing and transmission process from the application to the final display area. With the role and cooperation of each module, it ensures the accuracy, smoothness and quality of image rendering, and provides users with the final visual effect.

[0025] Setting up multiple such display refresh links allows for simultaneous refresh and display on multiple displays, supports multi-channel signal transmission, satisfies multi-screen refresh requirements, and provides more flexible display capabilities.

[0026] Based on the same concept, this application also provides a multi-screen refresh method, applied to the aforementioned multi-screen refresh system, the method comprising:

[0027] S10: The image drawing module draws preset image data and sends the preset image data to the image synthesis module.

[0028] S20: In response to the image refresh signal, the image synthesis module performs image synthesis processing on the preset image data to obtain image display data, and transmits the image display data to the display screen module for display.

[0029] The multi-screen refresh method enables different display modules to render images at the optimal refresh rate, ensuring the smoothness of each display module, improving the user experience, and reducing power consumption.

[0030] Step S10 includes:

[0031] The first image drawing unit sends an image drawing signal to the second drawing unit.

[0032] The second image drawing unit responds to the image drawing signal, draws preset image data, and sends the preset image data to the image synthesis module.

[0033] By dividing tasks and collaborating on a collaborative and hierarchical basis, the efficiency and quality of image rendering have been improved.

[0034] Before performing step S20, the following are included:

[0035] The display driver module transmits the preset image signal sent by the display module to the image transmission module.

[0036] The display driver module is used to transmit preset image signals, which unifies the data transmission interface.

[0037] The image transmission module converts the preset image signal into an image refresh signal and transmits the image refresh signal to the image synthesis module.

[0038] The preset image signal is a pulse signal, and the image refresh signal is a VSYNC signal. Converting the pulse signal to a VSYNC signal can ensure the synchronization of the image and the display module, improve image quality, avoid rendering delay, and ensure smooth image display.

[0039] In step S20, the image display data is transmitted sequentially to the display module through the image transmission module and the display driver module.

[0040] This image transmission method can further improve image display efficiency.

[0041] This application also provides a storage medium, which is a type of computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions for causing a computer to execute the multi-screen refresh method described above.

[0042] Compared with the prior art, the beneficial effects of this application are as follows:

[0043] This application achieves optimal image rendering at the best refresh rate for different display modules by setting up image rendering and image compositing modules on each display refresh link. This ensures the smoothness of each display module, improves user experience, and reduces power consumption. It solves the technical problem of low refresh efficiency, display stuttering, and high power consumption on other secondary screens due to the native rendering mechanism prioritizing the main screen's refresh rate. Attached Figure Description

[0044] Figure 1 This is a framework diagram of the multi-screen refresh system described in this application.

[0045] Figure 2 For application Figure 1 The flowchart of the multi-screen refresh system described above. Detailed Implementation

[0046] This application provides a multi-screen refresh system, method, and storage medium to solve the technical problem in the prior art where the native rendering mechanism relies primarily on the refresh rate of the main screen, resulting in low refresh efficiency of other secondary screens, screen stuttering, and high power consumption.

[0047] The following describes in further detail a multi-screen refresh system, method, and storage medium of this application with reference to specific embodiments and accompanying drawings.

[0048] Example 1:

[0049] Please see Figure 1 This application provides a multi-screen refresh system, characterized in that it includes at least N display screen refresh links, where N≥2.

[0050] Users need to modify the native rendering mechanism to achieve multi-screen refresh effects. The steps are as follows:

[0051] Change SurfaceFlinger (the image compositing module described below) to a multi-process model, so that each process refreshes one display screen.

[0052] Modify the HardwareComposer HAL (i.e., the image transmission module below) to a single-process multi-instance configuration, so that each instance corresponds to a display screen and generates the corresponding VSYNC signal.

[0053] And modify the Render Thread (i.e., the second drawing unit below) to send the image drawing data submitted by the APP to the SurfaceFlinger of the corresponding display screen.

[0054] Each display refresh link includes at least an image rendering module, an image compositing module, and a display module.

[0055] The image drawing module is used to draw preset image data and send the preset image data to the image synthesis module.

[0056] The image synthesis module is used to perform image synthesis processing on the preset image data in response to the image refresh signal to obtain image display data, and transmit the image display data to the display screen module for display.

[0057] The image drawing module includes a first image drawing unit and a second image drawing unit.

[0058] In this embodiment, the first image drawing unit is equivalent to the UI Thread, and the second image drawing unit is equivalent to the Render Thread.

[0059] The display refresh link also includes an application module; the application module runs on the display module, and both the first image drawing unit and the second image drawing unit are located in the application module.

[0060] It should be noted that both the UI Thread and the Render Thread are threads within the application module (i.e., the APP). They work together to draw the UI element views of the APP and transmit them to SurfaceFlinger.

[0061] The first image drawing unit is used to send an image drawing signal to the second drawing unit.

[0062] The first image rendering unit (i.e., UI Thread) is a user interface thread. When the UI Thread updates the view of the relevant UI element, it submits the corresponding drawing commands (image drawing signals) to the second rendering unit (i.e., RenderThread).

[0063] The second image drawing unit is used to draw preset image data in response to the image drawing signal, and send the preset image data to the image synthesis module.

[0064] When the second drawing unit receives the drawing command, it will draw the UI element view (preset image data) required by the corresponding APP and transmit the drawn UI element view to SurfaceFlinger.

[0065] The display refresh link also includes a display driver module and an image transmission module.

[0066] The display driver module is equivalent to DisplayDriver, and the image transmission module is equivalent to HardwareComposer HAL.

[0067] DisplayDriver is a type of display driver that can provide functions such as transmission, display settings, display mode management, parameter calibration, fault diagnosis, and power saving management. In this embodiment, the main function of DisplayDriver is transmission.

[0068] HardwareComposer (HAL) is a type of hardware compositor that provides a unified interface between applications and displays.

[0069] The display screen driver module is used to transmit the preset image signal sent by the display screen module to the image transmission module, and is also used to transmit image display data to the display screen module.

[0070] The DisplayDriver receives pulse signals (i.e., the preset image signals) sent by the display module and transmits them to the image transmission module, and is responsible for transmitting upper-layer data (i.e., the image display data) to the display module.

[0071] The image transmission module is used to convert the preset image signal into an image refresh signal and transmit the image refresh signal to the image synthesis module. It is also used to transmit the image display data from the image synthesis module to the display screen driver module.

[0072] The preset image signal is a pulse signal. In HardwareComposer HAL, the pulse signal transmitted from DisplayDriver is received and converted into a VSYNC signal (i.e., the image refresh signal) and transmitted to the upper layer (image compositing module).

[0073] In response to the VSYNC signal, SurfaceFlinger (i.e. the image compositing module) performs compositing processing on the UI element view to obtain a UI display view for display on the display module, and transmits the UI display view to the display module for display.

[0074] The first image drawing unit, the second image drawing unit, and the image synthesis module in each display refresh link are sequentially connected to and accessed by the same image transmission module. The other end of the image transmission module is connected to a display driving module, and the other end of the display driving module is connected to the display module in each display refresh link.

[0075] The first image drawing unit and the second image drawing unit within each APP are connected in sequence and then connected to the corresponding image compositing module. Each image compositing module is connected to the same image transmission module. The image transmission module is then connected to a unified interface, namely the display screen driver module. The display screen driver module is then connected to the display screen module corresponding to each APP.

[0076] For example, the current multi-screen refresh system has 3 display refresh links, and the refresh rates of the display modules on each link are 90fps, 60fps and 120fps respectively; the multi-screen refresh system can realize that each display module refreshes the image at its optimal refresh rate, ensuring the smoothness of the image display of each display module.

[0077] Example 2:

[0078] Please see Figure 2 This application also provides a multi-screen refresh method, applied to the aforementioned multi-screen refresh system, the method comprising:

[0079] S10: The image drawing module draws preset image data and sends the preset image data to the image synthesis module.

[0080] S20: In response to the image refresh signal, the image synthesis module performs image synthesis processing on the preset image data to obtain image display data, and transmits the image display data to the display screen module for display.

[0081] Step S10 includes:

[0082] The first image drawing unit sends an image drawing signal to the second drawing unit.

[0083] In this embodiment, the UI Thread sends drawing commands to the Render Thread.

[0084] The second image drawing unit responds to the image drawing signal, draws preset image data, and sends the preset image data to the image synthesis module.

[0085] After receiving the drawing command, RenderThread draws the UI element view required by the corresponding APP and transmits the drawn UI element view to SurfaceFlinger.

[0086] Before performing step S20, the following are included:

[0087] The display driver module transmits the preset image signal sent by the display module to the image transmission module.

[0088] The DisplayDriver receives pulse signals from the display module and sends the pulse signals to the HardwareComposer HAL.

[0089] The image transmission module converts the preset image signal into an image refresh signal and transmits the image refresh signal to the image synthesis module.

[0090] HardwareComposer HAL converts the pulse signal into a VSYNC signal and transmits it to SurfaceFlinger.

[0091] After receiving the VSYNC signal, SurfaceFlinger performs composite processing on the UI element view to obtain the UI display view for display on the display module.

[0092] In step S20, the image display data is transmitted sequentially to the display module through the image transmission module and the display driver module.

[0093] The UI display view is transmitted sequentially to the display module via HardwareComposer HAL and DisplayDriver for display.

[0094] Example 3:

[0095] This application also provides a storage medium, which is a type of computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions for causing a computer to execute the multi-screen refresh method described above.

[0096] In summary, this application provides a multi-screen refresh system, method, and storage medium. The multi-screen refresh system includes N display refresh links. Each display refresh link includes at least an image drawing module, an image compositing module, and a display module. The image drawing module is used to draw preset image data and send the preset image data to the image compositing module. The image compositing module is used to perform image compositing processing on the preset image data in response to an image refresh signal to obtain image display data, and then transmit the image display data to the display module for display. This application enables different display modules to perform image rendering at the optimal refresh rate, ensuring the smoothness of each display module, improving the user experience, and reducing power consumption.

[0097] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0100] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules according to the embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0102] Although the description of this application has been made in conjunction with the specific embodiments described above, it will be apparent to those skilled in the art that many substitutions, modifications, and variations can be made based on the foregoing. Therefore, all such substitutions, modifications, and variations are included within the spirit and scope of the appended claims.

Claims

1. A multi-screen refresh system, characterized in that, It includes at least N display refresh links, where N≥2; Each display refresh link includes at least an image rendering module, an image transmission module, an image compositing module, and a display module; The image drawing module is used to draw preset image data and send the preset image data to the image synthesis module; The image transmission module is used to transmit the image refresh signal to the image synthesis module; The image synthesis module is used to perform image synthesis processing on the preset image data in response to the image refresh signal to obtain image display data, and transmit the image display data to the display module for display. The image synthesis module is a multi-process module, with each process corresponding to refreshing one display module; the image transmission module is a single-process multi-instance module, with each instance corresponding to one display module.

2. The multi-screen refresh system according to claim 1, characterized in that, The image rendering module includes a first image rendering unit and a second image rendering unit; The first image drawing unit is used to send an image drawing signal to the second image drawing unit; The second image drawing unit is used to draw preset image data in response to the image drawing signal, and send the preset image data to the image synthesis module.

3. The multi-screen refresh system according to claim 2, characterized in that, The display refresh link also includes a display driver module; The display screen driving module is used to transmit the preset image signal sent by the display screen module to the image transmission module, so that the preset image signal can be converted into an image refresh signal by the image transmission module; It is also used to receive image display data obtained by the image transmission module from the image synthesis module, and to transmit the image display data to the display module.

4. The multi-screen refresh system according to claim 3, characterized in that, The display refresh link also includes an application module; The application module runs on the display module, and both the first image drawing unit and the second image drawing unit are located in the application module.

5. The multi-screen refresh system according to claim 4, characterized in that, The first image drawing unit, the second image drawing unit, and the image synthesis module in each display refresh link are sequentially connected to and accessed by the same image transmission module. The other end of the image transmission module is connected to a display driving module, and the other end of the display driving module is connected to the display module in each display refresh link.

6. A multi-screen refresh method, characterized in that, The method, applied to the multi-screen refresh system according to any one of claims 1-5, comprises: S10: The image drawing module draws preset image data and sends the preset image data to the image synthesis module; S20: In response to the image refresh signal, the image synthesis module performs image synthesis processing on the preset image data to obtain image display data, and transmits the image display data to the display screen module for display.

7. The multi-screen refresh method according to claim 6, characterized in that, Step S10 includes: The first image drawing unit sends an image drawing signal to the second image drawing unit; The second image drawing unit responds to the image drawing signal, draws preset image data, and sends the preset image data to the image synthesis module.

8. The multi-screen refresh method according to claim 7, characterized in that, Before performing step S20, the following are included: The display driver module transmits the preset image signal sent by the display module to the image transmission module; The image transmission module converts the preset image signal into an image refresh signal and transmits the image refresh signal to the image synthesis module.

9. The multi-screen refresh method according to claim 8, characterized in that, In step S20, the image display data is transmitted sequentially to the display module through the image transmission module and the display driver module.

10. A storage medium, one of the computer-readable storage media, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the multi-screen refresh method according to any one of claims 7-9.

Images