Screen display picture drawing and synthesizing method, device and electronic equipment

Abstract

The present disclosure relates to a method and device for drawing and synthesizing a screen display picture, and an electronic device, wherein the method comprises: determining screen presence state information of the electronic device, wherein the screen presence state information is a screen and no screen; in the case that the screen presence state information is no screen, controlling a screen display picture drawing and synthesizing module to stop drawing and synthesizing a display picture.

Description

Technical Field

[0001] This disclosure relates to screen display control technology, and more specifically, to a method, apparatus, and electronic device for drawing and compositing screen display images. Background Technology

[0002] For screenless electronic devices (e.g., screenless AI glasses, screenless smart home devices, IoT devices), the physical screen does not exist, but the system still performs the drawing and compositing of the screen display, resulting in unnecessary CPU resource consumption and power waste.

[0003] Therefore, a technique is needed to dynamically adjust the rendering and compositing of the screen display based on the presence or absence of a screen on electronic devices, in order to reduce power consumption and improve system performance. Summary of the Invention

[0004] One objective of this invention is to provide a new technical solution for a method of rendering and compositing images for screen display.

[0005] According to a first aspect of the present invention, a method for rendering and compositing screen display images is provided, comprising:

[0006] Determine the screen presence status information of the electronic device, wherein the screen presence status information includes whether there is a screen or not;

[0007] When the screen status information is "no screen", the control screen display rendering and compositing module stops rendering and compositing the display image.

[0008] Optionally, the method further includes:

[0009] When the screen presence status information indicates that there is a screen, the screen display rendering and compositing module is controlled to perform the operation of rendering and compositing the display image.

[0010] Optionally, determining the presence of screen status information of the electronic device includes:

[0011] Determine whether the electronic device itself has a physical screen;

[0012] If it is determined that the electronic device itself has a physical screen, the screen presence status information is determined to be "screen present".

[0013] Optionally, determining whether the electronic device itself has a physical screen includes:

[0014] When the electronic device is started, it is detected whether the current value of the flag corresponding to the physical screen presence status is a preset value, which is used to indicate that the electronic device has a physical screen;

[0015] If the current value of the flag corresponding to the physical screen existence status is a preset value, it is determined that the electronic device itself has a physical screen.

[0016] Optionally, determining the screen presence status information of the electronic device further includes:

[0017] If it is determined that the electronic device itself does not have a physical screen, determine whether the electronic device has created a virtual screen;

[0018] If it is determined that the electronic device has created a virtual screen, the screen existence status information is determined to be "screen present".

[0019] Optionally, determining whether the electronic device has created a virtual screen includes:

[0020] Determine whether the electronic device has received a screen projection command from an external screen projection device;

[0021] If it is determined that the electronic device has received a screen projection command from an external screen projection device, it is determined that the electronic device has created a virtual screen, wherein the virtual screen is used to draw and synthesize the screen display image of the screen projection device.

[0022] Optionally, the control screen display rendering and compositing module stops rendering and compositing the display screen, including:

[0023] The screen display rendering and compositing module is controlled to adjust the screen refresh rate to a first preset threshold.

[0024] The operation of controlling the screen display rendering and compositing module to execute the rendering and compositing of the display screen includes:

[0025] The screen display rendering and compositing module is controlled to adjust the screen refresh rate to a second preset threshold, wherein the second preset threshold is greater than the first preset threshold.

[0026] According to a second aspect of the present invention, a screen display image rendering and compositing apparatus is provided, comprising:

[0027] A screen presence status information determination module is used to determine the screen presence status information of an electronic device, wherein the screen presence status information is either "with screen" or "without screen".

[0028] The display screen rendering and compositing control module is used to control the screen display screen rendering and compositing module to stop rendering and compositing the display screen when the screen status information is "no screen".

[0029] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to execute a method for drawing and compositing a screen display image according to any one of the first aspects.

[0030] According to a fourth aspect of the present invention, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein when executed by a processor, the computer program instructions implement the screen display image rendering and composition method described in any one of the first aspects.

[0031] This disclosure provides a method for rendering and compositing screen display images. By determining the screen presence status information of an electronic device, the method can then execute a corresponding screen display rendering and compositing strategy based on whether the electronic device has a screen or not. When the electronic device is in a screenless state, the screen display rendering and compositing module is controlled to stop rendering and compositing the display image, thereby reducing CPU resource consumption and improving system performance.

[0032] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0034] Figure 1 This is a flowchart illustrating a method for drawing and compositing screen display images according to an embodiment of the present invention.

[0035] Figure 2 This is a flowchart illustrating a method for drawing and compositing screen display images according to an embodiment of the present invention.

[0036] Figure 3 This is a schematic block diagram of a screen display image rendering and compositing apparatus according to an embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0038] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0039] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0040] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0041] To address the aforementioned technical problems, this disclosure provides a method for rendering and compositing screen display images. By determining the screen presence status information of the electronic device, a corresponding screen display rendering and compositing strategy can be executed based on whether the electronic device has a screen or not. When the electronic device is in a screenless state, the screen display rendering and compositing module is controlled to stop rendering and compositing the display image, thereby reducing CPU resource consumption and improving system performance.

[0042] In one embodiment of the present invention, a method for rendering and compositing screen display images is provided. According to... Figure 1 As shown, the screen display image rendering and compositing method of this embodiment includes the following steps S110 to S120.

[0043] Step S110: Determine the screen presence status information of the electronic device, wherein the screen presence status information is either "with screen" or "without screen".

[0044] Step S110 specifically includes: determining the screen existence status information of the electronic device, including: determining whether the electronic device itself has a physical screen; if it is determined that the electronic device itself has a physical screen, determining the screen existence status information as having a screen.

[0045] A physical screen is a screen with physical hardware (such as an LCD or OLED panel).

[0046] Determining whether an electronic device has a physical screen involves: when the electronic device is powered on, checking if the current value of the flag indicating the presence of a physical screen is a preset value. If the current value of the flag indicates the presence of a physical screen is the preset value, then the electronic device has a physical screen. This allows for timely detection of the presence of a physical screen upon power-on, enabling the execution of appropriate screen display rendering and compositing strategies based on the result. If the current value of the flag indicates the presence of a physical screen is not the preset value, then the electronic device does not have a physical screen.

[0047] The flag identifier corresponding to the physical screen's presence status is stored in non-volatile storage, such as EEPROM, flash memory, or configuration files, to persistently record the physical screen's presence status of electronic devices.

[0048] If it is determined that the electronic device itself does not have a physical screen, the electronic device can also create a virtual screen to support the screen projection display function of an external projection device. In this case, step S110 specifically includes: determining whether the electronic device has created a virtual screen; if it is determined that the electronic device has created a virtual screen, determining that the screen existence status information is "screen exists". Since the screen display rendering and compositing module still needs to perform the operation of rendering and compositing the display screen after the electronic device creates a virtual screen, it is necessary to further determine whether the electronic device has created a virtual screen when it is determined that the electronic device has created a virtual screen to ensure that the screen projection display function of an external projection device is supported when it is determined that the electronic device has created a virtual screen.

[0049] Determining whether an electronic device has created a virtual screen specifically includes: determining whether the electronic device has received a screen projection command from an external screen projection device; if it is determined that the electronic device has received a screen projection command from an external screen projection device, it is determined that the electronic device has created a virtual screen, wherein the virtual screen is used to draw and synthesize the screen display image of the screen projection device.

[0050] Electronic devices can connect to a projection device via USB. Once connected, the projection device sends an adb shell command to the electronic device via USB. The adb shell command enables screen mirroring, allowing a composite screen display drawn on a virtual screen to be displayed on the projection device.

[0051] Step S120: When the screen status information is "no screen", control the screen display drawing and compositing module to stop drawing and compositing the display screen.

[0052] In some embodiments, the method further includes: when the screen presence status information indicates that a screen exists, controlling the screen display rendering and compositing module to perform the operation of rendering and compositing a composite display image. Specifically, when it is determined that the electronic device itself has a physical screen, controlling the screen display rendering and compositing module to perform the operation of rendering and compositing a composite display image; or, when it is determined that the electronic device has created a virtual screen, controlling the screen display rendering and compositing module to perform the operation of rendering and compositing a composite display image.

[0053] In some embodiments, controlling the screen display rendering and compositing module to stop rendering the composite display includes: controlling the screen display rendering and compositing module to adjust the screen refresh rate to a first preset threshold; controlling the screen display rendering and compositing module to perform the operation of rendering the composite display includes: controlling the screen display rendering and compositing module to adjust the screen refresh rate to a second preset threshold, wherein the second preset threshold is greater than the first preset threshold.

[0054] Taking Android devices as an example, the screen display rendering and composition module is SurfaceFlinger. SurfaceFlinger is a display service in the Android system responsible for managing the rendering and composition operations of the screen display. When the electronic device has neither a physical screen nor a virtual screen, SurfaceFlinger adjusts the screen refresh rate to a first preset threshold, such as 1Hz. Adjusting the screen refresh rate to the first preset threshold reduces the frequency of calls to the doComposition() function, thereby reducing CPU resource consumption. When the electronic device has a physical screen, or when a virtual screen has been created, SurfaceFlinger adjusts the screen refresh rate to a second preset threshold, such as 60Hz. Adjusting the screen refresh rate to the second preset threshold ensures the normal rendering and composition of the physical screen display or the virtual screen display. It should be noted that the doComposition() function is the core algorithm in SurfaceFlinger; the doComposition() function needs to be called once every time the screen display refreshes.

[0055] Figure 2 This is a flowchart illustrating a method for rendering and compositing screen display images according to an embodiment of the present invention. Figure 2 As shown, the screen display image rendering and compositing method provided in this embodiment includes the following steps S201 to S204.

[0056] Step S201: Determine whether the electronic device itself has a physical screen.

[0057] Specifically, when the electronic device is powered on, it checks whether the current value of the flag indicating the presence of a physical screen is a preset value. The preset value indicates that the electronic device has a physical screen. If the current value of the flag indicating the presence of a physical screen is the preset value, it is determined that the electronic device has a physical screen. If the current value of the flag indicating the presence of a physical screen is not the preset value, it is determined that the electronic device does not have a physical screen.

[0058] When the electronic device has a physical screen, step S202 is executed to control the screen display drawing and compositing module to perform the operation of drawing and compositing the display screen.

[0059] Specifically, the control module for drawing and compositing the screen display adjusts the screen refresh rate to the default value, for example, 60Hz.

[0060] If the electronic device does not have a physical screen, proceed to step S203 to determine whether the electronic device has created a virtual screen.

[0061] Specifically, once a connection is established between the electronic device and the projection device, it is determined whether an adb shell command sent by the projection device via USB connection has been received. If an adb shell command sent by the projection device via USB connection is received, it is determined that the electronic device has created a virtual screen.

[0062] If a virtual screen has already been created on the electronic device, perform step S202 as described above. Specifically, control the screen display rendering and compositing module to adjust the screen refresh rate to the default value, for example, 60Hz.

[0063] If the electronic device does not create a virtual screen, step S204 is executed to control the screen display rendering and compositing module to adjust the screen refresh rate to the lowest value.

[0064] An embodiment of the present invention also provides a device for drawing and compositing screen display images. According to... Figure 3 As shown, the screen display rendering and compositing device 300 includes a screen presence status information determination module 310 and a display rendering and compositing control module 320.

[0065] The screen presence status information determination module 310 is used to determine the screen presence status information of the electronic device, wherein the screen presence status information is either "with screen" or "without screen".

[0066] The display screen drawing and compositing control module 320 is used to control the screen display screen drawing and compositing module to stop drawing and compositing the display screen when the screen status information is "no screen".

[0067] In some embodiments, the display screen drawing and compositing control module 320 is further configured to control the screen display screen drawing and compositing module to perform the operation of drawing and compositing display screens when the screen existence status information is "screen present".

[0068] In some embodiments, the screen presence status information determination module 310 is used to determine whether the electronic device itself has a physical screen; if it is determined that the electronic device itself has a physical screen, the screen presence status information is determined to be that there is a screen.

[0069] In some embodiments, the screen presence status information determination module 310 is used to detect whether the current value of the flag identifier corresponding to the physical screen presence status is a preset value when the electronic device is started. The preset value is used to indicate that the electronic device has a physical screen. If the current value of the flag identifier corresponding to the physical screen presence status is a preset value, the module determines that the electronic device itself has a physical screen.

[0070] In some embodiments, the screen presence status information determination module 310 is used to determine whether the electronic device has created a virtual screen when it is determined that the electronic device itself does not have a physical screen.

[0071] If it is determined that the electronic device has created a virtual screen, the screen existence status information is determined to be "screen present".

[0072] In some embodiments, the screen presence status information determination module 310 is used to determine whether the electronic device has received a screen projection instruction sent by an external screen projection device; if it is determined that the electronic device has received a screen projection instruction sent by an external screen projection device, it is determined that the electronic device has created a virtual screen, wherein the virtual screen is used to draw the screen display image of the composite screen projection device.

[0073] In some embodiments, the display screen rendering and compositing control module 320 is also used to control the screen display screen rendering and compositing module to adjust the screen refresh rate to a first preset threshold.

[0074] The control screen display rendering and compositing module performs the operations of rendering and compositing the display image, including:

[0075] The control screen display rendering and compositing module adjusts the screen refresh rate to a second preset threshold, wherein the second preset threshold is greater than the first preset threshold.

[0076] An embodiment of the present invention also provides an electronic device, such as... Figure 4 As shown. The electronic device 400 includes a memory 420 and a processor 410. The memory 420 stores a computer program that controls the processor 410 to operate and execute the screen display image rendering and composition method provided according to any of the above embodiments.

[0077] The processor 400 is used to execute computer programs, which can be written using instruction sets of architectures such as x86, Arm, RISC, MIPS, and SSE. The memory 400 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk.

[0078] The electronic device can be any of the following: wearable device (e.g., AI glasses), smart home device, or Internet of Things device.

[0079] An embodiment of the present invention also provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the screen display image rendering and composition method described in any one of the first aspects.

[0080] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0081] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0082] Embodiments of this specification may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer instructions stored thereon for causing a processor to implement various aspects of the embodiments of this specification.

[0083] Computer-readable storage media can be tangible devices capable of holding and storing computer instructions for use by computer instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing computer instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0084] The computer instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this specification. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of computer instructions, which contains one or more executable computer instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0086] Various embodiments of this specification have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for rendering and compositing screen display images, characterized in that, include: Determine the screen presence status information of the electronic device, wherein the screen presence status information includes whether there is a screen or not; When the screen status information is "no screen", the control screen display rendering and compositing module stops rendering and compositing the display image.

2. The method according to claim 1, characterized in that, The method further includes: When the screen presence status information indicates that there is a screen, the screen display rendering and compositing module is controlled to perform the operation of rendering and compositing the display image.

3. The method according to claim 1, characterized in that, The determination of the screen status information of the electronic device includes: Determine whether the electronic device itself has a physical screen; If it is determined that the electronic device itself has a physical screen, the screen presence status information is determined to be "screen present".

4. The method according to claim 3, characterized in that, Determining whether the electronic device itself has a physical screen includes: When the electronic device is started, it is detected whether the current value of the flag corresponding to the physical screen presence status is a preset value, which is used to indicate that the electronic device has a physical screen; If the current value of the flag corresponding to the physical screen existence status is a preset value, it is determined that the electronic device itself has a physical screen.

5. The method according to claim 3, characterized in that, The determination of the screen status information of the electronic device also includes: If it is determined that the electronic device itself does not have a physical screen, determine whether the electronic device has created a virtual screen; If it is determined that the electronic device has created a virtual screen, the screen existence status information is determined to be "screen present".

6. The method according to claim 5, characterized in that, Determining whether an electronic device has created a virtual screen includes: Determine whether the electronic device has received a screen projection command from an external screen projection device; If it is determined that the electronic device has received a screen projection command from an external screen projection device, it is determined that the electronic device has created a virtual screen, wherein the virtual screen is used to draw and synthesize the screen display image of the screen projection device.

7. The method according to claim 2, characterized in that, The control screen display rendering and compositing module stops rendering and compositing the display screen, including: The screen display rendering and compositing module is controlled to adjust the screen refresh rate to a first preset threshold. The operation of controlling the screen display rendering and compositing module to execute the rendering and compositing of the display screen includes: The screen display rendering and compositing module is controlled to adjust the screen refresh rate to a second preset threshold, wherein the second preset threshold is greater than the first preset threshold.

8. A device for drawing and compositing images for screen display, characterized in that, include: A screen presence status information determination module is used to determine the screen presence status information of an electronic device, wherein the screen presence status information is either "with screen" or "without screen". The display screen rendering and compositing control module is used to control the screen display screen rendering and compositing module to stop rendering and compositing the display screen when the screen status information is "no screen".

9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to execute the screen display image rendering and compositing method according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the screen display image rendering and compositing method according to any one of claims 1 to 7.

Images