Image processing method and electronic equipment

Abstract

The invention discloses an image processing method and electronic equipment, and relates to the technical field of image processing. The method comprises the steps that a first application is operated, and the first application comprises a first window; and before the display duration of the first window reaches the first duration, generating first layer content of a first target layer associated with the first window, and displaying window content of the first window based on the first layer content. And after the display duration of the first window reaches the first duration, generating a second layer content of the first target layer, enhancing the value of a blue channel of a pixel point in the second layer content to obtain a third layer content, and displaying the window content of the first window based on the third layer content. Therefore, after one window is displayed for a long time, the eye protection effect can be achieved by enhancing the blue color of the content of the window.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and in particular, to an image processing method and an electronic device. BACKGROUND

[0002] Daily long-time use of electronic devices such as mobile phones and tablets can cause certain damage to the eyes, thereby affecting vision. At the same time, with the improvement of living standards and the enhancement of health awareness, people pay more and more attention to eye protection (which can be referred to as eye protection). Therefore, there is an urgent need for a solution that can achieve eye protection during the use of electronic devices. SUMMARY

[0003] The present application provides an image processing method and an electronic device, which can enhance the value of the blue channel of a pixel point in an image through an eye protection algorithm during the processing of a frame of image, thereby achieving eye protection.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] In a first aspect, the present application provides an image processing method, comprising: running a first application (hereinafter referred to as application program 1, application program 2, etc.), the first application comprising a first window (hereinafter referred to as window A, window B, etc.).

[0006] Before the display duration of the first window reaches a first duration (hereinafter referred to as duration 1), a first layer content of a first target layer associated with the first window is generated, and the window content of the first window is displayed based on the first layer content. That is, when the display duration of the first window is short, the window content can be displayed normally after rendering and composition, without the need to perform eye protection processing.

[0007] After the display duration of the first window reaches the first duration, a second layer content of the first target layer is generated, the value of the blue channel of a pixel point in the second layer content is enhanced (which can also be referred to as enhanced blue below), a third layer content is obtained, and the window content of the first window is displayed based on the third layer content. That is, after the display duration of the first window is long, the blue color in the layer content of the target layer needs to be enhanced after the layer content is obtained, and accordingly, the blue color in the finally displayed window content will be enhanced, thereby achieving eye protection.

[0008] In summary, by using the solution of the present application, the blue color in the window can be enhanced after the display duration of the window is long, the viewing comfort when a user gazes at the window is improved, and eye protection is achieved.

[0009] It should be noted that the target layer associated with a window refers to a layer used to draw window content to be displayed in the window. For example, the window is a video playing window, and the target layer includes a layer of video content, a layer of bullet screen, a layer of subtitles, etc. For another example, the window is a graphic (e.g., novel) reading window, and the target layer includes a layer of text, a layer of illustrations, etc.

[0010] In a possible design of the first aspect, the value of the blue channel of the pixel point in the second layer content is enhanced in the image synthesis stage or the image rendering stage.

[0011] In a possible design of the first aspect, the value of the blue channel of the pixel point in the second layer content is enhanced in the image synthesis stage or the image rendering stage, including:

[0012] In a case where the first window is a first type window (e.g., a target window 1 described below), the value of the blue channel of the pixel point in the second layer content is enhanced in the image synthesis stage. At least one target layer associated with the first type window can not be rendered to obtain layer content. For example, the at least one target layer is a video content layer, which can be directly transmitted to an image synthesizer from a graphic buffer in which data decoded by a video codec is stored, and then the data is refreshed to the layer by the image synthesizer, that is, the first type window can include a video playing window. Obviously, the at least one target layer of the first type window cannot be rendered to obtain layer content, and thus cannot be protected in the image rendering stage. At the same time, image synthesis is performed in each image processing, and thus the blue color can be enhanced in the image synthesis stage for the first type window, to achieve eye protection. Thus, the blue color in the video content layer can be enhanced, to achieve the effect of eye protection.

[0013] In a case where the first window is a second type window, the value of the blue channel of the pixel point in the second layer content is enhanced in the image rendering stage. All target layers associated with the second type window (e.g., a target window 2 described below) are rendered to obtain layer content, for example, the second type window includes a graphic (e.g., novel) reading window, and thus the blue color can be enhanced in the image synthesis stage for the second type window, to achieve eye protection, and the effect of eye protection can also be achieved. In addition, the power consumption in the image rendering stage is generally lower than that in the image synthesis stage, and accordingly, the eye protection in the image rendering stage has less impact on the performance of the device and is more conducive to ensuring the performance of the device.

[0014] It should be noted that in the following description, the first type window is a video window, and the second type window is a graphic reading window.

[0015] In a possible design manner of the first aspect, before the value of the blue channel of the pixel point in the second layer content is enhanced in the image synthesis stage, the method further includes: setting a GPU synthesis flag (as the force client composition mode below) of the first target layer to a first identifier (as identifier 1 below) in a case where the first window is a video window. It is determined to perform the image synthesis by using the GPU based on the first identifier. It should be noted that the GPU synthesis flag of a layer is the first identifier, and it is determined to perform the image synthesis by using the GPU.

[0016] Correspondingly, the value of the blue channel of the pixel point in the second layer content is enhanced in the image synthesis stage, including: the value of the blue channel of the pixel point in the second layer content is enhanced in the image synthesis stage by using the GPU. Thus, the eye protection can be implemented by using the powerful image processing capability of the GPU.

[0017] In a possible design manner of the first aspect, after the second layer content of the first target layer is generated, the method further includes: in a case where the first window is not a video playing window and a graphic reading window, the value of the blue channel of the pixel point in the second layer content is not enhanced, and the window content of the first window is displayed based on the second layer content.

[0018] Generally, in a scenario of watching a video through a video playing window for a long time or reading through a graphic reading window for a long time, the damage to the eyes of a user is relatively large, and the eye protection is often required in this case. In addition, the eye protection is generally not required. In addition, the power consumption is increased to implement the eye protection. Therefore, for the scenario that does not require the eye protection, the eye protection can not be implemented, so that on the one hand, the eye protection effect can be matched with the eye protection requirement, and on the other hand, the power consumption of the device can be reduced.

[0019] In a possible design manner of the first aspect, the eye protection is implemented in the image rendering stage, and the implementation is as follows:

[0020] The second layer content of the first target layer associated with the first window is generated, including: the second layer content is obtained by rendering in the first target layer.

[0021] In the image rendering stage, the value of the blue channel of the pixel point in the second layer content is enhanced to obtain third layer content, including: in the image rendering stage, obtaining a first screenshot of the second layer content, fusing the first screenshot with a first shader to obtain the third layer content, and the first shader includes an algorithm for enhancing the value of the blue channel (eye protection algorithm). That is, in the rendering stage, a screenshot and a fusion step are added to realize eye protection. Among them, through the screenshot, the object executing the eye protection algorithm can be obtained. Through the fusion, the eye protection object can be taken as the input of the eye protection algorithm, and then running the eye protection algorithm can obtain the result of the eye protection output, that is, the third layer content.

[0022] Based on the third layer content, the window content of the first window is displayed, including: synthesizing the layer content of a plurality of layers to obtain the window content of the first window and display, and the layer content of the plurality of layers includes the third layer content. That is, after realizing eye protection in the rendering stage, the window content can be further obtained through image synthesis (including layer superposition) and displayed. Thus, the picture after enhancing the blue color can be displayed in the first window, and eye protection can be realized.

[0023] In a possible design manner in the first aspect, the scheme for realizing eye protection in the image rendering stage is implemented as follows:

[0024] The second layer content of the first target layer associated with the first window is generated, including: rendering and / or refreshing in the first target layer to obtain the second layer content. Among them, the video content layer needs to obtain the layer content through refreshing, and other target layers can obtain the layer content through rendering.

[0025] In the image synthesis stage, the value of the blue channel of the pixel point in the second layer content is enhanced to obtain third layer content, including: in the image synthesis stage, obtaining a first screenshot of the second layer content, correcting the first screenshot according to the resolution of the first display screen, and the first display screen is the display screen for displaying the first window; fusing the corrected first screenshot with a second shader to obtain the third layer content, and the second shader includes an algorithm for enhancing the value of the blue channel. It should be noted that each frame of video content is obtained by decoding the video source through the video codec, and then taken as the layer content of the video content layer (a first target layer), but not rendered according to the buffer size (i.e. resolution) of the video content layer. Therefore, the resolution of the video content is most likely not matched with the resolution of the first display screen used for display, so after obtaining the screenshot of the video content, the screenshot is first corrected according to the resolution of the first display screen, so that the corrected screenshot has the same resolution as the first display screen, and then fused with the second shader, so that the layer content corresponding to the resolution of the first display screen can be obtained.

[0026] The window content of the first window is displayed based on the third layer content, including: superimposing the layer content of a plurality of layers to obtain the window content of the first window and display, and the layer content of the plurality of layers includes the third layer content. That is, after the eye protection is realized in the synthesis stage, the layer superposition through the synthesis stage is still needed to obtain the window content of the first window and display. Thus, the picture after the blue color is enhanced can be displayed in the first window, and the eye protection is realized.

[0027] In a possible design manner of the first aspect, before the value of the blue channel of the pixel point in the second layer content is enhanced, the following further includes: filtering the invalid layer in the first target layer to obtain the filtered first target layer; and the invalid layer includes at least one of the following: a layer of drawing a barrage content, a layer with a display frequency greater than a first frequency (frequency 1 in the following), and a layer with a buffer size greater than the resolution of the first display screen, and the first display screen is a display screen for displaying the first window. After the invalid layer is filtered out, some layers with high power consumption for realizing eye protection can be filtered out, the range of the layer for eye protection is reduced, and the pertinence of eye protection is improved. Correspondingly, the value of the blue channel of the pixel point in the second layer content is enhanced, including: the value of the blue channel of the pixel point in the second layer content of the filtered first target layer is enhanced.

[0028] In a possible design manner of the first aspect, the value of the blue channel of the pixel point in the second layer content is enhanced, including: the value of the blue channel of the pixel point in the second layer content is enhanced based on the resolution of the first display screen, the blur radius (blur radius recorded in the configuration table in the following) of the first target layer, and the buffer size of the first target layer. The blur radius is used to indicate the sampling specification when the value of the blue channel is enhanced, and the first display screen is a display screen for displaying the first window. In this way, the blue color can be enhanced in a targeted manner for different first target layers, and the rationality of eye protection is improved.

[0029] In a possible design manner of the first aspect, the blur radius and the enhancement degree of the value of the blue channel are in a positive correlation relationship. That is, the greater the blur radius, the higher the enhancement degree of the blue color, and the eye protection can be realized to a greater extent. The first ratio and the enhancement degree of the value of the blue channel are in a positive correlation relationship, and the first ratio is the ratio of the buffer size of the first target layer to the resolution of the first display screen. That is, the higher the first ratio, the higher the enhancement degree of the blue color, and the eye protection can be realized to a greater extent.

[0030] In a possible design of the first aspect, the method is applied to the electronic device, and the method further includes: correcting the blur radius based on ambient light brightness of an environment in which the electronic device is located, so that the blur radius can be matched with the current ambient light brightness. Correspondingly, the method of enhancing the value of the blue channel of the pixel point in the second layer content based on the resolution of the first display screen, the blur radius of the first target layer, and the buffer size of the first target layer includes: enhancing the value of the blue channel of the pixel point in the second layer content based on the resolution of the first display screen, the corrected blur radius, and the buffer size of the first target layer. In this way, the rationality of eye protection can be improved.

[0031] In a possible design of the first aspect, the method of correcting the blur radius of the first target layer based on the ambient light brightness of the environment in which the electronic device is located includes: when the ambient light brightness is first brightness, the corrected blur radius is first radius; and when the ambient light brightness is second brightness, the corrected blur radius is second radius, where the first brightness is lower than the second brightness, and the first radius is smaller than the second radius. That is, the higher the ambient light brightness, the smaller the corrected blur radius.

[0032] In a possible design of the first aspect, the method of correcting the blur radius of the first target layer based on the ambient light brightness of the environment in which the electronic device is located includes: when the ambient light brightness is first brightness, the corrected blur radius is first radius; and when the ambient light brightness is second brightness, the corrected blur radius is second radius, where the first brightness is lower than the second brightness, and the first radius is smaller than the second radius. That is, the higher the ambient light brightness, the smaller the corrected blur radius.

[0033] In a possible design of the first aspect, the method is applied to the electronic device, and the electronic device includes a display engine service, an image compositor, and a view root implementation.

[0034] In a possible design of the first aspect, the method is applied to the electronic device, and the electronic device includes a display engine service, an image compositor, and a view root implementation.

[0035] In a possible design of the first aspect, the method is applied to the electronic device, and the electronic device includes a display engine service, an image compositor, and a view root implementation.

[0036] In the case that the first window is a graphic-text reading window, the value of the blue channel of a pixel point in the second layer content is enhanced in the image rendering stage, including: in the case that the display engine service detects that the first window is a graphic-text reading window, the display engine service sends a second notification (such as including the layer white list and algorithm parameters in the following text) to the view root implementation. In response to the second notification, the view root implementation controls the value of the blue channel of the pixel point in the second layer content in the image rendering stage. That is, for the graphic-text reading window, the display service engine will notify the image root implementation, so as to control the eye protection in the image rendering stage through the image root implementation. For example, the image root implementation can integrate a shader (such as a first shader) for implementing eye protection in the rendering stage, so as to control the eye protection in the rendering stage.

[0037] In a possible design manner in the first aspect, after running the first application, the method further includes: the view root implementation registers the listening of the first window to the display engine service, and the second type window includes the window for which the view root implementation registers the listening. Through the registration of the listening, cross-process communication between the display engine service and the view root implementation is realized, and the display engine service feeds back the listening result of the first window to the view root implementation. Correspondingly, the display engine service sends the second notification to the view root implementation, including: the display engine service sends the second notification to the view root implementation that registers the listening of the first window. Thus, the view root implementation that registers the listening can be notified.

[0038] It can be understood that the view root implementations of multiple applications (such as multiple novel reading applications) can all register the listening of the corresponding window to the display engine service, and the display engine service can maintain a correspondence table of the view root implementation and the window based on the registration. Subsequently, after listening to a certain window, the view root implementation that registers the listening can be determined by looking up the table.

[0039] In the second aspect, the application provides an image processing method, including: displaying a first interface (such as the interface in the following Figure 18 ), the first interface including a video playing window (such as the video playing window 1801 in the following Figure 18 ) and a graphic-text reading window (such as the graphic-text reading window 1802 in the following Figure 18 ). Generating fourth layer content of a second target layer associated with the video playing window, enhancing the value of the blue channel of a pixel point in the fourth layer content in the image synthesis stage, obtaining fifth layer content, and displaying the window content of the video playing window based on the fifth layer content. Generating sixth layer content of a third target layer associated with the graphic-text reading window, enhancing the value of the blue channel of a pixel point in the sixth layer content in the image rendering stage, obtaining seventh layer content, and displaying the window content of the graphic-text reading window based on the seventh layer content.

[0040] To sum up, by using the application, for the scenario that there are a video playing window and a text and picture reading window running in the foreground, eye protection can be implemented for the video playing window in the image synthesis stage, so that the blue color in the video content layer can be enhanced to achieve the effect of eye protection. In addition, eye protection can be implemented for the text and picture reading window in the image synthesis stage, which not only can ensure the effect of eye protection, but also can reduce the impact on the device power consumption, and is more conducive to ensuring the use performance of the device.

[0041] In a possible design manner of the second aspect, the value of the blue channel of the pixel point in the fourth layer content is enhanced in the image synthesis stage, including: after the display duration of the video playing window reaches the first duration, the value of the blue channel of the pixel point in the fourth layer content is enhanced in the image synthesis stage. The value of the blue channel of the pixel point in the fifth layer content is enhanced in the image rendering stage, including: after the display duration of the text and picture reading window reaches the first duration, the value of the blue channel of the pixel point in the fifth layer content is enhanced in the image rendering stage.

[0042] In a possible design manner of the second aspect, the sixth layer content of the third target layer associated with the text and picture reading window is generated, including: the sixth layer content is rendered in the third target layer. The value of the blue channel of the pixel point in the sixth layer content is enhanced in the image rendering stage to obtain the seventh layer content, including: in the image rendering stage, a second screenshot of the sixth layer content is obtained, the second screenshot is fused with a third shader to obtain the seventh layer content, and the third shader includes an algorithm for enhancing the value of the blue channel. The window content of the text and picture reading window is displayed based on the seventh layer content, including: the layer content of a plurality of layers is synthesized to obtain the window content of the text and picture reading window and is displayed, and the layer content of the plurality of layers includes the seventh layer content.

[0043] In a possible design manner of the second aspect, the fourth layer content of the second target layer associated with the video playing window is generated, including: the fourth layer content is rendered and / or refreshed in the second target layer. The value of the blue channel of the pixel point in the fourth layer content is enhanced in the image synthesis stage to obtain the fifth layer content, including: in the image synthesis stage, a third screenshot of the fourth layer content is obtained, the third screenshot is corrected according to the resolution of the second display, and the corrected third screenshot is fused with a fourth shader to obtain the fifth layer content, and the fourth shader includes an algorithm for enhancing the value of the blue channel. The window content of the video playing window is displayed based on the fifth layer content, including: the layer content of a plurality of layers is superimposed to obtain the window content of the video playing window, and the layer content of the plurality of layers includes the fifth layer content.

[0044] In a possible design of the second aspect, before the image synthesis stage, the method further includes: filtering invalid layers in the second target layer to obtain a filtered second target layer. The invalid layers include at least one of the following: a layer in which a bullet screen content is drawn, a layer in which a display frequency is greater than the first frequency, and a layer in which a buffer size is greater than a resolution of the second display screen, and the second display screen is a display screen that displays the video playback window. The image synthesis stage includes: enhancing, at the image synthesis stage, the value of the blue channel of the pixel point in the fourth layer content of the filtered second target layer.

[0045] In a possible design of the second aspect, the image synthesis stage includes: enhancing, based on the resolution of the second display screen, the blur radius of the second target layer, and the buffer size of the second target layer, the value of the blue channel of the pixel point in the fourth layer content. The blur radius is used to indicate a sampling specification when the value of the blue channel is enhanced, and the second display screen is a display screen that displays the video playback window.

[0046] In a possible design of the second aspect, the method is applied to an electronic device, and the method further includes: correcting the blur radius based on ambient light brightness of an environment in which the electronic device is located. The image synthesis stage includes: enhancing, based on the resolution of the second display screen, the corrected blur radius, and the buffer size of the second target layer, the value of the blue channel of the pixel point in the fourth layer content.

[0047] It should be noted that the above description of the second aspect is mainly based on the video playback window, and the same applies to the graphic reading window. Therefore, the description of the graphic reading window is not repeated here.

[0048] In addition, the effects of the various possible designs of the second aspect can be found in the above description of the first aspect and the possible designs thereof, and are not repeated here.

[0049] In a third aspect, the present application provides an electronic device. The electronic device includes a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is configured to store computer program code including computer instructions. When the computer instructions are executed by the processor, the electronic device performs the method in the first aspect, the second aspect, and any possible design thereof.

[0050] In a fourth aspect, the present application provides a chip system applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected through lines; the interface circuits are configured to receive signals from the memory of the electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processors execute the computer instructions, the electronic device performs the method according to the first aspect, the second aspect, or any possible design of the first aspect and the second aspect.

[0051] In a fifth aspect, the present application provides a computer storage medium including computer instructions, when the computer instructions are executed on an electronic device, the electronic device performs the method according to the first aspect, the second aspect, or any possible design of the first aspect and the second aspect.

[0052] In a sixth aspect, the present application provides a computer program product, when the computer program product is executed on a computer, the computer performs the method according to the first aspect, the second aspect, or any possible design of the first aspect and the second aspect.

[0053] It can be understood that the electronic device provided in the third aspect, the chip system provided in the fourth aspect, the computer storage medium provided in the fifth aspect, and the computer program product provided in the sixth aspect can achieve the beneficial effects as described in the first aspect, the second aspect, or any possible design of the first aspect and the second aspect, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 A hardware structure diagram of a mobile phone provided by an embodiment of the present application;

[0055] Figure 2 A software architecture diagram of a mobile phone provided by an embodiment of the present application;

[0056] Figure 3 A basic principle diagram of image processing provided by an embodiment of the present application;

[0057] Figure 4 A sampling principle diagram of blur processing provided by an embodiment of the present application;

[0058] Figure 5 An algorithm flowchart of an eye protection algorithm provided by an embodiment of the present application;

[0059] Figure 6 An algorithm flowchart of another eye protection algorithm provided by an embodiment of the present application;

[0060] Figure 7 One of the effect diagrams of the image processing method provided by an embodiment of the present application;

[0061] Figure 8One of the timing interaction diagrams of the image processing method in the scheme one provided by the embodiments of the present application;

[0062] Figure 9 One of the object calling schematic diagrams of the image processing method in the scheme one provided by the embodiments of the present application;

[0063] Figure 10 Two of the object calling schematic diagrams of the image processing method in the scheme one provided by the embodiments of the present application;

[0064] Figure 11 Two of the timing interaction diagrams of the image processing method in the scheme one provided by the embodiments of the present application;

[0065] Figure 12 Two of the effect diagrams of the image processing method provided by the embodiments of the present application;

[0066] Figure 13 One of the timing interaction diagrams of the image processing method in the scheme two provided by the embodiments of the present application;

[0067] Figure 14 One of the object calling schematic diagrams of the image processing method in the scheme two provided by the embodiments of the present application;

[0068] Figure 15 Two of the object calling schematic diagrams of the image processing method in the scheme two provided by the embodiments of the present application;

[0069] Figure 16 Two of the timing interaction diagrams of the image processing method in the scheme two provided by the embodiments of the present application;

[0070] Figure 17 One of the effect diagrams of the combination of the scheme one and the scheme two provided by the embodiments of the present application;

[0071] Figure 18 Two of the effect diagrams of the combination of the scheme one and the scheme two provided by the embodiments of the present application. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to be limiting on the present application. As used in the specification and the appended claims of the present application, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that “at least one” and “one or more” as used in the embodiments herein indicates one or two or more (including two), unless otherwise indicated. The term “and / or” is used to describe the association relationship of the associated objects, which means that there can be three relationships; for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character “ / ” generally represents an “or” relationship between the associated objects.

[0073] In the present specification, the reference to “one embodiment” or “some embodiments” or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments” and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise indicated. The terms “comprising”, “including”, “having” and their variants mean “including but not limited to”, unless otherwise indicated. The term “connected” includes both direct and indirect connections, unless otherwise indicated. “First”, “second”, etc. are used only for descriptive purposes and should not be construed as indicating or implying relative importance or an indicated number of technical features.

[0074] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as “exemplary” or “for example” in the embodiments of the present application is not necessarily to be construed as preferred or advantageous over other embodiments or design solutions. Rather, use of the words “exemplary” or “for example” is intended to present concepts in a concrete manner.

[0075] The image processing method provided in this application can be applied to scenarios requiring eye protection when using electronic devices such as mobile phones and tablets (hereinafter referred to as eye protection scenarios). Taking a mobile phone as an example, eye protection scenarios can include scenarios where users spend a long time watching videos / binge-watching dramas on their phones (hereinafter referred to as video playback scenarios) or reading novels on their phones for extended periods (hereinafter referred to as reading scenarios). Specifically, in eye protection scenarios, during the processing of a single frame of an image, the electronic device can enhance the value of the blue channel of pixels in the image (hereinafter referred to as blue enhancement) through an eye protection algorithm, thereby achieving an eye protection effect.

[0076] For example, the aforementioned electronic devices may be mobile phones, tablets, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices, and / or smart city devices. This application does not impose any special limitations on the specific form of the electronic device.

[0077] See Figure 1 This is a hardware structure diagram of an electronic device. For example... Figure 1 As shown, taking a mobile phone as an example, the electronic device may include a processor 210, internal memory (RAM) 221, charging management module 240, power management module 241, battery 242, antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, audio module 270, display screen 294, and subscriber identification module (SIM) card interface 295, etc.

[0078] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone. In other embodiments, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0079] The processor 210 can include one or more processing units, such as: the processor 210 can include an application processor (AP), a modem processor, a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0080] The internal memory 221 can be used to store computer executable program code, which includes instructions. The processor 210 executes various functional methods or data processing of the mobile phone by running the instructions stored in the internal memory 221.

[0081] In some embodiments, the processor 210 can execute the image processing method by running the instructions stored in the internal memory 221.

[0082] The charging management module 240 is used to receive the charging input of the charger. The power management module 241 is used to connect the battery 242, and the charging management module 240 is connected with the processor 210. The power management module 241 receives the input of the battery 242 and / or the charging management module 240, and supplies power for the processor 210, the internal memory 221, the display screen 294, etc.

[0083] The wireless communication function of the mobile phone can be realized by the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor, and the baseband processor, etc.

[0084] The mobile phone can realize the display function through the GPU, the display screen 294, and the application processor, etc. The GPU is a microprocessor for image processing, which is connected with the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 can include one or more GPUs, which execute program instructions to generate or change display information.

[0085] In some embodiments, during the processing of a frame of image, the mobile phone can complete image rendering and image synthesis through the GPU.

[0086] Further, the eye-care algorithm can be implemented by a shader, which is a small program running on a GPU and used to process specific tasks in image rendering and image synthesis. Based on this, in actual implementation, the shader of the eye-care algorithm can be executed by the GPU in the process of image rendering or image synthesis to achieve the eye-care effect.

[0087] In some embodiments, after the image processing method proposed in the present application is executed, the display screen 294 can display blueish text and pictures to achieve the eye-care effect.

[0088] The mobile phone can realize audio functions through an audio module 270, such as a speaker, a receiver, a microphone, a headset interface, and an application processor, etc. For example, music playing, recording, etc.

[0089] The software system of the electronic device described above can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device. The layered architecture divides the software system of the electronic device into several layers, each layer has a clear role and division of labor, and the layers communicate with each other through software interfaces. TM ) system with a layered architecture. The layered architecture divides the software system of the electronic device into several layers, each layer has a clear role and division of labor, and the layers communicate with each other through software interfaces.

[0090] Referring to Figure 2 Taking a mobile phone as an example, the software and hardware architecture of the electronic device can include an application (application, APP) layer, an application framework layer, a native layer, and a kernel layer, etc.

[0091] Among them, the video player, the reader, the electronic mailbox, the memo, the alarm clock, the setting, the call, etc. can be installed in the application layer.

[0092] In the running process of the video player, the reader, etc. described above, the mobile phone can execute the image processing method provided by the embodiments of the present application to achieve the eye-care effect.

[0093] The application framework layer provides the application programming interface (application programming interface, API) and the programming framework for the application layer.

[0094] Among them, the application framework layer can include a plurality of system services (System Server), such as an activity management service (Activity Manager Service, AMS) and a display engine service (Display Engine Service, DES), etc.

[0095] The activity management service can be used for the start, switch, scheduling of system components (e.g. activities, services, content providers, broadcast receivers) and the management and scheduling of application processes.

[0096] The display engine service can be used to listen to whether the conditions for opening or closing the eye-care switch are met (i.e. listening to the switching conditions), such as listening to the switching of windows, the temperature of the electronic device, etc. It should be noted that the windows and activities are usually corresponding, one activity creates one window, and accordingly, listening to the switching of the window can also be understood as listening to the switching of the activity.

[0097] The display engine service can also control the opening and closing of the eye-care switch based on the listening results. After the eye-care switch is opened, the mobile phone will execute the eye-care algorithm to achieve the eye-care effect; after the eye-care switch is closed, the mobile phone will not execute the eye-care algorithm.

[0098] For example, the display engine service listens to the window of the video player or the reader running in the foreground for more than a preset time length (time length 1 and time length 2 in the following, such as 5 minutes (min)), and then the eye-care switch can be opened, so that the eye-care effect can be achieved in the case of long-time video watching or reading.

[0099] For another example, the display engine service closes the eye-care switch after the temperature of the electronic device is too high, so that the power consumption of the electronic device can be reduced by closing the eye-care switch in the case of high temperature.

[0100] Further, the display engine service can include a window switching listening module and a callback management module.

[0101] The listening module can be used for listening to the switching conditions, mainly listening to the windows, and feeding back to other modules (such as the image synthesizer (SurfaceFlinger, SF) and the view root implementation (ViewRootImpl, VRI) in the following) after listening to the switching conditions to achieve the opening and closing of the eye-care.

[0102] The callback management module can be used for other modules to register listening callbacks. Especially for some modules that originally have no cross-process (binder) communication link with the display engine service, such as the view root implementation, by registering the listening callback, the display engine service and these modules can achieve cross-process communication, such as the display engine service feeding back to these modules based on the listening results.

[0103] In some embodiments, the mobile phone provides a setting item for the eye-care function, and in response to the user's operation of opening the setting item, the mobile phone listens to whether the conditions for opening or closing the eye-care switch are met through the display engine service, and controls the opening and closing of the eye-care switch based on the listening results.

[0104] The application framework layer can also include a view root implementation (ViewRootImpl, VRI) and a view module.

[0105] Each window corresponds to a VRI instance, which is responsible for managing and drawing the window content. For example, through the VRI instance, the view hierarchy can be traversed, also known as do traversal, so as to realize drawing and event distribution.

[0106] The view module can be used to realize the rendering effect of each component in the window. For example, the view module can execute a render effect (RenderEffect).

[0107] The native layer provides various services for the upper layer (such as the application framework layer). For example, the native layer includes a surface flinger (SurfaceFlinger, SF), a hardware accelerated UI (Hardware Accelerated UI, HWUI), a graphics library, and the like. It should be noted that SurfaceFlinger can also be translated as surface drawer, surface drawing module, image composition processing service, and the like, which are not specifically limited in the present application.

[0108] The SurfaceFlinger can be used for refresh rate control, image composition control, and the like. For example, the SurfaceFlinger can assign a composition task to a hardware composer (Hardware Composer, HWC) or a GPU through decision-making.

[0109] Generally, the HWC mainly performs simple composition processing of layer superposition. If the composition task involves special effects such as rounded corners and blur, the SurfaceFlinger can assign the composition task to the GPU to complete the composition of special effects.

[0110] In some embodiments, the SurfaceFlinger can force the composition of each frame of image to be completed by the GPU, so that the GPU can execute an eye protection algorithm in the process of image composition to realize an eye protection effect.

[0111] The HWUI can be used for image rendering. For example, a skia graphics library (an open source 2D graphics library) is used to realize graphics rendering. Specifically, the HWUI can convert the drawing logic of an application into GPU executable rendering instructions, such as GL instructions, and send them to the GPU for execution.

[0112] In some embodiments, as in Scheme II below, the HWUI can also send the eye-care algorithm to the GPU, so that the GPU can execute the eye-care algorithm in the process of image rendering, to achieve the eye-care effect.

[0113] Further, the HWUI can invoke a render node (Render Node) and a rendering pipeline (such as a Skia Pipeline, a mechanism for rendering using a skia graphics library). Among them, the Render Node can execute an image filter (image filter). The Skia Pipeline can execute an image frame rendering (render frame).

[0114] Among them, the graphics library can provide drawing and operation of 2D graphics and 3D graphics in the application. For example, the graphics library can include at least one of the following: Open Graphics Library (OpenGL), OpenGL for Embedded Systems (OpenGL ES), Vulkan, etc.

[0115] At this point, it should be noted that the application program can invoke services, functional modules, etc. provided in the underlying, such as the application framework layer, the native layer, etc. when running, such as the application program can invoke VRI to execute drawing, invoke HWUI to execute rendering, and the resources occupied by these services, functional modules, etc. after invocation belong to the resources occupied by the application process.

[0116] The kernel layer includes drivers that drive hardware work, such as display drivers and GPU drivers. The display driver can drive the display screen to display images, such as images with eye-care effects. The GPU driver can drive the GPU to work, such as driving the GPU to execute image rendering and image composition.

[0117] The image processing method provided by the embodiments of the present application can be executed in an electronic device with the above-mentioned software and hardware structure.

[0118] Before detailing the image processing method provided by the embodiments of the present application, the basic flow of image processing will be briefly introduced in combination with the software and hardware structure of the electronic device:

[0119] Referring to Figure 3 On the application side, a blank layer can be created, and then drawing and rendering can be executed, and the GPU can execute rendering instructions to draw content in the blank layer to obtain a layer with drawn content.

[0120] For example, the application process (Application Process) can obtain drawing logic through VRI to achieve image drawing.

[0121] For example, the application process can resolve the draw logic through the HWUI and generate GPU executable rendering instructions, such as GL instructions, to realize image rendering. For example, the HWUI can use a rendering pipeline (such as a Skia Pipeline) to generate the rendering instructions.

[0122] For example, the application process can submit the rendering instructions to the GPU through the HWUI, and the GPU executes the rendering instructions to draw the image content into the memory corresponding to the layer through a geometry stage, a rasterization stage, a fragment shader, and the like, thereby obtaining the content layer.

[0123] It can be understood that multiple applications can use the above-mentioned application-side process to draw content in a blank layer and obtain a content layer.

[0124] Continuing to refer to Figure 3 On the display system side, the SurfaceFlinger can decide to use GPU composition or HWC composition. If it is decided to use HWC composition, the HWC can superimpose the content layers corresponding to multiple applications to obtain an image frame and send it to the display screen for display. If it is decided to use GPU composition, the GPU can perform composition processing, such as performing composition of rounded corners, blurring, and the like, and perform layer superimposition to obtain an image frame, which is then sent to the display screen for display through the HWC.

[0125] For example, the SF process (SurfaceFlinger Process) can identify the visible layer through the SF (a main object in the SF process).

[0126] For example, the SF process can output information required for composition to a rendering engine (such as a Skia Render Engine) or the HWC through a hardware output (Output) and an output layer (OutputLayer) for image composition. If it is decided to use GPU composition, the information is output to the rendering engine, and if it is decided to use HWC composition, the information is output to the HWC. It should be noted that the Output is a virtual screen, and in general, each screen has a corresponding Output.

[0127] In addition, if it is decided to use GPU composition, the SF process can also execute a creat runtime effect shader process through the rendering engine to create a shader required for composition and encapsulate it into a composition instruction (such as a GL instruction) and send it to the GPU. Finally, the GPU executes the composition instruction to realize image composition through a geometry stage, a rasterization stage, a fragment shader, and the like.

[0128] Embodiments of the present application will illustrate the specific implementation of the eye protection algorithm here:

[0129] The electronic device can implement sampling of the blue channel by blurring processing and assigning the highest weight to the blue channel, i.e., the weight of the blue channel is higher than the weights of the red channel and the green channel, thereby enhancing the blue color of each pixel point in the image.

[0130] For example, the number of sampling points is n (n≥2, n is an integer), and for any pixel point in the image to be processed, the electronic device can calculate the color (R', G', B') of the pixel point through blurring processing as (k1*(R+R1+R2+…+Rn) / (n+1), k2*(G+G1+G2+…+Gn) / (n+1), k3*(B+B1+B2+…+Bn) / (n+1)).

[0131] wherein k1, k2 and k3 are the weights of the red (R) channel, the green (G) channel and the blue (B) channel in turn, k1, k2 and k3 all belong to (0, 1), and k3>k1, k3>k2, i.e., the weight of the blue channel is the highest.

[0132] Further, k1, k2 and k3 can be constants or variables. In a specific implementation, k1, k2 and k3 are related to the resolution of the source data and the display resolution. The source data can be a video source or an image source, such as in the scenario of playing a video, the source data is a video source.

[0133] For example, the weights (k1, k2, k3) are related to the ratio (such as p) of the resolution of the source data and the display resolution. It can be understood that blurring processing will reduce the definition and affect the viewing experience. The larger p is, the greater the ratio of the resolution of the source data and the display resolution is, indicating that less details are lost when displaying the image, and the larger p is, the smaller the ratio of the resolution of the source data and the display resolution is, indicating that more details are lost when displaying the image. Based on this, the larger p is, the larger k3 can be set by the electronic device, thereby achieving the eye protection effect to a greater extent, and correspondingly, k1 and k2 are smaller. And the smaller p is, the smaller k3 can be set by the electronic device, thereby ensuring the definition of the image to a greater extent, and correspondingly, k1 and k2 are larger. In this way, the eye protection effect can be achieved while the definition of the image is taken into account. For example, k3=m*p, m is a constant, and the larger p is, the larger k3 is.

[0134] Wherein, R, R1, R2…Rn are color values of the red channel of the pixel point, the first sampling point, the second sampling point, and the nth sampling point in turn, G, G1, G2…Gn are color values of the green channel of the pixel point, the first sampling point, the second sampling point, and the nth sampling point in turn, and B, B1, B2…Bn are color values of the blue channel of the pixel point, the first sampling point, the second sampling point, and the nth sampling point in turn.

[0135] The electronic device can sample in a convolutional manner, and the convolution kernel can be in the form of 15*15, 9*9, 3*3, etc., that is, the blur radius (denoted as rad) is (15 / 2)+1=8, (9 / 2)+1=5, (3 / 2)+1=2 in turn.

[0136] Wherein, the blur radius can be a constant. Alternatively, the blur radius can be a variable matched with different eye protection scenes. For example, in the scene where the A player plays a video, the blur radius is rad1, in the scene where the B player plays a video, the blur radius is rad2, and rad2 is different from rad1. For another example, in the scene where the video player plays a video, the blur radius is rad3, and in the scene where the reader reads a book, the blur radius is rad4, and rad4 is different from rad3.

[0137] Further, the electronic device can continuously sample or interval (jump) sample. Continuous sampling means that every point within the blur radius is sampled, and interval sampling means that at least one pixel point is sampled within the blur radius. It can be understood that interval sampling can reduce the number of samples, thereby reducing the amount of calculation.

[0138] Taking the blur radius of 5 as an example, referring to Figure 4 , for the pixel point A in the image 301, the electronic device can sample the 36 pixel points (the pixel points filled with black in the figure) in the surrounding 9*9 region with the pixel point A as the center, that is, n=36, and calculate the color (R’, G’, B’) of the pixel point A after the blur processing.

[0139] In some embodiments, the electronic device can perform the above blur processing in Linear space or LMS (Long-Medium-Short) space, thereby simplifying the calculation. Wherein, L in LMS represents Long wave, M represents Medium wave, and S represents Short wave, which correspond to three kinds of light waves to which cone cells are sensitive.

[0140] In a specific implementation, the electronic device performs the above blur processing in Linear space. In this implementation, referring to Figure 5 , the eye protection algorithm includes the following steps:

[0141] Step 1, RGB to Linear space.

[0142] The electronic device converts the RGB color of each pixel point to the Linear space to obtain the color in the Linear space. This process can be simply referred to as Gamma.

[0143] For example, the electronic device squares the color value of each channel in the RGB color to obtain the color in the Linear space.

[0144] For example, the electronic device converts the RGB color (R, G, B) of pixel point A in the image to the Linear space to obtain the color (x, y, z) of pixel point A in the Linear space. Figure 5 2 2 2

[0145] Further, the electronic device can normalize the RGB color first and then convert it to the Linear space. For example, the electronic device converts the RGB color (R, G, B) of pixel point A in the image to the Linear space to obtain the color (x, y, z) of pixel point A in the Linear space. Figure 5 2 2 2

[0146] Step 2, blur processing.

[0147] The electronic device performs blur processing on each pixel point in the Linear space to obtain the color after blur processing.

[0148] ​​​​​​​​For details, please refer to the above description of the blurring process. Only the color values of the R channel, the color values of the G channel, and the color values of the B channel are replaced by the corresponding color values in the Linear space, such as (R', G', B') = (k1*(R+R1+R2+…+Rn) / (n+1), k2*(G+G1+G2+…+Gn) / (n+1), k3*(B+B1+B2+…+Bn) / (n+1)) can be replaced by (R', G', B') = (k1*(x+x1+x2+…+xn) / (n+1), k2*(y+y1+y2+…+yn) / (n+1), k3*(z+z1+z2+…+zn) / (n+1)). Wherein, x, x1, x2…xn are the color values of the first channel (such as x channel) of the pixel point, the first sampling point, the second sampling point, and the nth sampling point in the Linear space, respectively; y, y1, y2…yn are the color values of the second channel (such as y channel) of the pixel point, the first sampling point, the second sampling point, and the nth sampling point in the Linear space, respectively; z, z1, z2…zn are the color values of the third channel (such as z channel) of the pixel point, the first sampling point, the second sampling point, and the nth sampling point in the Linear space, respectively.

[0149] For example, the electronic device samples 36 black-filled pixel points around the pixel point A in the Linear space, performs blurring processing, and calculates the color (R', G', B') of the pixel point A in the Linear space after blurring processing. Figure 5

[0150] Step 3, Linear space to RGB.

[0151] Since the blurring processing is performed in the Linear space, the electronic device needs to convert back to the RGB color, and the electronic device can display the image of the RGB color. This process can be referred to as deGamma.

[0152] For example, the electronic device converts the color (R', G', B') of the pixel point A in the Linear space after blurring processing to the RGB space, and can obtain the RGB color (R'', G'', B'') of the pixel point A.

[0153] For example, the electronic device converts the color (R', G', B') of the pixel point A in the Linear space after blurring processing to the RGB space, and can obtain the RGB color (R'', G'', B'') of the pixel point A. Figure 5

[0154] In another specific implementation, the electronic device performs the above blurring processing in the LMS space. In this implementation, please refer to Figure 6 The eye protection algorithm includes the following steps:​​

[0155] Step 4: Convert RGB to LMS color space.

[0156] Electronic devices convert the RGB colors of each pixel to the LMS color space to obtain the color in the LMS space.

[0157] Electronic devices can convert RGB colors to the LMS color space using matrix multiplication. For example, RGB colors can be viewed as a 1x3 matrix, which is then multiplied by a 3x3 transformation matrix (denoted as M). 3*3 Then, a 1*3 matrix can be obtained, which can be converted to the color values ​​of the L, M and S channels in the LMS space respectively.

[0158] For example, Figure 6 If the RGB color of pixel A is (R, G, B), then the color after conversion to LMS color space is (L, M, S) = (R, G, B) * M. 3*3 .

[0159] For example, M 3*3 = [m11, m12, m13; m21, m22, m23; m31, m32, m33], where mij represents the value in the i-th row and j-th column of the matrix. The color can then be converted to the LMS color space using the following formula:

[0160] L = m11*R + m21*G + m31*B;

[0161] M = m1² * R + m2² * G + m3² * B;

[0162] S = m13*R + m23*G + m33*B.

[0163] Furthermore, during the conversion, red can be reduced while blue and / or green can be increased, thereby further enhancing the eye protection effect. For example, in the conversion matrix, m33 > m11, m22 > m11, so that blue and green can be increased as much as possible while red is reduced.

[0164] For example, M 3*3 One possible example is [0.201, 0.022882, 0.0; 0.735, 0.845, 0.010; 0.044, 0.092, 0.899], with corresponding L = 0.201*R + 0.735*G + 0.044*B, M = 0.022882*R + 0.845*G + 0.092*B, and S = 0.0*R + 0.010*G + 0.899*B.

[0165] Step 5, blurring.

[0166] Electronic devices perform blurring on each pixel in the LMS space to obtain the blurred color.

[0167] For details, please refer to the previous explanation of blurring. You only need to replace the color values ​​of the R channel, G channel, and B channel with the corresponding color values ​​in the LMS space. For example, (R', G', B')=(k1*(R+R1+R2+…+Rn) / (n+1), k2*(G+G1+G2+…+Gn) / (n+1), k3*(B+B1+B2+…+Bn) / (n+1)) can be replaced with (R', G', B')=(k1*(L+L1+L2+…+Ln) / (n+1), k2*(M+M1+M2+…+Mn) / (n+1), k3*(S+S1+S2+…+Sn) / (n+1)). Where L, L1, L2...Ln are the color values ​​of the pixel, the 1st sampling point, the 2nd sampling point...the nth sampling point in the LMS space, respectively; M, M1, M2...Mn are the color values ​​of the pixel, the 1st sampling point, the 2nd sampling point...the nth sampling point in the LMS space, respectively; and S, S1, S2...Sn are the color values ​​of the pixel, the 1st sampling point, the 2nd sampling point...the nth sampling point in the LMS space, respectively.

[0168] For example, electronic devices Figure 6 The 36 black-filled pixels surrounding pixel A are sampled, blurred, and the blurred color (R', G', B') of pixel A in LMS space is calculated.

[0169] Step 6: Convert LMS space to RGB.

[0170] Since the blurring process is performed in LMS space, the electronic device needs to convert it back to RGB color before it can display the RGB color image.

[0171] Similar to step 4, electronic devices can convert LMS color space back to RGB color using matrix multiplication. For example, the color (R', G', B') in LMS color space can be viewed as a 1*3 matrix, which is then multiplied by a 3*3 transformation matrix (denoted as N). 3*3 If we use this method, we can obtain a 1*3 matrix, which gives us the RGB colors (R″′, G″′, B″′).

[0172] For example, electronic devices will Figure 6 The color (R', G', B') of pixel A after blurring in LMS space is converted to RGB space to obtain the RGB color of pixel A: (R″′, G″′, B″′) = (R', G', B') * N3*3 .

[0173] For example, N 3*3 =[4.028,-0.123,0.001;3.153,1.123,0.010;0.092,0.075,1.200], accordingly, R'''=4.028*R'-3.153*G'+0.092*B', G'''=-0.123*R'+1.123*G'-0.075*B', B'''=0.001*R'-0.010*G'+1.200*B'.

[0174] It should be noted that the above Figure 5 and Figure 6 Two specific implementations shown can be combined. For example, the electronic device can first convert RGB to Linear space, then convert from Linear space to LMS space, perform blur processing in LMS space, then convert back to Linear space, and finally convert back to RGB space.

[0175] The electronic device can configure the above eye protection algorithm in the electronic device, and subsequently in the process of image processing (such as image rendering or image synthesis), the electronic device can use the eye protection algorithm to enhance blue, achieving the eye protection effect.

[0176] The electronic device can implement the above eye protection algorithm in the form of a shader. The GPU in the electronic device can execute the shader when performing image rendering or image synthesis, and the blue in the processed image can be enhanced. Finally, the electronic device can display the image after enhancing the blue, achieving the eye protection effect.

[0177] The following will introduce the present application scheme in detail by using the eye protection algorithm to enhance blue in the process of image rendering and image synthesis, respectively.

[0178] Scheme one, in the process of image synthesis, using the eye protection algorithm to enhance blue.

[0179] Based on the foregoing introduction of image synthesis, it is known that in some scenarios that require synthesis of rounded corners, blur, etc., the electronic device can perform image synthesis through the GPU, while in scenarios that mainly perform layer superposition, the electronic device can perform image synthesis through the HWC. Based on this, in scheme one, the electronic device can be configured to perform image synthesis using the GPU, i.e., to force the GPU to perform image synthesis, and to perform the eye protection algorithm in the synthesis process. Thus, the eye protection effect is achieved.

[0180] In some embodiments, the electronic device mainly executes the eye protection algorithm on some behaviors that are harmful to eyes to achieve the eye protection effect. Based on this, the electronic device can detect whether there is an eye protection demand at present. If there is an eye protection demand, the electronic device sets to execute image synthesis by using the GPU. If there is no eye protection demand, the electronic device does not set to execute image synthesis by using the GPU. In this way, the eye protection demand can be targetedly met, and the load of the GPU can be reduced in the case where there is no eye protection demand.

[0181] For example, the process in which the electronic device detects whether there is an eye protection demand at present includes:

[0182] The electronic device detects the continuous activity duration of the foreground window. If the continuous activity duration exceeds duration 1, such as 5 min, 10 min, etc., the electronic device can further detect whether the window is a target window. The target window is a window of an eye protection scene, such as a video playing window, a text reading (such as a novel) window, etc. If the window is the target window, it indicates that there is an eye protection demand. It should be noted that in order to distinguish from the target window in the following scheme two, the target window in the scheme one can be recorded as target window 1.

[0183] On the contrary, if the continuous activity duration of the foreground window does not exceed duration 1, or the foreground window is not the target window 1, the electronic device can detect that there is no eye protection demand.

[0184] In practice, the foreground of the electronic device can present multiple windows at the same time, and part of the windows is not the target window 1, so the eye protection effect does not need to be achieved for the part of the windows. For example, the foreground has a status bar (used to display the battery power, network signal, etc.) window and a video playing window, the status bar window is not the target window 1, and the video playing window is the target window 1. Therefore, the eye protection effect can be achieved only for the video playing window, and the eye protection effect does not need to be achieved for the status bar window.

[0185] Further, when the electronic device executes image synthesis by using the GPU, the eye protection algorithm can be executed for the target layer (referred to as target layer) associated with the target window 1, and the eye protection algorithm is not executed for other layers except the target layer, so as to achieve the eye protection effect for the target window 1.

[0186] For example, the electronic device can set to execute image synthesis by using the GPU for the target layer, and does not set to execute image synthesis by using the GPU for other layers. Subsequently, when the GPU executes image synthesis, the eye protection algorithm can be executed for the eye protection layer which is set to execute image synthesis by using the GPU.

[0187] It should be noted that as long as one layer is set to perform image composition by GPU, GPU is used to perform image composition on all layers involved in the composition, but the GPU will not perform eye protection algorithm for layers that are not set to perform image composition by GPU.

[0188] In addition, it should be noted that not all blank layers need to be rendered by GPU to obtain a content layer as described above. Figure 3 As shown, the blank layer is obtained by GPU rendering processing. For example, for a video playing scenario, after creating a blank layer, the video file can be decoded by a video codec (MediaCodec) to obtain a video frame, and then the video frame is refreshed to the blank layer by the image compositor (SurfaceFlinger) to obtain a content layer. This process does not require GPU to participate in rendering. However, unlike image rendering, image composition is usually required for each frame of image.

[0189] That is, compared with image rendering, each frame of image has an image composition process. Therefore, solution one can be applied to the processing of each frame of image, especially in the video playing scenario, the eye protection effect can be achieved by forcing GPU to compose.

[0190] Taking a video playing scenario as an example, referring to Figure 7 Before solution one is used, the electronic device can display interface 701, and the video screen of interface 701, such as person 7011 and object 7012, is normally displayed. After solution two is used, the electronic device can display interface 702, and the edges of the video screen, such as person 7011 and object 7012, in interface 702 present a blueish effect, which is represented by a gray shadow in the figure, so that the eye protection effect can be achieved.

[0191] It should be noted that the presentation of the eye protection effect in this paper, such as Figure 7 and the presentation of the eye protection effect represented by a gray shadow in Figure 12 , Figure 17 and Figure 18 below are illustrative, and do not limit the actual eye protection effect.

[0192] The following further combines the software and hardware constitution of the electronic device to further introduce solution one in detail. Specifically, referring to Figure 8 , solution one includes the following steps:

[0193] S801, the application program 1 receives a start event of the window A.

[0194] The application program 1 can be any application program in the electronic device, such as a video player, a reader, a call, an alarm clock, etc.

[0195] The window A can be any window displayed during the running of the application 1. Taking the application 1 as a video player for example, the window A can be a home page window, a video playing window, etc.

[0196] The start event is used to trigger the display of the window A in the foreground.

[0197] For example, the window A is a home page window of a video player, and the start event is an event of starting the video player, such as a click event on an icon of the video player in a desktop.

[0198] For another example, the window A is a video playing window, and the start event can be a video playing event, such as a click event on a video cover.

[0199] S802, the application 1 sends a creation request 1 to the activity manager, and the creation request 1 is used to request the creation of an activity A corresponding to the window A.

[0200] After receiving the start event of the window A, the application 1 needs to create an activity corresponding to the window A, which can be used to manage the window A, such as controlling the foreground and background switching of the window A, operating the window, etc.

[0201] S803, the activity manager creates the activity A.

[0202] S804, after the window A is switched to the foreground by the activity A, the activity manager sends a notification 1 of the window A being switched to the foreground to the display service engine.

[0203] In a specific implementation, the activity manager executes an on Resume() method for the activity A, indicating that the window A is switched to the foreground, and then the display engine service can be sent the notification 1 of the window A being switched to the foreground.

[0204] S805, after the duration of the active time of the window A in the foreground exceeds a duration 1, the display service engine detects whether the window A is a target window 1. If yes, S806 is executed.

[0205] For example, after receiving the notification 1, the display engine service can time the window A to obtain the duration of the active time of the window A in the foreground.

[0206] In some embodiments, the display engine service can match the identifier of the window A with a window whitelist, and the identifier of the target window 1 is recorded in the window whitelist. For example, the identifiers of a video playing window, a graphic reading window, etc. are recorded in the window whitelist. If the identifier of the window A is in the window whitelist, it indicates that the window A is the target window 1; if the identifier of the window A is not in the window whitelist, it indicates that the window A is not the target window 1.

[0207] For example, the notification 1 can include the identification of the activity A, such as a name. The display service engine can determine the identification of the window A based on the identification of the activity A in the notification 1. Those skilled in the art can understand that the identification of the activity A and the identification of the window A are corresponding, and thus the display service engine can determine the identification of the window A through the identification of the activity A.

[0208] Of course, in actual implementation, the notification 1 can also directly carry the identification of the window A, and the display service engine can match the identification of the window A in the notification 1 with the window whitelist after receiving the notification 1. The embodiments of the present application do not make specific limitations on this.

[0209] If the duration of the continuous activity of the window A in the foreground exceeds the duration 1 and the window A is the target window 1, it indicates that some eye-straining windows run in the foreground for a long time, and there is an eye protection demand, and S807 can be further executed to realize eye protection.

[0210] If the duration of the continuous activity of the window A in the foreground exceeds the duration 1, but the window A is not the target window 1, there is no eye protection demand, and the display service engine can not need to execute subsequent eye protection processing for the window A.

[0211] In addition, if the duration of the continuous activity of the window A in the foreground does not exceed the duration 1, the display engine service can continue to count for the window A until the duration of the continuous activity of the window A in the foreground exceeds the duration 1, and then further detect whether the window A is the target window 1. Or, until the window A exits the running in the foreground, and the duration of the continuous activity of the window A in the foreground still does not exceed the duration 1, and then end the counting.

[0212] In a specific implementation, similar to the window A entering the running in the foreground, the activity manager can send the notification 2 that the window A exits the foreground to the display engine service when the on Pause() method is executed for the activity A. In this way, the display engine service can determine that the window A exits the running in the foreground based on the notification 2.

[0213] Further, in the case that the duration of the continuous activity of the window A in the foreground does not exceed the duration 1 or the window A is not the target window 1, the electronic device can process the window content of the window A according to the normal processing flow and display.

[0214] For example, the layer content of the target layer associated with the window A can be obtained through rendering and / or refreshing. It should be noted that if the target layer is a video content layer or the like, which does not need to obtain the layer content through rendering, the layer content can be obtained through refreshing. Then, the multiple layers (including the target layer) are composed, and the window content of the window A can be obtained and displayed.

[0215] It should be noted that after the electronic device is powered on and the initialization of the display engine service is completed, the display service engine can identify whether each window A has eye protection needs through S804-S805.

[0216] S806, the display engine service sends the layer white list of window A and the algorithm parameters of the eye protection algorithm to the image compositor. The algorithm parameters include indication information 1 and blur radius.

[0217] For ease of description, the information carrying the layer white list of window A and the algorithm parameters of the eye protection algorithm can be referred to as the first notification.

[0218] Among them, the layer white list includes the layer identifier of the target layer associated with window A, such as the layer name. It can be understood that the target layer associated with window A is the layer used to draw the window content in window A.

[0219] For example, window A is a video playing window, and the target layer in the layer white list includes the layer of video content, the layer of bullet screen, and the layer of subtitles.

[0220] For example, window A is a graphic reading window, and the target layer in the layer white list includes the layer of text and the layer of illustrations.

[0221] Among them, indication information 1 is used to indicate to start eye protection.

[0222] Among them, the blur radius is used to indicate the sampling specification, such as 15*15, 9*9, etc.

[0223] In actual implementation, a configuration table can be set in the display engine service, and the configuration table records the layer white list and the blur radius corresponding to each target window 1. In this way, the display engine service can obtain the matching layer white list and blur radius based on window A.

[0224] In a specific implementation, the display service engine can establish a configuration table based on the window identifier and the layer identifier. Generally, the identifier of the target window 1 and the layer identifier of the layer used to draw the window content of the target window 1 have the same keyword. For example, the video playing window and the layer used to draw the window content in the video playing window, such as the layer of video content, the layer of bullet screen, and the layer of subtitles, usually have the keyword video in the identifier, so the window and the layer carrying the keyword video can be configured as a corresponding target window 1 and layer white list in the configuration table.

[0225] Further, the display engine service can also dynamically adjust the blur radius based on the ambient light. Wherein, the brighter the ambient light, the greater the impact of the image clarity on the viewing, in this case, the display engine service can reduce the blur radius found in the configuration table, thereby reducing the impact of the eye-care algorithm on the image clarity. Conversely, the darker the ambient light, the relatively smaller the impact of the image clarity on the viewing, in this case, the display engine service can increase the blur radius found in the configuration table, thereby improving the eye-care effect.

[0226] Of course, if the blur radius is a constant, such as fixed at 9, the blur radius can also not be included in the algorithm parameters.

[0227] S807, the image compositor sets the GPU composition flag of the target layer to 1 based on the layer white list and the algorithm parameters, and adds the blur radius in the layer attribute of the target layer. The target layer is the layer included in the layer white list.

[0228] It can be understood that the image compositor can decide to output the layer with visible content after receiving the drawing content layer corresponding to each foreground application (each application has a corresponding window), which is used for image composition.

[0229] On the one hand, the image compositor can filter out the target layer from the layer with visible content based on the layer white list. Wherein, the layer with visible content includes the target layer associated with window A, such as layer A1 and layer A2, and also includes the layers of other windows (such as the status bar window, other windows of application program 1, and the windows of other applications running in the foreground at the same time), such as layer B1 and layer C1, and the layer white list only includes the target layer associated with window A, such as the above-mentioned layer A1 and layer A2.

[0230] The image manager matches the layer with visible content with the layer white list. If a layer with visible content (such as the above-mentioned layer A1 and layer A2) is in the layer white list, the image manager can filter out the layer with visible content as the target layer. If a layer with visible content (such as the above-mentioned layer B1 and layer C1) is not in the layer white list, the image manager can filter out the layer with visible content as the target layer. In this way, the image manager can determine the target layer that needs to execute the eye-care algorithm.

[0231] Further, the image compositor can also filter out invalid layers in combination with the layer attribute, and only execute the eye-care algorithm on the valid target layer, so as to narrow the range of eye-care layers and further improve the pertinence of eye-care.

[0232] Wherein, the invalid layer includes at least one of the following: a bullet screen layer, a layer with a display frequency greater than frequency 1, and a layer with a resolution greater than the screen resolution, which will be described below.

[0233] First, the barrage layer.

[0234] The barrage layer is used to display the barrage content. The user usually pays attention to the video content and the subtitle, but not the barrage content. Therefore, the barrage layer usually does not need to perform the eye-care algorithm. In addition, the display frequency of the barrage content is usually high, such as 60 Hz. Therefore, the eye-care algorithm needs to be performed at a higher frequency, which consumes a large amount of power. Based on this, the image compositor can filter out the barrage layer, which can improve the pertinence of eye-care and reduce power consumption.

[0235] In a specific implementation, the image compositor can filter out the layer whose background attribute in the layer attribute is transparent background, so as to filter out the barrage layer. It can be understood that the background of the barrage layer is all transparent pixels, so as not to block the video content behind. Therefore, the image compositor can accurately filter out the barrage layer based on the background attribute.

[0236] It should be noted that, especially in the implementation manner in which the display service engine establishes the configuration table based on the window identifier and the layer identifier, the barrage layer is usually in the layer whitelist of the video playing window. Therefore, the image compositor can correct the target layer determined based on the configuration table by filtering out the barrage layer.

[0237] Second, the layer whose display frequency is greater than frequency 1.

[0238] The higher the display frequency of the layer, the higher the power consumption of the GPU in performing the eye-care algorithm to obtain the image and display. Based on this, the image compositor can filter out the layer whose display frequency in the layer attribute is greater than frequency 1 (such as 48 Hz, 60 Hz), so as to reduce power consumption.

[0239] It can be understood that filtering out the layer whose display frequency is greater than frequency 1 can also filter out the barrage layer.

[0240] Third, the layer whose resolution is greater than the display resolution.

[0241] The resolution of the layer is the size of the buffer of the layer, which can reflect the resolution of the data source, such as the video source. For the layer of the video content, the size of the buffer naturally needs to store the video content corresponding to the resolution of the video source, and the size of the buffer is the resolution of the video source. In addition, the higher the resolution of the layer, the higher the resolution of the data source, and the more pixel points the GPU needs to process when performing the eye-care algorithm, and the greater the calculation amount.

[0242] The display resolution refers to the resolution of the display screen to be displayed. For example, if the display screen to be displayed is a small-size display screen, the resolution of the display screen is small, and if the display screen to be displayed is a large-size display screen, the resolution is large.

[0243] It can be understood that if the resolution of the layer is greater than the display resolution, it means that after the eye-care algorithm is performed on the layer, further compression is needed to match the display resolution, which causes the waste of part of the resources for performing the eye-care algorithm, that is, resource waste. For example, the resolution of the layer is 4k, corresponding to 4096*2160 pixels, and the display resolution is 2k, corresponding to 2048*1080 pixels. Then, the GPU needs to perform the eye-care algorithm on 4096*2160 pixels, and then compress it into 2048*1080 pixels and send it to the display, which means that about (4096*2160-2048*1080) pixels of resources are wasted.

[0244] Based on this, the image compositor can filter out the layer whose resolution in the layer attribute is greater than the display resolution, to avoid resource waste. For example, if the display resolution is 2k, then layers with a resolution higher than 2k are filtered out.

[0245] On the other hand, the image compositor can set the layer attribute for the target layer (or the effective target layer, the same below). For example, the image compositor can perform layer setting to set the layer attribute.

[0246] The setting of the layer attribute includes setting the GPU composition flag to identification 1, so as to instruct to use the GPU to perform image composition. For example, identification 1 is true. For example, the image compositor can perform layer setting to set the forced client composition mode of the target layer to true, so as to set the GPU composition flag to identification 1.

[0247] After setting identification 1, the image compositor can decide to use the GPU to perform image composition according to identification 1. Specifically, in the visible layer with content, if the GPU composition flag of one layer is identification 1, the image compositor can decide to use the GPU to perform image composition.

[0248] In addition, the setting of the layer attribute also includes setting the blur radius, which is used to determine the specification of the blur processing later.

[0249] S808, in the case that identification 1 is set in at least one layer, the image compositor generates an eye-care shader of the target layer according to the layer attribute of the target layer and the display resolution.

[0250] The layer attribute of the target layer includes the resolution of the target layer and the blur radius.

[0251] Among them, for the display resolution, please refer to the relevant description in S808, which will not be repeated here.

[0252] Among them, the eye protection shader includes an eye protection algorithm for enhancing the blue color of the pixel points in the layer content of the target layer.

[0253] The image compositor can calculate the weight of the blur processing of the target layer using the resolution of the target layer and the display resolution. For example, k3=m*p, p is the ratio of the resolution of the target layer to the display resolution. For details, please refer to the relevant description of the weight setting in the eye protection algorithm, which will not be repeated here. It should be noted that in some high dynamic range (High Dynamic Range, HDR) scenarios, the resolution of the target layer is usually higher, and accordingly, when calculating the weight, the resolution corresponding to the HDR effect is used for calculation, so that the calculated weight can match the HDR effect.

[0254] Of course, if the weight is a constant, the image compositor can also not calculate the weight of the blur processing based on the resolution of the target layer and the display resolution.

[0255] The image compositor can substitute the weight of the target layer and the blur radius of the target layer into the eye protection algorithm to generate the eye protection shader of the target layer.

[0256] S809, the image compositor sends a synthesis instruction 1 to the GPU, and the synthesis instruction 1 includes an eye protection instruction of the target layer generated based on the eye protection shader.

[0257] Among them, the eye protection instruction of the target layer included in the synthesis instruction 1 can instruct the GPU to execute the eye protection algorithm for the target layer, such as executing the eye protection algorithm for the content in the target layer to enhance the blue color. That is, the eye protection instruction is integrated into the synthesis instruction of each frame of image.

[0258] Further, the eye protection instruction includes a snapshot instruction and a fusion instruction. Among them, the snapshot instruction is used to instruct the GPU to take a snapshot of the original content (i.e. the content without adding eye protection effect) of the target layer. Among them, the fusion instruction is used to instruct the GPU to fuse the snapshot and the eye protection shader, so that each pixel point of the snapshot can be used as the input of the eye protection algorithm to obtain the result of the eye protection output, i.e. the result of enhancing the blue channel value of the pixel point, so that the eye protection effect can be realized on the original content.

[0259] It should be noted that the video content of each frame is obtained by decoding the video source by the video codec, and then the layer content of the video content layer (a target layer) is obtained, and is not rendered according to the buffer size (that is, the resolution) of the video content layer, so the resolution of the video content is most likely not matched with the resolution of the display screen used for display. Therefore, after obtaining the screenshot of the video content, the screenshot can be corrected according to the resolution of the display screen, so that the corrected screenshot has the same resolution as the display screen, and then the eye-care shader is fused, so that the layer content corresponding to the resolution of the display screen can be obtained, which will not cause resource waste, and the resolution of the layer content will not be less than the resolution of the display screen.

[0260] S810, the GPU executes the composition instruction 1 to generate the image 1.

[0261] In this way, when the GPU executes the composition instruction 1, the eye-care algorithm can be executed on the target layer based on the eye-care shader of the target layer, and the blue color in the layer can be enhanced.

[0262] In this way, when the GPU executes the composition instruction 1, the eye-care algorithm can be executed on the target layer based on the eye-care shader of the target layer, and the blue color in the layer can be enhanced.

[0263] It should be noted that the above S807-S810 belong to the process of image composition. In actual implementation, before the above S807-S810, steps of image processing such as rendering / refreshing can also be executed to generate a layer with content. For details, refer to the related description in the foregoing Figure 3 .

[0264] S811, the GPU sends the image 1 to the display screen.

[0265] For example, the GPU can send the image 1 to the display screen through the HWC.

[0266] S812, the display screen displays the image 1.

[0267] In the image 1, the window A is included, and the blue color in the window content of the window A is enhanced.

[0268] Subsequently, the above S808-S812 can be executed in a loop during the processing of each frame of image, so that the blue color in the window A in each frame of image is enhanced. It should be noted that in S808, after the eye-care shader is generated, the image compositor can store the eye-care shader in a cache, and when S808 is executed again subsequently, the eye-care shader in the cache can be reused, so that the eye-care shader does not need to be generated every frame, and the operation amount of image processing can be reduced. It should be noted that in the scheme of dynamically adjusting the blur radius based on the ambient light, a new eye-care shader can be generated based on the adjusted blur radius every frame.

[0269] In addition, the above Figure 8 flow mainly takes a window A as an example to illustrate the specific implementation of the eye-care effect. In practice, after any window is started in the foreground, a similar flow can be used to implement the eye-care effect.

[0270] It can be understood that after the electronic device uses the GPU to perform image composition on the target layer of the window A, if the window A exits the foreground running, such as being returned to the background running or being closed, the target layer of the window A and related data thereof are released, and accordingly, the electronic device cannot continue to perform the eye-care algorithm on the target layer of the window A. That is, after the window A exits the foreground running, the display engine service does not need to send a notification of closing the eye-care to the image compositor, and the eye-care algorithm on the target layer of the window A can be cancelled.

[0271] The specific implementation of S804-S809 in the flow shown in Figure 8 will be further described below in combination with the process of object calling.

[0272] The above S806 can be referred to ① in Figure 9 and Figure 10 After the display engine service (DES) in the system server process identifies the eye-care demand, the display engine service (DES) in the system server process can send the layer white list and the algorithm parameter to the SF (a main object in the SF process) in the SF process.

[0273] The above S807 can be referred to ② in Figure 9 The SF in the SF process can execute layer setting based on the layer white list and the algorithm parameter, set the GPU composition flag of the target layer to 1, and add the blur radius in the layer attribute of the target layer.

[0274] Further, referring to ② in Figure 10 :

[0275] The SF in the SF process can output (Output) a virtual object of a display screen, a synchronization layer whitelist and algorithm parameters to hardware, so that the Output can obtain parameters required in a processing process of a next frame image.

[0276] When refreshing the next frame, the SF can perform a commit process and a present process. In the commit process, the SF can find an Output to be output, i.e., a display screen to be displayed, and perform the present process to the Output. Through the present process, the corresponding Output can know the content to be displayed.

[0277] Then, the Output can perform a prepare process, filter a target layer based on the layer whitelist and the algorithm parameters obtained through the synchronization, and perform Layer Setting on the target layer, set a GPU composition flag of the target layer to identification 1, and add a blur radius in a layer attribute of the target layer.

[0278] The process of filtering the target layer includes that the Output can traverse each Layer (i.e., a visible layer with content) output by an Output layer, match the Layer with the layer whitelist, and if the Layer is included in the layer whitelist, it is indicated that the Layer is the target layer.

[0279] The process of performing Layer Setting on the target layer includes that after determining that a certain Layer is the target layer, the Output can set a force client composition mode of the Layer to true, so as to set the GPU composition flag to identification 1.

[0280] The above S808 can refer to ③-⑥ in Figure 9

[0281] In ③ of Figure 9 , the SF process (such as the Output therein) can decide to cut GPU composition based on identification 1, i.e., decide to perform image synthesis by using a GPU.

[0282] Further, refer to Figure 10 ​In step ③, Output can determine the compositing method (updateCompositionState). By querying the compositing identifiers of each Layer output by OutputLayer, if a Layer's force clientcomposition mode is set to true, it can decide to use the GPU to perform image compositing. Next, Output can execute the finish frame process, indicating that the necessary steps for a frame have been completed.

[0283] exist Figure 9 In section ④, OutputLayer can output the algorithm parameters and buffer size (i.e., the layer resolution) of each layer. Figure 9 In section ⑤, Output can output the display resolution. Figure 9 In section ⑥, the rendering engine (such as Skia Render Engine) can generate eye-friendly shaders based on the above algorithm parameters, layer resolution, and display resolution.

[0284] Further, see Figure 10 Sections ④-⑥ describe the process: The Render Engine executes the Drawlayer internal process, creating a runtime effect shader (createRuntimeEffectShader), and wrapping the eye-protection algorithm (an object that the SF process can call) into a shader object. The shader needs to bind some external parameters, such as the blur radius and blur weights, to achieve the eye-protection effect. Specifically, the process of binding external parameters includes: the RenderEngine calculating the weights (e.g., calculating the blur weights in the eye-protection algorithm object based on the layer's resolution and display resolution, including the aforementioned k1, k2, and k3); the Render Engine object initializing the eye-protection algorithm (e.g., executing the buildRuntimeEffectShader process); the Render Engine binding the blur radius and weights to the eye-protection algorithm (e.g., executing the createRuntimeEffectShader process); and finally, the eye-protection algorithm being parsed (e.g., executing the build shader process) and wrapped into an eye-protection shader with bound external parameters.

[0285] The above S809 can be found in Figure 9 In section ⑦, the Render Engine can submit the eye-friendly shader to the GPU, enabling the algorithm to take effect during the compositing stage.

[0286] Further, seeFigure 10 In ⑦, the Render Engine can execute the Flush procedure to submit the eye-care shader to the GPU.

[0287] The S810 can refer to Figure 9 In ⑧, the GPU can execute the eye-care shader, so as to integrate the eye-care algorithm in the compositing stage and realize the eye-care effect.

[0288] Further, after the eye-care is enabled, the electronic device can further detect whether a condition for exiting the eye-care is met. If the condition for exiting the eye-care is met, the electronic device can end executing the eye-care algorithm by the GPU in the process of image compositing.

[0289] In a specific implementation, the condition for exiting the eye-care includes that the temperature of the electronic device exceeds a temperature threshold. It can be understood that the temperature of the electronic device increases, which usually leads to performance degradation. At the same time, the GPU executing the eye-care algorithm leads to increased power consumption. Based on this, the electronic device can end executing the eye-care algorithm by the GPU in the process of image compositing to realize the eye-care effect, such as canceling the image compositing by the GPU, when the temperature exceeds the temperature threshold.

[0290] It should be noted that in the process of processing a frame of image, the electronic device can itself need to perform image compositing by the GPU, such as compositing a rounded corner effect by the GPU, therefore, ending executing the eye-care algorithm by the GPU in the process of image compositing does not mean not performing image compositing by the GPU.

[0291] Referring to Figure 11 The scheme one can further include the following steps:

[0292] S1101, the display engine service detects whether a condition for exiting the eye-care is met.

[0293] S1102, after detecting that the condition for exiting the eye-care is met, the display engine service sends indication information 2 to the image compositor.

[0294] The indication information 2 indicates to close the eye-care.

[0295] S1103, the image compositor sets a GPU compositing flag of a target layer to an identifier 2 based on the indication information 2.

[0296] The identifier 2 indicates not to perform image compositing by the GPU. For example, the forceclientcomposition mode of the target layer is set to false, so as to realize setting the GPU compositing flag of the target layer to the identifier 2.

[0297] In a case where the GPU synthesis flag bit of all the layers output by the image synthesizer is identification 2, a default image synthesis manner, i.e., a manner of HWC synthesis, can be adopted to complete image synthesis. Details are shown in S1104-S1107.

[0298] S1104, in a case where the GPU synthesis flag bit of all the layers is identification 2, the image synthesizer sends a synthesis instruction 2 to the HWC, and the synthesis instruction 2 does not include an eye-care shader.

[0299] S1105, the HWC executes the synthesis instruction 2 to synthesize an image 2.

[0300] For example, the HWC superimposes the layers to obtain the image 2, and in this process, the HWC does not execute the eye-care algorithm and does not enhance the blue color.

[0301] S1106, the HWC sends the image 2 to the display screen.

[0302] S1107, the display screen displays the image 2.

[0303] For example, in a case where the window A does not exit the foreground, the window A is also included in the image 2, but the blue color in the window A is not enhanced, and the eye-care cannot be achieved.

[0304] Scheme two, in the process of image rendering, the eye-care algorithm is used to enhance the blue color.

[0305] The electronic device can execute image rendering by the GPU. Based on this, the electronic device can execute the eye-care algorithm in the process of image rendering by the GPU, so as to achieve the eye-care effect.

[0306] In addition, the power consumption of the GPU in the process of image rendering is generally lower than that in the process of image synthesis, and accordingly, the execution of the eye-care algorithm in the process of image rendering by the GPU has less impact on the power consumption of the GPU, which is beneficial to guarantee the use performance of the electronic device.

[0307] Similar to the foregoing scheme one, in some embodiments, the electronic device mainly executes the eye-care algorithm on some behaviors that are harmful to the eyes to achieve the eye-care effect. Based on this, the electronic device can detect whether there is an eye-care demand at present. If there is an eye-care demand, the electronic device executes the eye-care algorithm in the process of image rendering by the GPU. If there is no eye-care demand, the electronic device does not execute the eye-care algorithm in the process of image rendering by the GPU. In this way, the eye-care demand can be targetedly met, and the load of the GPU can be reduced in a case where there is no eye-care demand.

[0308] For example, the electronic device can run the target window in the foreground, and the target window is in the foreground for a duration longer than duration 1. When the electronic device detects that there is an eye protection demand, the target window is in the foreground for a duration longer than duration 1. Similarly, to distinguish the target window in scheme 1, the target window in scheme 2 can be referred to as target window 2.

[0309] In practice, the foreground of the electronic device can present multiple windows at the same time, and part of the windows are not target window 2, and the eye protection effect does not need to be implemented for the part of the windows.

[0310] Further, when the electronic device performs image synthesis by using the GPU, the eye protection algorithm can be performed for the target layer (referred to as target layer) involved in the target window 2.

[0311] The part not described in detail in the above scheme 2 can be referred to the description of scheme 1 in the foregoing description, and will not be described here.

[0312] It should be noted that based on the foregoing description, in the scenario of playing a video, the image synthesizer (SurfaceFlinger) can refresh the video frame to the blank layer to obtain the content layer, and the process does not need to be rendered by the GPU. At the same time, in addition to the above-mentioned scenario of playing a video, such as a scenario of reading by a reader, the GPU is usually involved in image rendering to obtain the content layer. It can be seen that scheme 2 is mainly applicable to the scenario in which the GPU is involved in image rendering, and accordingly, the target window 2 in scheme 2 can be different from the target window 1 in scheme 1.

[0313] Typically, the target window 1 in scheme 1 can include a video playing window and a graphic-text reading window; and the target window 2 in scheme 2 can include a graphic-text reading window, but usually does not include a video playing window.

[0314] Taking a reading scenario as an example, referring to Figure 12 Before scheme 2 is adopted, the electronic device can display interface 1201, and the text such as “ABC” in interface 1201 is normally displayed. After scheme 2 is adopted, the electronic device can display interface 1202, and the text such as “ABC” in interface 1202 is surrounded by a blue effect, which is represented by a gray shadow in the figure, so that the eye protection effect can be achieved.

[0315] The following further combines the hardware and software constitution of the electronic device to further introduce scheme 2 in detail. Specifically, referring to Figure 13 Scheme 2 includes the following steps:

[0316] S1301, the application program 2 receives a start event of window B.

[0317] The application program 2 can be any application program in the electronic device, and the window B can be any window displayed during running of the application program 2.

[0318] In S1302, the application program 2 sends a creation request 2 to the activity manager, where the creation request 2 is used to request creation of an activity B corresponding to the window B.

[0319] In S1303, the activity manager creates the activity B.

[0320] In S1304, after switching the window B to the foreground through the activity B, the activity manager sends a notification 3 of the window B switching to the foreground to the display service engine.

[0321] In S1305, after the duration of the activity of the window B in the foreground exceeds a duration 1, the display service engine detects whether the window B is a target window 2. If yes, S1306 is performed.

[0322] It should be noted that the target window 2 in S1305 can be different from the target window 2 in the foregoing S805. For example, the target window 2 in S1305 generally does not include a video playing window.

[0323] In S1306, the display engine service sends a layer whitelist of the window B and algorithm parameters of the eye-care algorithm to the view root implementation, where the algorithm parameters include the indication information 1 and the blur radius.

[0324] For ease of description, information carrying the layer whitelist of the window B and the algorithm parameters of the eye-care algorithm can be referred to as a second notification.

[0325] It should be noted that there is originally no cross-process (binder) communication link between the display engine service and the view root implementation. Based on this, the application program 2 can register a listener to the display engine service through the view root implementation, and the window identifier of the target window 2 (such as the window B) can be carried during the registration. Subsequently, the display engine service can feed back the layer whitelist and the algorithm parameters to the view root implementation after detecting that the duration of the activity of the target window 2 exceeds the duration 1. In this way, cross-process communication between the display engine service and the view root implementation can be achieved.

[0326] Subsequently, the view root implementation can also cancel the registration to the display engine service after the life cycle of the target window 2 ends, such as after doDie. In this way, the display engine service no longer continues the listener of the target window 2.

[0327] In the second scheme, the eye-care algorithm is executed in the process of image rendering, and therefore the eye-care algorithm should be integrated into the rendering process. Accordingly, the display engine service can send the layer whitelist and algorithm parameters to the view root implementation for the view root implementation to integrate the eye-care algorithm into the rendering process.

[0328] It can be understood that the view root implementation corresponds to a window, and generally one window corresponds to one instance of the view root implementation, such as that the window B corresponds to one instance of the view root implementation (denoted as view root implementation B). Accordingly, the display service engine can send the layer whitelist and algorithm parameters to the view root implementation that registers the listening of the corresponding window. For example, the view root implementation B registers the listening of the window B to the display engine service, and then sends the layer whitelist and algorithm parameters to the view root implementation B.

[0329] For the parts not described in detail in S1301-S1306 above, refer to the description of S801-S806 above, which will not be repeated here.

[0330] S1307. The view root implementation sets the eye-care shader of the target layer based on the layer whitelist and the algorithm parameters, and the target layer is included in the layer whitelist.

[0331] Different from the first scheme above, in the second scheme, there is no need to use GPU composition compulsorily, and therefore there is no need to set the GPU composition flag of the target layer to 1.

[0332] For example, the view root implementation can calculate the weight of the blur processing, bind the weight and the blur radius to the eye-care algorithm, and generate the eye-care shader of each target layer.

[0333] For the parts not described in detail in S1307 above, refer to the description of S808 above, which will not be repeated here.

[0334] S1308. The view root implementation sets the eye-care shader of the target layer to the hardware-accelerated image rendering component.

[0335] In this way, the hardware-accelerated image rendering component can subsequently generate rendering instructions for implementing the eye-care effect based on the eye-care shader.

[0336] S1309. The view root implementation sends the drawing logic 1 to the hardware-accelerated image rendering component.

[0337] After each frame refresh, the view root implementation can send the drawing logic 1, so that the subsequent GPU can render a new frame of image.

[0338] S1310. The hardware-accelerated image rendering component processes the drawing logic 1 to obtain the rendering instruction 1, and the rendering instruction 1 includes the eye-care instruction of the target layer generated based on the eye-care shader.

[0339] For example, the hardware accelerated image rendering component can generate the skia encapsulated rendering instruction 1 based on the drawing logic 1, so that the GPU can execute.

[0340] In the rendering instruction 1, the eye-care instruction for the target layer is included, which can instruct the GPU to execute the eye-care algorithm for the target layer, such as executing the eye-care algorithm on the content drawn in the target layer to enhance the blue color. That is, the eye-care instruction is integrated into the rendering instruction of each frame of image.

[0341] Further, the eye-care instruction includes a snapshot instruction and a fusion instruction. The snapshot instruction is used to instruct the GPU to take a snapshot of the original content (i.e., the content without the eye-care effect) rendered by the target layer. The fusion instruction is used to instruct the GPU to fuse the snapshot and the eye-care shader, so that each pixel point of the snapshot can be taken as an input of the eye-care algorithm to obtain the result of the eye-care output, i.e., the result of enhancing the value of the blue channel of the pixel point, so that the eye-care effect can be realized on the original content.

[0342] S1311, the hardware accelerated image rendering component sends the rendering instruction 1 to the GPU.

[0343] S1312, the GPU executes the rendering instruction 1, draws each layer, and fuses the eye-care shader in the target layer.

[0344] For the target layer, the GPU takes a snapshot of the original content of the target layer by executing the eye-care instruction, and fuses the eye-care shader based on the original content of the target layer, so as to realize the eye-care effect.

[0345] In this way, the GPU can enhance the blue color in the target layer during the execution of image rendering.

[0346] At this point, the image rendering is completed. Subsequently, image composition, display sending and other steps can be performed, and finally image 3 is obtained and sent to the display screen for display.

[0347] S1313, the display screen displays the image 3.

[0348] The image 3 includes the window B, and the blue color in the window B is enhanced.

[0349] Subsequently, the above S1309-S1312 can be executed in a loop during the processing of each frame of image, so that the blue color in the window B is enhanced in each frame of image.

[0350] In addition, in the process of the above Figure 13 flow, a window B is mainly taken as an example to illustrate the specific implementation of the eye-care effect. In practice, after starting any window in the foreground, a similar process can be used to realize the eye-care effect.

[0351] It can be understood that if the window B exits the foreground running, such as returning to the background running or being closed, the target layer of the window B and its related data are released, and accordingly, the electronic device cannot continue to perform the eye-care algorithm on the target layer of the window B. That is, after the window B exits the foreground running, the display engine service does not need to send a notification of closing eye-care to the view root implementation, and the eye-care algorithm on the target layer of the window B can be cancelled.

[0352] The above-mentioned process of object calling is further described in detail below in combination with the specific implementation of the flow shown in Figure 13 .

[0353] Referring to (1) in Figure 14 , before S1306, the view root implementation in the application process can register a listener to the display engine service in the system service process. For example, after the window is created, such as after the setView is performed, the view root implementation corresponding to the window can register a listener to the display engine service.

[0354] Further, referring to (1) in Figure 15 , the view root implementation can register a listener to the display engine service through the eye-care algorithm (an object of the eye-care algorithm that can be called by the application process). Wherein, by registering a listener through the eye-care algorithm, the parameters required by the eye-care algorithm can be determined, so that the display engine service needs to feed back the parameters such as the blur radius after identifying that there is an eye-care demand.

[0355] The above-mentioned S1304 and S1305 can refer to (8) in Figure 15 , after the on Resume() method is performed, so that the window runs in the foreground, the display engine service can listen to the window switching. In this case, the display engine service can perform the checking of the window whitelist, temperature, time and other information, and detect whether the eye-care demand is met.

[0356] The above-mentioned S1306 can refer to (2) in Figure 14 and Figure 15 , after the display engine service listens to the duration of the continuous activity of the target window 2 in the foreground exceeding the duration 1, the display engine service can feed back (callback, or called callback) the layer whitelist and the algorithm parameters corresponding to the target window 2 to the view root implementation.

[0357] The above-mentioned S1307 can refer to (3) in Figure 14 , after the view root implementation receives the layer whitelist and the algorithm parameters, the view root implementation can generate the eye-care shader of the target layer.

[0358] Further, referring to Figure 15In (3) of the above, the view root implementation can transmit the blur radius to the view, the view can accept the radius (acceptRadius), and the view root implementation can set the eye-care shader through the view, such as executing applyeffect implementation to set the eye-care shader.

[0359] The above S1308 can refer to Figure 14 In (4) of the above, the view root implementation can send the eye-care shader to the HWUI in the application process through the view.

[0360] For example, the view root implementation can operate the root node (DecorView, i.e. Figure 14 the view in the above), encapsulate the eye-care shader as a runtime shader (RuntimeShader), and set the eye-care shader to the render node corresponding to the HWUI by executing setRenderEffect (createRuntimeShaderEffect (eye-care shader)). For example, the view can parse the eye-care shader into an image filter (ImageFilter) through the execution of the render effect (RenderEffect) as a filter for the rendering of the render node.

[0361] The above S1310 can refer to Figure 14 In (5) of the above, the rendering pipeline (such as Skia Pipeline) in the HWUI can perform traversal based on the traversal of the view root implementation, traverse each render node, and encapsulate the rendering instructions of each render node. If the render node has the effect of the eye-care shader, the HWUI can also encapsulate the eye-care instructions, such as the screenshot instruction and the fusion instruction.

[0362] The above S1311 can refer to Figure 14 In (6) of the above, the rendering pipeline in the HWUI can sequentially send the rendering instructions to the GPU in the order of traversal, so that the eye-care algorithm is effective in the rendering stage. For example, the rendering pipeline can execute the Flush process to sequentially send the rendering instructions to the GPU.

[0363] The above S1312 can refer to Figure 14 In (7) of the above, the GPU can execute the eye-care shader, so that the eye-care algorithm is integrated in the rendering stage, and the eye-care effect is achieved.

[0364] Similarly to the first scheme, after the eye-care is started, the electronic device can also detect whether the condition for exiting the eye-care is met. If the condition for exiting the eye-care is met, the electronic device can end the execution of the eye-care algorithm in the process of image rendering through the GPU.

[0365] Referring toFigure 16 The second scheme can further include the following steps:

[0366] S1601, the display engine service detects whether the condition for exiting the eye care is met.

[0367] S1602, after detecting that the condition for exiting the eye care is met, the display engine service sends indication information 2 to the view root implementation.

[0368] The indication information 2 indicates to close the eye care.

[0369] S1603, the view root implementation cancels the eye care shader set in the hardware-accelerated image rendering component based on the indication information 2.

[0370] For example, the view root implementation can send a cancellation instruction of the eye care shader to the corresponding rendering node in the hardware-accelerated image rendering component, so that the corresponding rendering node can delete the eye care shader.

[0371] S1604, the view root implementation sends drawing logic 2 to the hardware-accelerated image rendering component.

[0372] Similarly, the view root implementation can send the drawing logic 2 after each frame refresh, so that the subsequent GPU can render a new frame of image.

[0373] S1605, the hardware-accelerated image rendering component processes to obtain rendering instruction 2 based on the drawing logic 2.

[0374] Since the eye care shader has been cancelled, the rendering instruction 2 obtained by the hardware-accelerated image package no longer includes the eye care instruction obtained based on the eye care shader.

[0375] S1606, the hardware-accelerated image rendering component sends the rendering instruction 2 to the GPU.

[0376] S1607, the GPU executes the rendering instruction 2 to draw each layer.

[0377] Since the rendering instruction 2 does not include the eye care instruction, the GPU will not further fuse the eye care shader on the original content of the target layer, so as to not realize the eye care effect.

[0378] Similarly, subsequent steps such as image composition and display sending can also be performed, and finally image 4 is obtained and sent to the display screen for display.

[0379] S1608, the display screen displays the image 4.

[0380] If the window B still does not exit the running in the foreground, the window B is also included in the image 4, but the blue color in the window B is not enhanced.

[0381] Therefore, it should be noted that Scheme 1 and Scheme 2 can be combined. Below, using Scheme 1 (where the target window includes a video playback window) and Scheme 2 (where the target window includes a text and image reading window) as examples, several typical combination methods are introduced:

[0382] For combination method one, see Figure 17 In (a) of the example, when the electronic device is running a video playback window 1701 in the foreground, Scheme 1 can be adopted, which allows the GPU to perform an eye-protection algorithm on the target layer involved in the video playback window 1701 during the image synthesis process, thereby achieving an eye-protection effect when playing the video. Subsequently, when switching to... Figure 17 After (b) in the above, when the electronic device is running the image and text reading window 1702 in the foreground, the second scheme can be adopted, so that the GPU performs an eye protection algorithm on the target layer involved in the image and text reading window 1702 during the image rendering process, so as to achieve an eye protection effect when reading.

[0383] Of course, the same applies when switching from a text and image reading window running in the foreground to a video playback window running in the foreground.

[0384] For combination method two, see [link / reference] Figure 18 If an electronic device runs a video playback window 1801 and a text and image reading window 1802 in a split-screen manner in the foreground, the electronic device can adopt Scheme 2, which allows the GPU to perform an eye protection algorithm on the target layer involved in the text and image reading window 1802 during the image rendering process, and can adopt Scheme 1, which allows the GPU to perform an eye protection algorithm on the target layer involved in the video playback window 1801 during the image compositing process, thereby achieving an eye protection effect while playing videos and reading.

[0385] Of course, the electronic device can also run the video playback window and the text reading window in the foreground in the following ways: the text reading window runs in full screen, and the video playback window runs in a floating window; or, the video playback window runs in full screen, and the text reading window runs in a floating window, etc. This application does not make specific limitations on this.

[0386] By combining the above, electronic devices can adopt Scheme 2 for text and image reading windows, which can reduce the impact on GPU power consumption while achieving eye protection; and adopt Scheme 1 for video playback windows, which can achieve eye protection without GPU participation in image rendering.

[0387] In actual implementation, after detecting that the eye protection requirement is met, the electronic device can further decide to adopt scheme one or scheme two, and implement the eye protection effect based on the decision result. Specifically, the electronic device can decide based on the window white lists of the target windows to which scheme one and scheme two are respectively applicable. If the current window (such as the video playing window 1701 described above) is in the window white list of scheme one, scheme one is adopted to implement the eye protection effect. If the current window (such as the video playing window 1702 described above) is in the window white list of scheme two, scheme two is adopted to implement the eye protection effect.

[0388] Further, the window white list to which scheme two is applicable can be a window white list composed of the target windows for which the view root implementation registers to listen to the display engine service. The window white list to which scheme one is applicable is a white list pre-configured in the display engine service. After detecting that the eye protection requirement is met, the display engine service can query whether the current window is a target window for which registration to listen is performed. If the current window is a target window for which registration to listen is performed, the display engine service can feed back the layer white list and algorithm parameters to the view root implementation, so as to implement the eye protection effect through scheme two. If the current window is not a target window for which registration to listen is performed, the display engine service can feed back the layer white list and algorithm parameters to the image compositor, so as to implement the eye protection effect through scheme one.

[0389] Of course, if the foreground of the electronic device runs multiple windows, and at least two windows of the multiple windows meet the eye protection requirement, the electronic device can detect, for each window of the at least two windows, a window white list in which the window is located.

[0390] If the at least two windows are both in the white list of scheme one, the electronic device can implement the eye protection effect for the at least two windows through scheme one.

[0391] If the at least two windows are both in the white list of scheme two, the electronic device can implement the eye protection effect for the at least two windows through scheme two.

[0392] If a part of the at least two windows (such as the video playing window 1801 in FIG. 18) is in the window white list of scheme one, and another part of the at least two windows (such as the graphic-text reading window 1802 in FIG. 18) is in the window white list of scheme two, the electronic device can implement the eye protection effect for the part of the windows through scheme one, and implement the eye protection effect for the other part of the windows through scheme two. Figure 18 Figure 18

[0393] ​​The electronic device can include a memory and one or more processors (such as a CPU, a GPU, and the like). The memory and the processor are coupled. The memory is configured to store computer program codes, the computer program codes including computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps of the device in the above method embodiments.

[0394] The electronic device can include a memory and one or more processors (such as a CPU, a GPU, and the like). The memory and the processor are coupled. The memory is configured to store computer program codes, the computer program codes including computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps of the device in the above method embodiments.

[0395] The electronic device can include a memory and one or more processors (such as a CPU, a GPU, and the like). The memory and the processor are coupled. The memory is configured to store computer program codes, the computer program codes including computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps of the device in the above method embodiments.

[0396] The electronic device can include a memory and one or more processors (such as a CPU, a GPU, and the like). The memory and the processor are coupled. The memory is configured to store computer program codes, the computer program codes including computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps of the device in the above method embodiments.

[0397] In addition, the embodiments of the present application also provide a device, which can be a chip, a component or a module. The device can include a processor and a memory connected to each other. The memory is configured to store computer execution instructions. When the device is running, the processor can execute the computer execution instructions stored in the memory, so that the chip performs the image processing method in the above method embodiments.

[0398] The electronic device, the computer storage medium, the computer program product or the chip provided by the embodiments of the present application are used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be described herein again.

[0399] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0400] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the division of the apparatus embodiments is merely an example, and the division of the modules or units can be different, for example, multiple units or components can be combined or integrated into another unit, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0401] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, i.e., can be located in one place or distributed in multiple different places. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0402] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0403] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or substantially or all or part of the technical solutions that make contributions to the prior art can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0404] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting them. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. An image processing method, characterized by, The method comprises: running a first application, the first application comprising a first window; generating first layer content of a first target layer associated with the first window before a display duration of the first window reaches a first duration, displaying window content of the first window based on the first layer content; generating second layer content of the first target layer after the display duration of the first window reaches the first duration, enhancing a value of a blue channel of a pixel point in the second layer content to obtain third layer content, and displaying window content of the first window based on the third layer content.

2. The method of claim 1, wherein, The enhancement of the value of the blue channel of the pixel point in the second layer content comprises: enhancing the value of the blue channel of the pixel point in the second layer content in an image synthesis stage or an image rendering stage.

3. The method of claim 2, wherein, The enhancement of the value of the blue channel of the pixel point in the second layer content in the image synthesis stage or the image rendering stage comprises: in a case where the first window is a video playing window, enhancing the value of the blue channel of the pixel point in the second layer content in the image synthesis stage; in a case where the first window is a graphic-text reading window, enhancing the value of the blue channel of the pixel point in the second layer content in the image rendering stage.

4. The method of claim 3, wherein, Before the enhancement of the value of the blue channel of the pixel point in the second layer content in the image synthesis stage, the method further comprises: in a case where the first window is a video window, setting a GPU synthesis flag bit of the first target layer to a first identifier; determining to perform image synthesis by using a GPU based on the first identifier; The enhancement of the value of the blue channel of the pixel point in the second layer content in the image synthesis stage comprises: enhancing the value of the blue channel of the pixel point in the second layer content in the GPU image synthesis stage.

5. The method according to claim 3 or 4, characterized in that, After the generation of the second layer content of the first target layer, the method further comprises: in a case where the first window is not the video playing window and the graphic-text reading window, not enhancing the value of the blue channel of the pixel point in the second layer content, and displaying window content of the first window based on the second layer content.

6. The method according to any one of claims 2-5, characterized in that, The generation of the second layer content of the first target layer associated with the first window comprises: rendering the second layer content in the first target layer; The enhancement of the value of the blue channel of the pixel point in the second layer content in the image rendering stage to obtain third layer content comprises: in the image rendering stage, obtaining a first screenshot of the second layer content, fusing the first screenshot with a first shader to obtain the third layer content, and the first shader comprises an algorithm for enhancing the value of the blue channel; The display of the window content of the first window based on the third layer content comprises: synthesizing layer content of a plurality of layers to obtain the window content of the first window and display, and the layer content of the plurality of layers comprises the third layer content.

7. The method according to any one of claims 2-5, characterized in that, The generation of the second layer content of the first target layer associated with the first window comprises: rendering and / or refreshing the second layer content in the first target layer; In the image compositing stage, the blue channel values ​​of pixels in the second layer are enhanced to obtain the third layer content, including: In the image compositing stage, a first screenshot of the second layer content is obtained, the first screenshot is corrected according to the resolution of the first display screen, and the corrected first screenshot is merged with the second shader to obtain the third layer content. The second shader includes an algorithm to enhance the value of the blue channel. The first display screen is the display screen that displays the first window. The step of displaying the window content of the first window based on the content of the third layer includes: The content of multiple layers is superimposed to obtain the window content of the first window and then displayed. The content of the multiple layers includes the content of the third layer.

8. The method according to any one of claims 1-7, characterized in that, Before enhancing the blue channel values ​​of pixels in the second layer content, the method further includes: Filter out invalid layers in the first target layer to obtain a filtered first target layer; wherein, the invalid layers include at least one of the following: a layer for drawing bullet screen content, a layer with a display frequency greater than a first frequency, and a layer with a buffer size greater than the resolution of the first display screen, wherein the first display screen is the display screen that displays the first window; Enhancing the blue channel values ​​of pixels in the second layer content includes: Enhance the blue channel value of pixels in the second layer content of the first target layer after filtering.

9. The method according to any one of claims 1-8, characterized in that, Enhancing the blue channel values ​​of pixels in the second layer content includes: Based on the resolution of the first display screen, the blur radius of the first target layer, and the buffer size of the first target layer, the value of the blue channel of the pixels in the content of the second layer is enhanced; The blur radius is used to indicate the sampling specification when enhancing the value of the blue channel, and the first display screen is the display screen that displays the first window.

10. The method according to claim 9, characterized in that, The blur radius is positively correlated with the degree of enhancement of the blue channel value; The first ratio is positively correlated with the enhancement level of the blue channel value, and the first ratio is the ratio of the buffer size of the first target layer to the resolution of the first display screen.

11. The method according to claim 9 or 10, characterized in that, The method is applied to an electronic device, and the method further includes: The blur radius is corrected based on the ambient light level of the environment in which the electronic device is located; The enhancement of the blue channel value in the content of the second layer based on the resolution of the first display screen, the blur radius of the first target layer, and the buffer size of the first target layer includes: Based on the resolution of the first display screen, the corrected blur radius, and the buffer size of the first target layer, the value of the blue channel of the pixels in the content of the second layer is enhanced.

12. The method of claim 11, wherein, The step of correcting the blur radius of the first target layer based on the ambient light intensity of the environment in which the electronic device is located includes: The ambient light brightness is a first brightness, and the corrected blur radius is a first radius; The ambient light brightness is the second brightness, and the corrected blur radius is the second radius; Wherein, the first brightness is lower than the second brightness, and the first radius is greater than the second radius.

13. The method of any one of claims 3-5, wherein, The method is applied to an electronic device, the electronic device comprising a display engine service, an image compositor and a view root implementation; In a case where the first window is a video playing window, enhancing a value of a blue channel of a pixel in the second layer content in an image composition stage, comprising: In a case where the display engine service detects that the first window is a video playing window, the display engine service sends a first notification to the image compositor; In response to the first notification, the image compositor controls to enhance the value of the blue channel of the pixel in the second layer content in the image composition stage; In a case where the first window is a graphic reading window, enhancing a value of a blue channel of a pixel in the second layer content in an image rendering stage, comprising: In a case where the display engine service detects that the first window is a graphic reading window, the display engine service sends a second notification to the view root implementation; In response to the second notification, the view root implementation controls to enhance the value of the blue channel of the pixel in the second layer content in the image rendering stage.

14. The method of claim 13, wherein, After running a first application, the method further comprises: The view root implementation registers a listening of the first window to the display engine service, the second type window comprising a window of which the listening is registered by the view root implementation; The display engine service sends a second notification to the view root implementation, comprising: The display engine service sends the second notification to the view root implementation which registers the listening of the first window.

15. An image processing method, characterized by, The method comprises: displaying a first interface, the first interface comprising a video playing window and a graphic reading window; generating fourth layer content of a second target layer associated with the video playing window, enhancing a value of a blue channel of a pixel in the fourth layer content in an image composition stage to obtain fifth layer content, and displaying window content of the video playing window based on the fifth layer content; generating sixth layer content of a third target layer associated with the graphic reading window, enhancing a value of a blue channel of a pixel in the sixth layer content in an image rendering stage to obtain seventh layer content, and displaying window content of the graphic reading window based on the seventh layer content.

16. The method of claim 15, wherein, enhancing the value of the blue channel of the pixel in the fourth layer content in the image composition stage, comprising: enhancing the value of the blue channel of the pixel in the fourth layer content in the image composition stage after a display duration of the video playing window reaches a first duration; enhancing the value of the blue channel of the pixel in the fifth layer content in the image rendering stage, comprising: enhancing the value of the blue channel of the pixel in the fifth layer content in the image rendering stage after a display duration of the graphic reading window reaches a first duration.

17. The method according to claim 15 or 16, characterized in that The generating of the sixth layer content of the third target layer associated with the graphic reading window, comprising: rendering the sixth layer content in the third target layer; enhancing the value of the blue channel of the pixel in the sixth layer content in the image rendering stage to obtain the seventh layer content, comprising: In the image rendering stage, a second screenshot of the sixth layer content is obtained, the second screenshot is fused with a third shader to obtain the seventh layer content, and the third shader includes an algorithm for enhancing the value of the blue channel; The method further includes: The window content of the video playing window is displayed based on the fifth layer content.

18. The method according to any one of claims 15-17, characterized by, The window content of the video playing window is obtained by superimposing the layer content of multiple layers, and the layer content of the multiple layers includes the fifth layer content. Before the step of enhancing the value of the blue channel of the pixel point in the fourth layer content in the image synthesis stage, the method further includes: The invalid layer in the second target layer is filtered to obtain a filtered second target layer, wherein the invalid layer includes at least one of the following: a layer for drawing a barrage content, a layer with a display frequency greater than a first frequency, and a layer with a buffer size greater than the resolution of the second display screen, and the second display screen is a display screen for displaying the video playing window. The step of enhancing the value of the blue channel of the pixel point in the fourth layer content in the image synthesis stage includes: In the image synthesis stage, the value of the blue channel of the pixel point in the fourth layer content of the filtered second target layer is enhanced. The step of enhancing the value of the blue channel of the pixel point in the fourth layer content in the image synthesis stage includes:

19. The method according to any one of claims 15-18, characterized by, The value of the blue channel of the pixel point in the fourth layer content is enhanced based on the resolution of the second display screen, the blur radius of the second target layer, and the buffer size of the second target layer. The blur radius is used to indicate the sampling specification when the value of the blue channel is enhanced, and the second display screen is a display screen for displaying the video playing window. The method is applied to an electronic device, and the method further includes: The blur radius is corrected based on the ambient light brightness of the environment in which the electronic device is located.

20. The method of any one of claims 15-19, wherein, The step of enhancing the value of the blue channel of the pixel point in the fourth layer content based on the resolution of the second display screen, the blur radius of the second target layer, and the buffer size of the second target layer includes: The value of the blue channel of the pixel point in the fourth layer content is enhanced based on the resolution of the second display screen, the corrected blur radius, and the buffer size of the second target layer. includes:

21. The method of claim 20, wherein, ​ ​ ​ ​ 22. An electronic device, comprising: ​ An electronic device comprising a display, one or more processors, and one or more memories; the one or more processors coupled with the display, and the one or more memories; the one or more memories configured to store computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-21.

23. A computer readable storage medium having stored thereon computer instructions, wherein, Computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of any of claims 1-21.

24. A computer program product comprising computer instructions, characterized in that, A computer program product that, when executed on a computer, causes the computer to perform the method of any of claims 1-21.

Images