Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an image processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: responding to an application program in the electronic equipment to enter a running state, and obtaining an application layer of the application program and a system layer of the electronic equipment; performing synthesis processing on the application layer and the system layer based on a first processor in the electronic equipment to obtain a first image; determining a target processing parameter of a second processor in the electronic equipment based on the state parameter of the electronic equipment; wherein different state parameters correspond to different target processing parameters; and performing super-resolution processing on the first image based on the target processing parameter to obtain a second image, and performing picture display based on the second image. According to the image processing method provided by the invention, the flexibility and the processing effect of the second processor for super-resolution processing can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of mobile product image quality optimization, and in particular, to an image processing method and device, an electronic device, a storage medium, and a program product. BACKGROUND

[0002] With the rapid development of technology, many electronic devices are equipped with display functions to meet the viewing needs of users. For electronic devices with display functions, the layers in the system need to go through image processing procedures such as graphics composition, rendering, and detail processing to form a display image, and then display the image on the display screen.

[0003] In related technologies, many application programs of electronic devices have poor user experience at low resolution, such as game application programs, short video application programs, and the like. Therefore, an electronic device usually has a super-resolution processing module to improve the picture resolution of these application programs. However, the processing parameters of the super-resolution processing module are pre-set, and in the actual processing process, there may be problems such as insufficient super-resolution processing, affecting the picture definition, or excessive super-resolution processing causing picture distortion. SUMMARY

[0004] To overcome the problems in related technologies, the present disclosure provides an image processing method, device, electronic device, storage medium, and program product. The image processing method provided by the present disclosure can improve the flexibility and processing effect of super-resolution processing by a second processor.

[0005] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, comprising:

[0006] In response to an application program in an electronic device entering a running state, an application layer of the application program and a system layer of the electronic device are obtained;

[0007] The application layer and the system layer are subjected to composition processing based on a first processor in the electronic device, to obtain a first image;

[0008] Based on a state parameter of the electronic device, a target processing parameter of a second processor in the electronic device is determined; wherein the target processing parameter corresponding to different state parameters is different;

[0009] The first image is subjected to super-resolution processing based on the target processing parameter, to obtain a second image, and a picture is displayed based on the second image.

[0010] In some embodiments, the composition processing of the application layer and the system layer based on the first processor in the electronic device to obtain the first image comprises:

[0011] adjust the application layer and the system layer to a first target resolution based on the resolution of the application layer, wherein the first target resolution is less than or equal to the resolution of the application layer;

[0012] perform a synthesis process on the adjusted application layer and the system layer based on the first processor to obtain the first image, wherein the first image has the first target resolution.

[0013] In some embodiments, the adjusting the application layer and the system layer to a first target resolution based on the resolution of the application layer comprises:

[0014] determining a preset range in which the resolution of the application layer is located, wherein each resolution located in the preset range is less than or equal to the resolution of the application layer;

[0015] determining a first target resolution from each resolution located in the preset range;

[0016] adjusting the application layer and the system layer to the first target resolution.

[0017] In some embodiments, the adjusting the application layer and the system layer to a first target resolution comprises:

[0018] adjusting a system resolution of the electronic device to the first target resolution based on the first processor, wherein the system resolution is related to a number of pixels processed by the first processor when performing image processing;

[0019] applying the system resolution to the application layer and the system layer.

[0020] In some embodiments, the determining a target processing parameter of a second processor in the electronic device based on a state parameter of the electronic device comprises:

[0021] adjusting a sub-parameter corresponding to a type of the state parameter in the target processing parameter based on the state parameter of the electronic device;

[0022] wherein the sub-parameters corresponding to different types of the state parameter are different.

[0023] In some embodiments, the state parameter comprises a temperature parameter and a current system resolution of the electronic device, and the sub-parameter comprises a processing range and a gain value, the processing range is used to determine each detail area to be enhanced in the first image by the second processor, and the gain value is used to determine an intensity of gain processing on each of the detail areas by the second processor.

[0024] The adjusting the sub-parameter in the target processing parameter corresponding to the type of the state parameter based on the state parameter of the electronic device comprises:

[0025] When the temperature parameter is greater than a preset temperature threshold, and the current system resolution is greater than a preset resolution threshold, determining a second target resolution based on the current system resolution; the second target resolution is less than the system resolution and the preset resolution threshold; the temperature parameter comprises a shell temperature of the electronic device and / or a chip temperature of the second processor;

[0026] In response to the current system resolution being adjusted to the second target resolution, determining the processing range and the gain value based on the second target resolution.

[0027] In some embodiments, the state parameter comprises a resolution of the application layer;

[0028] The determining the target processing parameter of the second processor in the electronic device based on the state parameter of the electronic device comprises:

[0029] When the temperature parameter is less than or equal to the preset temperature threshold, determining the target processing parameter based on the resolution of the application layer; or,

[0030] When the system resolution is less than or equal to the preset resolution threshold, determining the target processing parameter based on the resolution of the application layer.

[0031] In some embodiments, the state parameter comprises an application type of the application program and a scene type corresponding to the application layer; the sub-parameter comprises a jaggies parameter of an anti-aliasing filter in the second processor;

[0032] The determining the target processing parameter of the second processor in the electronic device based on the state parameter of the electronic device comprises:

[0033] Determining the jaggies parameter according to the application type and / or the scene type.

[0034] According to a second aspect of the embodiments of the present disclosure, an image processing apparatus is provided, comprising:

[0035] An obtaining module configured to, in response to an application program in an electronic device entering a running state, obtain an application layer of the application program and a system layer of the electronic device;

[0036] A synthesizing module configured to perform synthesizing processing on the application layer and the system layer based on a first processor in the electronic device to obtain a first image;

[0037] a determining module, configured to determine a target processing parameter of a second processor in the electronic device based on a state parameter of the electronic device; wherein the target processing parameter corresponding to different state parameters is different;

[0038] a super-resolution processing module, configured to perform super-resolution processing on the first image based on the target processing parameter to obtain a second image, and perform picture display based on the second image.

[0039] In some embodiments, the synthesizing module is further configured to adjust the application layer and the system layer to a first target resolution based on the resolution of the application layer; wherein the first target resolution is less than or equal to the resolution of the application layer; perform synthesizing processing on the adjusted application layer and the system layer based on the first processor to obtain the first image; wherein the first image has the first target resolution.

[0040] In some embodiments, the synthesizing module is further configured to determine a preset range in which the resolution of the application layer is located; wherein each resolution located in the preset range is less than or equal to the resolution of the application layer; determine a first target resolution from each resolution located in the preset range; and adjust the application layer and the system layer to the first target resolution.

[0041] In some embodiments, the synthesizing module is further configured to adjust a system resolution of the electronic device to the first target resolution based on the first processor; wherein the system resolution is related to the number of pixels processed when the first processor performs image processing; and apply the system resolution to the application layer and the system layer.

[0042] In some embodiments, the super-resolution processing module is further configured to adjust a sub-parameter corresponding to a type of the state parameter in the target processing parameter based on a state parameter of the electronic device; wherein the sub-parameter corresponding to different types of the state parameter is different.

[0043] In some embodiments, the state parameter comprises a temperature parameter of the electronic device and a current system resolution; the sub-parameter comprises a processing range and a gain value, the processing range being used to determine each detail area to be enhanced in the first image by the second processor, and the gain value being used to determine an intensity of gain processing of each detail area by the second processor; the super-resolution processing module is further configured to, when the temperature parameter is greater than a preset temperature threshold and the current system resolution is greater than a preset resolution threshold, determine a second target resolution based on the current system resolution; the second target resolution is less than the system resolution; the temperature parameter comprises a shell temperature of the electronic device and / or a chip temperature of the second processor; and in response to the current system resolution being adjusted to the second target resolution, the processing range and the gain value are determined based on the second target resolution.

[0044] In some embodiments, the state parameter comprises a resolution of the application layer; and the super-resolution processing module is further configured to, when the temperature parameter is less than or equal to the preset temperature threshold, determine the target processing parameter based on the resolution of the application layer, or when the system resolution is less than or equal to the preset resolution threshold, determine the target processing parameter based on the resolution of the application layer.

[0045] In some embodiments, the state parameter comprises an application type of the application program and a scene type corresponding to the application layer; and the sub-parameter comprises a jaggies parameter of an anti-aliasing filter in the second processor; and the super-resolution processing module is further configured to determine the jaggies parameter according to the application type and / or the scene type.

[0046] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, comprising a processor; a memory for storing computer programs or instructions; wherein the processor executes the computer programs or instructions to implement the steps of the method of the first aspect.

[0047] According to a fourth aspect of embodiments of the present disclosure, a non-transitory computer readable storage medium is provided, the storage medium storing computer programs or instructions, when the computer programs or instructions in the storage medium are executed by a processor, the steps of the method of the first aspect are implemented.

[0048] According to a fifth aspect of embodiments of the present disclosure, a computer program product is provided, comprising computer programs or instructions, when the computer programs or instructions are executed by a processor, the steps of the method of the first aspect are implemented.

[0049] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects:

[0050] In the image processing method provided by the embodiments of the present disclosure, the first image can be obtained by synthesizing the application layer and the system layer of the application program by using the first processor when the application program is in a running state; the target processing parameter of the second processor can be determined according to the state parameter of the electronic device; and the second image can be obtained by super-resolution processing the first image by using the determined target processing parameter. In this way, the target processing parameter can be determined according to the real-time state parameter of the electronic device, so that the determined target processing parameter is more scientific and reasonable. Compared with the processing parameter with a fixed value, the second processor can adapt to the current working state of the electronic device to perform appropriate and flexible super-resolution processing on the first image, which can improve the problem of poor picture definition caused by insufficient super-resolution processing and the problem of picture distortion caused by excessive super-resolution processing, and improve the picture display effect.

[0051] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0053] Figure 1 FIG. 1 is a flow diagram of an image processing method according to an example embodiment.

[0054] Figure 2 FIG. 2 is a schematic diagram of the principle of image synthesis, transmission and display in the related art.

[0055] Figure 3 FIG. 3 is a schematic diagram of the principle of image synthesis, transmission and display according to an example embodiment.

[0056] Figure 4 FIG. 4 is a schematic diagram of the image processing process based on the second processor according to an example embodiment.

[0057] Figure 5 FIG. 5 is a structural schematic diagram of an image processing device according to an example embodiment.

[0058] Figure 6 FIG. 6 is a structural block diagram of an electronic device according to an example embodiment. DETAILED DESCRIPTION

[0059] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following description of exemplary embodiments is not representative of all possible embodiments consistent with the present disclosure. Instead, it is merely an example of apparatus and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0060] Figure 1 is a flow chart of an image processing method according to an exemplary embodiment. As shown in Figure 1 , the image processing method is used in a terminal, and the method mainly includes the following steps:

[0061] Step 101, in response to an application in the electronic device entering a running state, obtaining an application layer of the application and a system layer of the electronic device;

[0062] Step 102, based on a first processor in the electronic device, synthesizing the application layer and the system layer to obtain a first image;

[0063] Step 103, based on a state parameter of the electronic device, determining a target processing parameter of a second processor in the electronic device; wherein the target processing parameter corresponding to different state parameters is different;

[0064] Step 104, based on the target processing parameter, performing super-resolution processing on the first image to obtain a second image, and performing picture display based on the second image.

[0065] The image processing method proposed in the embodiments of the present disclosure is applied to an electronic device with picture display function; here, the electronic device includes a fixed terminal, a mobile terminal or a portable electronic device; the fixed terminal includes but is not limited to a vehicle terminal, a television, etc.; the mobile terminal includes but is not limited to a mobile phone, a tablet computer, etc.; the portable electronic device includes but is not limited to a smart watch, etc., and the embodiments of the present disclosure do not make further limitation.

[0066] Among them, the first processor and the second processor in the electronic device respectively perform synthesis and super-resolution processing on the layers inside the system, and send the processed image to the display for picture display.

[0067] In step 101, in the case that the application in the above-mentioned electronic device enters a running state, it is determined that the user is using the application, at this time the content in the application needs to be presented on the display screen for the user to watch; therefore, in response to the application in the electronic device entering a running state, the first processor of the electronic device is used to obtain the application layer and the system layer.

[0068] It should be noted that the application program entering the running state indicates that the application program is being used by the foreground process. For the application program opened by the background process, since it does not need to be displayed, it is not necessary to obtain the content of the application program in the background.

[0069] Here, the application layer of the application program includes: obtaining at least one frame layer from the to-be-displayed image frame of the application program; it should be noted that the application layer can be obtained by the embodiment of the disclosure in response to the operation instruction of the user on the application program; for example, in response to the user opening the main page of the application layer, the main page layer of the application program is obtained. The operation instruction can be clicking the icon of the application program, or a voice instruction for the application program, etc.

[0070] The above-mentioned system layer of the electronic device includes: obtaining a layer associated with the above-mentioned application layer in the electronic device, or a layer that must be displayed; wherein the layer associated with the above-mentioned application layer can be a layer that needs to be displayed together with the application layer; for example, when the application layer is a video layer of a video playing software, the layer displayed by the auxiliary application layer includes a volume adjustment layer and a shortcut operation layer; the layer that must be displayed at least includes a status bar layer, and the status bar layer includes a top status bar layer and / or a bottom status bar layer; the content of the top status bar layer includes at least one of the following: time, power, network condition, attribute content (such as alarm condition, Bluetooth state, etc.); the content of the bottom status bar layer includes at least one of the following: a previous level return key and a home return key.

[0071] In step 102, in the case that the first processor obtains the application layer and the system layer, the application layer and the system layer are synthesized.

[0072] Here, the first processor is a core processor in the electronic device, which is arranged on the mainboard of the electronic device and is used for data processing and function control in the electronic device; for example, the first processor can perform logical calculation in the electronic device, and can also control the electronic device to perform audio external output, picture display, wireless signal transmission, etc. The first processor includes a system on chip (SoC), a central processing unit (CPU), or a micro processing unit (MPU), etc.

[0073] The first processor obtains the application layer and the system layer, and transmits the application layer and the system layer to a synthesis module in the first processor according to a preset transmission channel. The synthesis module performs layer superposition processing based on the position, row and column pixels of the application layer and the system layer, to obtain the first image.

[0074] It should be noted that in the embodiments of the present disclosure, the preset transmission channel can constrain the application layer and the system layer based on the system resolution, so that the application layer and the system layer transmitted to the synthesis module by the transmission channel are consistent with the system resolution.

[0075] In step 103, the embodiments of the present disclosure also acquire the state parameter by using the first processor, and transmit the state parameter to the second processor, so that the second processor determines the appropriate target processing parameter according to the state parameter, and performs subsequent image processing on the first image based on the determined target processing parameter.

[0076] It should be noted that the second processor and the first processor are arranged at different positions of the mainboard of the electronic device; wherein the second processor includes a display processing unit (DPU), and the DPU is externally arranged on the first processor and has a super resolution (SR) function, which can recover image details and other data information based on the information of the output image of the first processor, so as to improve the image quality. Since the original resolution of the image of part of the application program is low, the resolution of the first image is low, so the image quality of the display screen of these application programs can be improved by externally arranging the DPU, so as to provide better viewing experience for users.

[0077] Here, the first processor is provided with a display serial interface (DSI) and a serial peripheral interface (SPI), and the first image can be transmitted to the second processor such as DPU through the DSI; and the state parameter can be transmitted to the second processor through the SPI.

[0078] The state parameters of the electronic device include the state parameters of each functional module in the electronic device and the state parameters of each application program in the electronic device. Here, the state parameters of the functional module can be the temperature, working response time, memory occupation, etc. of the specified functional module, such as the temperature parameter of the first processor and / or the second processor in the embodiments of the present disclosure; the state parameters of each application program can be the type of the application program running in the foreground, or the power consumption parameter of the application program in the background, etc. Of course, the state parameter can also include other types of parameters, which will not be described and exemplified herein.

[0079] In the embodiments of the present disclosure, different types of state parameters can be used to determine different target processing parameters, or different types of state parameters are used to determine the same target processing parameter; wherein the target processing parameters are different when the state parameters are different. It can be understood that when the parameter values of the state parameters are different, the determined target processing parameters can be different; or when the types of the state parameters are different, different types of target processing parameters can be determined.

[0080] For example, the above-mentioned target processing parameters can include a denoising parameter of the second processor for denoising processing of the input image, or a model parameter of the image processing model, and of course can also include parameters related to detail processing, etc.

[0081] The denoising parameter is a parameter of a denoising filter in the second processor, which is used to reflect the strength of the filtering processing; the image processing model can be a convolutional neural network model or a generative adversarial network model, which is used to learn the rules of the first image, so as to generate a second image with higher clarity and more prominent details; the model parameter can be the weight or bias of the convolution kernel in the model, the type of the activation function, the weight of the fully connected layer, etc. Of course, the target processing parameter can also include other types of parameters, which will not be described and exemplified herein.

[0082] It should be noted that, because the state parameters of the electronic device include the state parameters of each functional module in the electronic device and the state parameters of each application program in the electronic device, according to the state parameters, the quality of the output image, the rate of processing the image, the unnecessary power consumption generated during the processing of the image by the first processor and the second processor, and other problems affecting the user experience and system load can be estimated, so that the present disclosure determines the appropriate target processing parameter through the state parameter, which can make the super-resolution processing of the first image balanced with the working state of the electronic device, and can output appropriate second images; and the process of processing the image can not only reduce the problem of insufficient image processing affecting the display of the picture, but also improve the problem of picture distortion caused by excessive processing of the image.

[0083] In some examples of the present disclosure, the state parameter of the functional module can be a temperature parameter of the first processor, because the temperature rise will affect the image processing of the processor, causing unnecessary noise to be introduced into the first image output by the first processor, so that the embodiments of the present disclosure can increase the parameter value of the denoising parameter.

[0084] In other examples of the present disclosure, the state parameters of each application program can include the state parameters of the foreground running application program and at least one background started application program; the state parameters of the application program include the type of the foreground application program, the picture content; the real-time power consumption of the background application program, the background running time, etc., which are not limited by the present disclosure.

[0085] At this time, the real-time power consumption of the background application program can cause the electronic device to rise in temperature, and therefore, without obtaining the temperature parameter, the real-time power consumption of the background application program can be used to determine the de-noising parameter. When the image processing model is set in the second processor, the application type and / or the picture content of the foreground application program (application program corresponding to the application layer of the synthesized first image) can be used to determine the model parameter; here, the application type and / or the picture content determine the need for image resolution display, for example, if the application program belongs to game software or short video playing software, higher display resolution is needed, if the application program belongs to reading software, higher display resolution is not needed; in addition, if the picture content is simulated 3D content, higher display resolution is needed, if the picture content is static content, higher resolution is not needed. In this way, the present disclosure can determine the model parameter and other target processing parameters of the second processor based on the application type and / or the picture type.

[0086] It should be noted that when at least one target processing parameter needs two or more state parameters to be determined, the present disclosure can pre-set the weight or priority of the state parameter, so as to determine the appropriate target processing parameter based on the state parameter.

[0087] In step 104, after determining the target processing parameter, the present disclosure uses the second processor to perform super-resolution processing on the first image to obtain a second image.

[0088] Here, the image quality of the second image is higher than that of the first image; wherein the higher image quality is manifested in higher definition, richer image details, smaller noise and / or smoother picture, etc.

[0089] For example, the present disclosure can perform de-noising processing on the first parameter according to the determined de-noising parameter to obtain a de-noised image; and then use the model parameter and the image processing model to perform further detail processing to obtain the second image.

[0090] In the image processing method provided in the embodiments of the present disclosure, the first processor can be used to synthesize the application layer and the system layer of the application program to obtain a first image when the application program enters a running state; the target processing parameter of the second processor is determined according to the state parameter of the electronic device; and the first image is super-resolution processed to obtain a second image by using the determined target processing parameter. In this way, the target processing parameter is determined according to the real-time state parameter of the electronic device, so that the determined target processing parameter is more scientific and reasonable. Compared with the fixed numerical processing parameter, the second processor can adapt to the current working state of the electronic device to perform appropriate and flexible super-resolution processing on the first image, which can improve the problem of poor picture definition caused by insufficient super-resolution processing and the problem of picture distortion caused by excessive super-resolution processing, and improve the picture display effect.

[0091] It should be noted that generally, the setting of the super-resolution processing module in the first processor will affect the response rate of the first processor (hereinafter, the CPU is taken as an example for illustration); therefore, in actual implementation, the second processor (hereinafter, the DPU is taken as an example for illustration) external to the CPU is used for image super-resolution processing. In the actual implementation process, the preset transmission channel arranged in the CPU can constrain the application layer and the system layer by using the system resolution.

[0092] Referring to Figure 2 , Figure 2 is a schematic diagram of the principle of image synthesis, transmission and display in the related art; as shown in Figure 2 , the super-target layer 211 (application layer requiring super-resolution processing) and the system status bar layer 212 are transmitted to the layer synthesis module 214 through the layer transmission channel 213 (Source Pipe) respectively; after the layer synthesis module 214 synthesizes the image, the CPU 21 transmits the synthesized image to the external DPU 22 through the display transmission interface 215. The synthesized image is transmitted to the display panel 23 after being super-resolution processed by the external DPU 22.

[0093] Here, in the case that the original resolution of the super-target layer 211 is low, the layer transmission channel 213 will perform image scale-up operation by referring to the higher system resolution, which increases the system power consumption; for example, Figure 2 , the original output image resolution is 1080P, but the system resolution is 1200P, at this time, the CPU will first enlarge it to 1200P, and then synthesize the 1200P super-target layer 211 and the system status bar layer 212 into a 1200P synthesized image; therefore, the CPU 21 transmits more pixels to the DPU 22 through the DSI interface 215, and the transmission efficiency will decrease obviously, and unnecessary power consumption is also generated.

[0094] To overcome the above problems, the embodiment of the present disclosure proposes an optimization method for image synthesis, transmission and display process in a related technology. In some embodiments, the above-mentioned synthesis processing of the application layer and the system layer based on the first processor in the electronic device obtains a first image, comprising:

[0095] Based on the resolution of the application layer, the application layer and the system layer are adjusted to a first target resolution; wherein the first target resolution is less than or equal to the resolution of the application layer;

[0096] Based on the first processor, the adjusted application layer and the system layer are synthesized to obtain a first image; wherein the first image has a first target resolution.

[0097] In the embodiment of the present disclosure, since the second processor DPU has the effect of increasing the definition and resolution, in order to increase the DSI transmission efficiency and reduce unnecessary power consumption in the transmission process and image processing process, the embodiment of the present disclosure can control the resolution of the first image synthesized by the first processor CPU to be lower.

[0098] Here, the layer resolution represents how many pixels are contained per unit length in the layer; the layer resolution is positively correlated with the definition of the layer, but negatively correlated with the size of the layer file.

[0099] Therefore, in the embodiment of the present disclosure, the first target resolution which is the same as or smaller than the resolution of the application layer can be determined and applied based on the resolution of the application layer, and the resolution of the system layer and the resolution of the application layer are constrained based on the first target resolution, so that the synthesis module of the CPU can process less pixel data when synthesizing, and the pixel transmitted to the DPU by the DSI is also less, which effectively improves the transmission efficiency and reduces unnecessary power consumption.

[0100] In the embodiment of the present disclosure, the way to determine the first target resolution can be to determine the first target resolution corresponding to the current resolution of the application layer in each preset resolution. Here, each preset resolution is a resolution having a preset relationship with the screen resolution of the display screen of the electronic device; for example, each preset resolution is obtained by reducing the screen resolution 1200P by the same ratio, or each preset resolution is obtained by scaling the screen resolution 1200P in different forms. For example, the embodiment of the present disclosure determines each preset resolution to be 1200P, 1080P, 864P and 720P. Among them, the way to determine the first target resolution in each preset resolution is to screen the preset resolution lower than the current resolution of the application layer, and determine one of the screened preset resolutions closest to the resolution of the application layer as the first target resolution.

[0101] The embodiment of the present disclosure can effectively increase the DSI transmission efficiency and reduce unnecessary power consumption in the transmission process and the image processing process by constraining the system layer and the application layer to the first target resolution and synthesizing the first image with the first target resolution based on the first processor.

[0102] In some embodiments, adjusting the application layer and the system layer to the first target resolution based on the resolution of the application layer comprises:

[0103] determining a preset range in which the resolution of the application layer is located; wherein each resolution in the preset range is less than or equal to the resolution of the application layer;

[0104] determining the first target resolution from each resolution in the preset range;

[0105] adjusting the application layer and the system layer to the first target resolution.

[0106] Here, the process of determining the first target resolution by the embodiment of the present disclosure can also be determined according to the preset range; for example, the embodiment of the present disclosure can determine a preset range in which the resolution of the application layer is located; wherein the determined preset range is a range determined according to a fixed resolution length with the resolution of the application layer as the maximum value; therefore, each resolution in the preset range is less than or equal to the resolution of the application layer and is an integer, and each resolution can be continuously distributed in the preset range or discretely distributed.

[0107] For example, when the resolution of the application layer is determined to be 1080P, the range in which the resolution of the application layer is located can be determined to be 1080P-1000P. When the resolution of the application layer is determined to be 864P, the range in which the resolution of the application layer is located can be determined to be 864P-764P.

[0108] In the embodiment of the present disclosure, in order to unify the processing rules and effectively improve the processing efficiency, in some examples of the present disclosure, the lowest resolution in each preset range can be used as a candidate resolution; in another example of the present disclosure, the average of each resolution in the preset range that is less than the intermediate resolution can be calculated, and the average can be used as the candidate resolution; after determining the preset range in which the resolution of the application layer is located, the candidate resolution corresponding to the preset range can be used as the first target resolution; wherein the resolution of the application layer can be equal to the candidate resolution of the preset range, or can be less than the resolution of the preset range. Subsequently, the application layer and the system layer are adjusted to the first target resolution.

[0109] Here, the embodiment of the present disclosure can adjust the layer resolution to be consistent before the first processor performs the synthesis processing, so as to reduce the system waste as much as possible and improve the transmission efficiency of the first image after synthesis.

[0110] In some embodiments, the above-mentioned adjusting the application layer and the system layer to the first target resolution comprises:

[0111] Based on the first processor, the system resolution of the electronic device is adjusted to the first target resolution; wherein the system resolution is related to the number of pixels processed by the first processor when processing the image;

[0112] The system resolution is applied to the application layer and the system layer.

[0113] Here, because the above-mentioned preset transmission channel can apply the system resolution to the application layer and the system layer, specifically, the application layer and the system layer transmitted are constrained by the system resolution by scaling according to the size. Therefore, the embodiment of the present disclosure can first adjust the system resolution of the electronic device to the first target resolution, and then scale the input layer by using the preset transmission channel to obtain the system layer and the application layer with the first target resolution output to the synthesis module.

[0114] Referring to Figure 3 , Figure 3 is a schematic diagram of image synthesis, transmission and display according to an exemplary embodiment. Wherein the application layer 311 and the system layer 312 are transmitted to the synthesis module 314 through the preset transmission channel 313 (Source Pipe); after the synthesis module 314 synthesizes the first image, the first processor 31 transmits to the external second processor 32 by using the DSI interface 315. Here, the first processor 31 also transmits the above-mentioned obtained state parameters to the second processor 32 by using the SPI interface (not shown). Figure 3

[0115] ​For example, the synthetic display process SurfaceFlinger first detects the foreground running application and the system plug-in. When the application layer of the application and the system layer of the system plug-in are detected, the resolution of the application layer of the original output of the application is 1080P, and the corresponding first target resolution is determined. For example, the resolution of the application layer is 1080P, and the first target resolution is determined as 1080P. The embodiment of the present disclosure first adjusts the system resolution from 1200P to 1080P by using the function function in SurfaceFlinger, and then reduces the system layer from 1200P to 1080P in the preset transmission channel 313. After that, the first processor 31 synthesizes the first image of 1080P by using the synthesis module 314, and transmits the first image of 1080P to the second processor 32 for subsequent image processing by using the DSI interface 315.

[0116] Therefore, the embodiment of the present disclosure can reduce the pixels transmitted to the DPU, so that the transmission efficiency is obviously improved, and unnecessary power consumption is effectively reduced.

[0117] In some embodiments, the above determining the target processing parameter of the second processor in the electronic device based on the state parameter of the electronic device comprises:

[0118] Based on the state parameter of the electronic device, the sub-parameter corresponding to the type of the state parameter in the target processing parameter is adjusted; wherein the sub-parameters corresponding to different types of state parameters are different.

[0119] In the embodiment of the present disclosure, the state parameters include different types of parameters of the state parameters of each functional module in the electronic device and the state parameters of each application in the electronic device, wherein the state parameters of the functional module and the state parameters of the application can have different types.

[0120] For example, each state parameter of the functional module has a type representing the power consumption state of the functional module itself and a type representing the use condition of the functional module itself; the type representing the power consumption state of the functional module includes the temperature parameter and the power consumption parameter of the functional module, for example, the temperature parameter includes the temperature of the processor itself and the temperature of the battery; the type representing the use condition of the functional module itself includes the power of the battery module.

[0121] The state parameters of the application program have a type representing an attribute of the application program and a type representing different application functions in the application program, wherein the type representing the attribute of the application program includes an application program type, and the application program type includes a game software type, a short video software type, a reading software type, an album software type, and the like; different application program types have different resolution requirements for final picture display; the type representing different application functions in the application program includes a networked battle function in the game software, a man-machine battle function, and a dialogue function type; and further includes a live broadcast function, a comment function, and a local cache viewing function of the short video software, and the like. Different types of different functions of different application programs have different resolution requirements for final picture display.

[0122] Here, the sub-parameters of the target processing parameters corresponding to different types of state parameters are different, that is, different types of state parameters obtained by the present disclosure can be used to determine a sub-parameter of a target processing parameter; wherein each sub-parameter can be determined by only one type of state parameter, and can also be determined by different types of state parameters together.

[0123] In the embodiments of the present disclosure, a parameter pre-adjustment model is arranged in the second processor DPU, which can determine a suitable sub-parameter of at least one target processing parameter based on the state parameter, and transmit the determined sub-parameter to an image processing function module applying the sub-parameter for super-resolution processing.

[0124] The embodiments of the present disclosure can determine a sub-parameter of at least one target processing parameter in the target processing parameter by using different types of state parameters of the terminal device, which helps to more reasonably and specifically control subsequent super-resolution processing, reduces the problems of insufficient image detail processing or excessive image processing, and improves the quality of the second image obtained based on the final result.

[0125] In some embodiments, the state parameters include a temperature parameter of the electronic device and a current system resolution; the sub-parameters include a processing range and a gain value, the processing range is used to determine each detail area to be strengthened in the first image determined by the second processor, and the gain value is used to determine the intensity of gain processing of each detail area by the second processor;

[0126] The above adjustment of the sub-parameter of the target processing parameter corresponding to the type of the state parameter based on the state parameter of the electronic device includes:

[0127] When the temperature parameter is greater than a preset temperature threshold, and the current system resolution is greater than a preset resolution threshold, a second target resolution is determined based on the current system resolution; wherein the second target resolution is less than the system resolution and the preset resolution threshold; the temperature parameter includes a shell temperature of the electronic device and / or a chip temperature of the second processor;

[0128] In response to the current system resolution being adjusted to the second target resolution, a processing range and a gain value are determined based on the second target resolution.

[0129] It should be noted that the shell temperature of the electronic device is the temperature of the battery module in the electronic device, which can be measured by a first temperature sensor arranged in the electronic device for the battery module; the chip temperature of the second processor can be measured by a second temperature sensor arranged in the electronic device for the second processor.

[0130] Here, for judging whether the temperature parameter is greater than the preset temperature threshold, in some examples, a first temperature threshold can be preset in the electronic device; after obtaining the shell temperature and the chip temperature, weighted average processing is performed to obtain an average temperature, and when the average temperature is greater than the first temperature threshold, it is determined that the temperature parameter is greater than the temperature threshold. In other examples, a second temperature threshold and a third temperature threshold can be determined in the electronic device; when the shell temperature is greater than the second temperature threshold or the chip temperature is greater than the third temperature threshold, it is determined that the temperature parameter is greater than the temperature threshold.

[0131] In the embodiments of the present disclosure, the preset resolution threshold represents the minimum resolution that the system resolution can be adjusted to; if the system resolution is adjusted to be less than the threshold, a large amount of image content will be lost, affecting the quality of the subsequent second image.

[0132] In the embodiments of the present disclosure, the preset temperature threshold is used to represent that the temperature rise of the electronic device affects the running progress of the entire electronic device, resulting in the work of each functional module of the electronic device being affected; in the case where the parameter pre-adjustment model of the DPU determines that the temperature parameter is greater than the preset temperature threshold and the system resolution is greater than the preset resolution threshold, a second target resolution smaller than the current system resolution is determined, and then the parameter pre-adjustment model generates a resolution adjustment instruction, the resolution adjustment instruction having a second target resolution value, and the DPU transmits the resolution adjustment instruction to the SPI interface of the CPU through the built-in SPI interface; after the CPU receives the resolution adjustment instruction, the system resolution is adjusted to the second target resolution; in this way, by suppressing the system resolution, the system loss of the CPU performing image processing and synthesis according to the system resolution is reduced, which helps to alleviate the adverse effects of the temperature rise.

[0133] It should be noted that after the system resolution is suppressed, the resolution of the output layer will be smaller due to the system suppression of the system plug-in and the foreground application, resulting in insufficient overall image clarity; therefore, after the CPU adjusts the system resolution, a corresponding adjustment completion instruction is transmitted through the DSI interface, and the DPU responds to the adjustment completion instruction to determine that the current system resolution has been adjusted to the second target resolution, at which time the processing range and the gain value in the second processor can be re-determined based on the second target resolution, so that the processing range and the gain value more suitable for the current situation are used to enhance the processing of the detail area.

[0134] The parameter pre-adjustment model inputs the re-determined processing range and gain value into the post-super-resolution processing module; the post-super-resolution processing module performs edge detection on the first image according to the processing range to determine at least one detail area corresponding to the processing range; and performs gain enhancement processing on each detail area based on the gain value to obtain a detail-processed image, and obtains the second image based on the detail-processed image.

[0135] Here, the first image processed by the post-super-resolution processing module can be an image obtained after the image transmitted by the DSI is processed by image processing steps such as denoising and contrast adjustment, or can be an image directly transmitted by the DSI, or can be an image obtained after being processed by other image processing steps.

[0136] It can be understood that generally after the system resolution is suppressed, the resolution of the image transmitted by the DSI to the second processor DPU is smaller, and the image details are not perfect, so the processing range determined by the embodiments of the present disclosure needs to be larger and the gain value needs to be stronger; the embodiments of the present disclosure can pre-set a first matching relationship between the second target resolution and the processing range, and a second matching relationship between the second target resolution and the gain value, and the first matching relationship and the second matching relationship are in a positive correlation state; in this way, after the system resolution is suppressed, the processing range and the gain value corresponding to the processed system resolution can be determined. Of course, in other examples of the present disclosure, the determination of the processing range and the gain value according to the processed system resolution can be implemented by a network model provided in the electronic device.

[0137] In the embodiments of the present disclosure, suppressing the system resolution based on the temperature parameter of the electronic device can effectively reduce the system power consumption and improve the working efficiency of the electronic device, and determining the target processing parameter according to the adjusted system resolution can make the super-resolution processing of the image adapt to the system resolution state in time and improve the flexibility of image processing and picture display.

[0138] In some embodiments, the state parameter includes a resolution of an application layer;

[0139] The target processing parameter of the second processor in the electronic device is determined based on a state parameter of the electronic device, including:

[0140] When the temperature parameter is less than or equal to a preset temperature threshold, the target processing parameter is determined based on a resolution of the application layer; or,

[0141] When the system resolution is less than or equal to a preset resolution threshold, the target processing parameter is determined based on the resolution of the application layer.

[0142] In combination with the foregoing of the present disclosure, when the temperature parameter is less than or equal to a preset temperature parameter threshold, it indicates that the running progress of the current electronic device is normal, and the redundant power consumption loss is controllable; when the current system resolution is less than or equal to a preset resolution threshold, it indicates that further reducing power consumption by continuing to suppress the system resolution will affect the normal functional use of the electronic device; therefore, when any one of the above two conditions is met, or both conditions are met, the parameter pre-adjustment model of the DPU determines the processing range and the gain value of the post-super-resolution processing module according to the resolution of the application layer obtained.

[0143] In the embodiments of the present disclosure, the resolution of the application layer can be the current resolution of the application layer, or the initial resolution of the application layer before inputting to the CPU through a preset transmission channel; the present disclosure does not limit this. When the current resolution of the application layer is used to determine the target processing parameter, the processing range and the gain value determined by the parameter pre-adjustment model at this time can be effectively corresponding to the current state of the image. When the initial resolution of the application layer is used, the processing parameter and the gain value determined by the parameter pre-adjustment model can correspond to the initial resolution requirement of the application program, and meet the actual requirement of the user.

[0144] Here, the present disclosure can set a third matching relationship between the resolution and the processing range, and a fourth matching relationship between the resolution and the gain value; based on the third matching relationship and the fourth matching relationship, the corresponding processing range and the gain value can be determined when the resolution of the application layer is obtained; of course, in other examples of the present disclosure, the determination of the processing range and the gain value according to the resolution of the application layer can be implemented by a network model provided in the electronic device.

[0145] It should be noted that, because the embodiments of the present disclosure can determine the processing range and the gain value according to the resolution of the application layer, in the case where the resolution of the application layer represents that the current definition of the application layer has reached the display state of the screen resolution, or represents that the definition of the application layer has met the requirements of the user, the third matching relationship and the fourth matching relationship can be used to determine a smaller processing range and a lower gain value, so as to reduce the intensity of the detail processing of the first image, reduce the phenomenon of picture distortion caused by excessive super-resolution processing, and improve the objectivity of the final picture display. Of course, in the case where the resolution of the application layer represents that the current definition of the application layer is low, the third matching relationship and the fourth matching relationship can be used to determine a larger processing range and a higher gain value, so as to enhance the detail processing effect and increase the definition of the second image finally sent for display.

[0146] In this way, the embodiments of the present disclosure can determine the processing range and the gain value according to the resolution of the application layer, so as to make the super-resolution processing more flexible, thereby reducing picture distortion or insufficient definition and other problems, and improving the display effect of the second image.

[0147] In some embodiments, the state parameter includes an application type of the application program and a scene type corresponding to the application layer, and the sub-parameter includes a jaggies parameter of an anti-aliasing filter in the second processor.

[0148] The above determining, based on the state parameter of the electronic device, the target processing parameter of the second processor in the electronic device includes:

[0149] According to the application type and / or the scene type, the jaggies parameter is determined. Here, in combination with the foregoing of the present disclosure, the state parameter includes a state parameter of the application program, and the state parameter of the application program has a type representing an attribute of the application program and a type representing different application functions in the application program. In the embodiments of the present disclosure, the application type of the application program belongs to the type representing the attribute of the application program, and the scene type corresponding to the application layer belongs to the type representing the different application functions.

[0150] For example, when the application type is a game, the scene type can be a networked battle scene, a man-machine battle scene, or a dialogue scene; when the application type is a short video, the scene type includes a live broadcast scene, a comment scene, and a local cache viewing scene.

[0151] Here, because the application type and the scene type have different requirements for the picture display of the application program, the embodiments of the present disclosure can flexibly determine the jaggies parameter of the anti-aliasing filter in the DPU of the second processor according to the application type and the scene type.

[0152] It should be noted that because there are more dynamic pictures in the game online battle scene, the output application layer has more jaggies, and therefore a larger range and greater degree of jaggie smoothing needs to be performed using an anti-jaggie filter. Therefore, in order to apply the picture display in this scenario, the embodiments of the present disclosure can set a fifth matching relationship between the application type and / or the scene type and the jaggie parameter. In this way, after the DPU obtains the application type and / or the scene type transmitted by the CPU, the parameter pre-adjustment model finds the corresponding jaggie parameter through the fifth matching relationship.

[0153] Here, the jaggie parameter can be the weight of the convolution kernel in the anti-jaggie filter, the sampling rate, the order of the filter, etc. The anti-jaggie filter filters, convolves, samples, and repeats the first image input to the anti-jaggie filter based on the jaggie parameter determined by the parameter pre-adjustment model, to obtain the effect of image jaggie smoothing processing, reduce the degree of jaggie on the edge of the first image, and improve the smoothness of the first image. The convolution operation is to convolve the first image converted to the frequency domain using the adjusted weight of the convolution kernel, blur the sharp edges, and reduce the high-frequency signal. The subsequent sampling operation can determine the repeated components of the signal spectrum, process the repeated spectral components according to the adjusted sampling rate and filter order, thereby reducing the jaggie effect.

[0154] In addition, if the current application type and scene type indicate that there can not be too many jaggie effects in the first image, at this time, according to the fifth matching relationship described above, the appropriate anti-jaggie strength can be determined, so that the anti-jaggie smoothing processing is more flexible, and the problem of image information loss caused by excessive image processing is reduced.

[0155] The following exemplary describes the electronic device to which the image processing method proposed by the embodiments of the present disclosure is applied as a mobile phone. The mobile phone is provided with many application programs that have super-resolution display needs, such as game application programs, short video application programs, etc. The image processing method proposed by the embodiments of the present disclosure aims to improve the resolution of the layers of these application programs and improve the picture display effect.

[0156] First, when the related art uses an external DPU for super-resolution processing, there is a case where the resolution of the output layer of the application program itself is low and does not reach the system resolution. At this time, the flow as shown in Figure 2 is used to adjust the other system status bar layer and the super-application layer to a higher system resolution, and during the synthesis and transmission through the DSI interface, the image transmission is performed at a higher resolution, which not only cannot effectively improve the picture quality input to the DPU, but also causes a loss in power consumption.

[0157] In addition, the processing parameters of the external DPU in the related art are fixed, so that when the resolution of the layer output by the application itself is high and has sufficient definition, the improvement of the picture definition is limited, and even the use of fixed super-resolution parameters can easily cause the picture to be too sharp, resulting in the problem of over-hard distortion. At the same time, if the power consumption condition of the mobile phone is poor at this time, the DPU function is continuously enabled in the case that the super-resolution processing benefit is not large, and the endurance time will be greatly shortened.

[0158] To solve the above problems, the present disclosure provides an image processing method for improving image display effect and system power consumption; see Figure 3 and Figure 4 ; here, Figure 4 is a schematic diagram of an image processing process based on a second processor according to an exemplary embodiment.

[0159] In Figure 3 , the first processor 31 activates the synthesis display process SurfaceFlinger, and uses SurfaceFlinger to detect the foreground running application and system plug-in first. When the application layer of the application and the system layer of the system plug-in are detected, the original output application layer 311 of the application and the system layer 312 output by the system plug-in are obtained; here, the system layer 312 includes the status bar layer. SurfaceFlinger obtains the resolution of the application layer, and determines the first target resolution according to the resolution of the application layer; here, a plurality of preset resolutions such as 1200P, 1080P, 864P, etc. can be set as different resolution gears; at this time, if the system resolution is consistent with the first target resolution corresponding to the resolution of the application layer, the system resolution is not adjusted. If the system resolution is not consistent with the first target resolution corresponding to the resolution of the application layer, the system resolution is adjusted to the first target resolution corresponding to the resolution of the application layer. After adjusting the system resolution, the application layer 311 and the system layer 312 are transmitted by using the preset transmission channel 313, and in the transmission process, the resolutions of the application layer 311 and the system layer 312 are adjusted to be consistent with the adjusted system resolution. When the application layer 311 and the system layer 312 are transmitted to the synthesis module 314 through the preset transmission channel 313; the synthesis module 314 synthesizes the first image with the first target resolution, and at this time the first processor 31 transmits the first image to the external second processor 32 by using the DSI interface 315. Here, the first processor 31 also transmits the above-mentioned state parameters to the second processor 32 by using the SPI interface.

[0160] In Figure 4In the specific implementation, after the parameter pre-adjustment model 321 of the second processor 32 obtains the state parameters, the parameter pre-adjustment model 321 determines whether the temperature parameters are greater than a preset temperature threshold value based on the shell temperature and the chip temperature in the state parameters. If the temperature parameters are greater than the preset temperature threshold value, the parameter pre-adjustment model 321 further determines whether the current system resolution in the state parameters is greater than a preset resolution threshold value. If the current system resolution is greater than the preset resolution threshold value, the parameter pre-adjustment model 321 determines a second target resolution and generates a resolution adjustment instruction, and the SPI interface 316 transmits the resolution adjustment instruction. The first processor 31 adjusts the current system resolution to the second target resolution based on the resolution adjustment instruction. Subsequently, the parameter pre-adjustment model 321 increases the processing range and the gain value based on the second target resolution.

[0161] Here, if the temperature parameters are less than or equal to the preset temperature threshold value, the parameter pre-adjustment model 321 determines appropriate processing range and gain value based on the resolution of the application layer in the state parameters, and a third matching relationship between the resolution and the processing range and a fourth matching relationship between the resolution and the gain value. Here, if the resolution of the application layer represents that the current definition of the application layer has reached the display state of the screen resolution, the present disclosure can determine smaller processing range and lower gain value based on the third matching relationship and the fourth matching relationship, so as to reduce the intensity of the detail processing of the first image, reduce the phenomenon of picture distortion caused by excessive super-resolution processing, and improve the objectivity of the final picture display.

[0162] In the embodiment of the present disclosure, the parameter pre-adjustment model 321 further captures the picture style of the to-be-displayed layer of the application program by using the application type and / or the scene type in the obtained state parameters, so as to determine the aliasing parameter of the anti-aliasing filter.

[0163] Here, in combination with Figure 4 , the parameter pre-adjustment model 321 can transmit the aliasing parameter determined based on the state parameters to the anti-aliasing filter 322, and transmit the processing range and the gain value to the post-super-resolution processing module 323. Here, if the re-determined aliasing parameter is the same as the original aliasing parameter, no adjustment is made, and if there is a difference, the aliasing parameter can be updated. At this time, the embodiment of the present disclosure can update the aliasing parameter according to a specified step size until the re-determined aliasing parameter is updated, so as to reduce the picture flicker that can be perceived by the user due to the scene change and the rapid and drastic change of the parameter.

[0164] In the embodiment of the present disclosure, the anti-aliasing filter 322 can perform aliasing smoothing processing on the input first image based on the re-determined aliasing parameter, to obtain a third image. The post-super-resolution processing module 323 performs detail enhancement processing on the third image based on the re-determined processing range and the gain value, to obtain the second image. It should be noted that, before being input to the anti-aliasing filter 322, the first image can also be subjected to other pre-processing such as noise reduction, filtering, contrast enhancement processing, and the like, which is not limited in the embodiment of the present disclosure.

[0165] Here, in the case of obtaining the second image after super-resolution processing based on the first image, the resolution of the second image can be 1200P consistent with the screen resolution; in combination with the above-mentioned Figure 4 , the 1200P image can be transmitted to the display screen 33 for picture display. Figure 3

[0166] Here, before the external DPU performs super-resolution processing, the shell temperature and the chip temperature, the current system resolution, the resolution of the application layer, the application type, the scene type and other state parameters are transmitted to the external DPU through the SPI interface, and the preset parameter pre-adjustment model in the DPU adjusts the processing range of the detail area setting, the gain value of the detail gain intensity, the sawtooth parameter of the sawtooth feature setting, and so on according to the above input parameters. In this way, the target processing parameters can be weakened when the application layer has high clarity, the picture distortion can be avoided, the sawtooth parameter can be updated according to the picture scene, the picture sawtooth feeling can be effectively reduced, and when the temperature of the mobile phone rises sharply, the system resolution can be suppressed and the detail processing can be strengthened, so that the power consumption and temperature rise are reduced under the condition of less picture quality sacrifice, and the battery life of the mobile phone is prolonged.

[0167] Figure 5 is a structural schematic diagram of an image processing device according to an example embodiment. As shown in Figure 5 , the image processing device 500 mainly includes:

[0168] The acquisition module 501 is configured to acquire an application layer of an application program and a system layer of an electronic device in response to the application program entering a running state.

[0169] The synthesis module 502 is configured to perform synthesis processing on the application layer and the system layer based on a first processor in the electronic device to obtain a first image.

[0170] The determination module 503 is configured to determine target processing parameters of a second processor in the electronic device based on state parameters of the electronic device; wherein the target processing parameters corresponding to different state parameters are different.

[0171] The processing module 504 is configured to perform super-resolution processing on the first image based on the target processing parameters to obtain a second image, and perform picture display based on the second image.

[0172] In some embodiments, the synthesis module 502 is further configured to adjust the application layer and the system layer to a first target resolution based on the resolution of the application layer; wherein the first target resolution is less than or equal to the resolution of the application layer; perform synthesis processing on the adjusted application layer and system layer based on the first processor to obtain the first image; wherein the first image has the first target resolution.​

[0173] In some embodiments, the synthesizing module 502 is further configured to determine a preset range in which resolutions of the application layer are located; each resolution in the preset range is less than or equal to the resolution of the application layer; determine a first target resolution from the resolutions in the preset range; and adjust the application layer and the system layer to the first target resolution.

[0174] In some embodiments, the synthesizing module 502 is further configured to adjust, based on the first processor, a system resolution of the electronic device to a first target resolution; the system resolution is related to a number of pixels processed by the first processor when performing image processing; and the system resolution is applied to the application layer and the system layer.

[0175] In some embodiments, the super-resolution processing module 504 is further configured to adjust, based on a state parameter of the electronic device, a sub-parameter in the target processing parameter corresponding to a type of the state parameter; different types of state parameters correspond to different sub-parameters.

[0176] In some embodiments, the state parameter includes a temperature parameter of the electronic device and a current system resolution; the sub-parameter includes a processing range for determining each detail area to be enhanced in the first image by the second processor and a gain value for determining an intensity of gain processing of each detail area by the second processor; and the super-resolution processing module 504 is further configured to determine a second target resolution based on the current system resolution when the temperature parameter is greater than a preset temperature threshold and the current system resolution is greater than a preset resolution threshold; the second target resolution is less than the system resolution and the preset resolution threshold; the temperature parameter includes a shell temperature of the electronic device and / or a chip temperature of the second processor; and the processing range and the gain value are determined based on the second target resolution in response to the current system resolution being adjusted to the second target resolution.

[0177] In some embodiments, the state parameter includes a resolution of the application layer; and the super-resolution processing module 504 is further configured to determine the target processing parameter based on the resolution of the application layer when the temperature parameter is less than or equal to the preset temperature threshold, or based on the resolution of the application layer when the system resolution is less than or equal to the preset resolution threshold.

[0178] In some embodiments, the state parameter includes an application type of the application program and a scene type corresponding to the application layer; and the sub-parameter includes a jaggies parameter of an anti-aliasing filter in the second processor; and the super-resolution processing module 504 is further configured to determine the jaggies parameter according to the application type and / or the scene type.

[0179] With regard to the apparatuses in the above-described embodiments, a specific manner in which each module performs an operation has been described in detail in the embodiments related to the method, and thus detailed description will not be made here.

[0180] Figure 6 is a structural block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 600 can be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0181] Referring to Figure 6 , the electronic device 600 can include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0182] The processing component 602 generally controls the overall operation of the electronic device 600 such as the operation associated with at least one of display, telephone call, data communication, camera operation, and recording operation. The processing component 602 can include one or more processors 620 to execute instructions to complete all or a part of steps of the above-described methods. In addition, the processing component 602 can include one or more modules to facilitate interaction between the processing component 602 and other components. For example, the processing component 602 can include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0183] The memory 604 is configured to store various types of data to support operations of the electronic device 600. Examples of these data include at least one of instructions for at least one application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, and videos. The memory 604 can be implemented by any type of volatile or nonvolatile memory devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disc or a compact disc.

[0184] The power component 606 provides power to the various components of the electronic device 600. The power component 606 can include at least one of a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 600.

[0185] The multimedia component 608 includes a screen providing an output interface between the electronic device 600 and a user. In some embodiments, the screen can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensor can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0186] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0187] The I / O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which can be a keyboard, a click wheel, and buttons, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0188] The sensor component 614 includes one or more sensors for providing various state assessments for the electronic device 600. For example, the sensor component 614 can detect an open / closed position of the electronic device 600, relative positioning of components, such as a display and a keypad of the electronic device 600, a change in position of the electronic device 600 or a component of the electronic device 600, presence or absence of user contact with the electronic device 600, orientation or acceleration / deceleration of the electronic device 600, and temperature changes of the electronic device 600. The sensor component 614 can include an optical sensor for use in imaging applications. The sensor component 614 can also include a near sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 614 can further include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 614 can further include at least one of an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, and a temperature sensor, among others.

[0189] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an example embodiment, the communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 616 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0190] In exemplary embodiments, the electronic device 600 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements.

[0191] In exemplary embodiments, a non-transitory computer-readable storage medium including instructions, such as a memory 604 including executable instructions or a computer program, is also provided, which can be executed by the processor 620 of the electronic device 600 to complete the above-described method. For example, the non-transitory computer-readable storage medium can be a ROM, a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0192] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform any of the above-described image processing methods of the embodiments of the present disclosure. For example, the method includes: in response to an application in an electronic device entering a running state, obtaining an application layer of the application and a system layer of the electronic device; performing synthesis processing on the application layer and the system layer based on a first processor in the electronic device to obtain a first image; determining a target processing parameter of a second processor in the electronic device based on a state parameter of the electronic device; wherein the target processing parameter corresponding to different state parameters is different; performing super-resolution processing on the first image based on the target processing parameter to obtain a second image, and performing picture display based on the second image.

[0193] The embodiments of the present disclosure provide a computer program product, which includes: a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium, and the processor executes the computer program or executable instructions, so that the computer device performs any of the above-described image processing methods of the embodiments of the present disclosure.

[0194] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the disclosure be construed as including any paterns of this disclosure that can be derived from the description and illustrations presented herein without departing from the scope and spirit of the disclosure. The specification and examples are exemplary only, with the true scope and spirit of the disclosure being indicated by the claims.

[0195] It is to be understood that the disclosure is not limited to the precise construction hereinafter described and as shown in the attached drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the disclosure is limited only by the claims appended hereto.

Claims

1. An image processing method, characterized by, The application comprises: in response to an application in an electronic device entering a running state, obtaining an application layer of the application and a system layer of the electronic device; based on a first processor in the electronic device, performing synthesis processing on the application layer and the system layer to obtain a first image; based on a state parameter of the electronic device, determining a target processing parameter of a second processor in the electronic device; wherein the target processing parameter corresponding to different state parameters is different; based on the target processing parameter, performing super-resolution processing on the first image to obtain a second image, and performing picture display based on the second image.

2. The method of claim 1, wherein, The synthesis processing on the application layer and the system layer based on the first processor in the electronic device to obtain the first image comprises: based on the resolution of the application layer, adjusting the application layer and the system layer to a first target resolution; wherein the first target resolution is less than or equal to the resolution of the application layer; based on the first processor, performing synthesis processing on the adjusted application layer and the system layer to obtain the first image; wherein the first image has the first target resolution.

3. The method of claim 2, wherein, The adjustment of the application layer and the system layer to the first target resolution based on the resolution of the application layer comprises: determining a preset range in which the resolution of the application layer is located; wherein each resolution located in the preset range is less than or equal to the resolution of the application layer; determining a first target resolution from each resolution located in the preset range; adjusting the application layer and the system layer to the first target resolution.

4. The method of claim 2, wherein, The adjustment of the application layer and the system layer to the first target resolution comprises: based on the first processor, adjusting the system resolution of the electronic device to the first target resolution; wherein the system resolution is related to the number of pixels processed by the first processor when performing image processing; applying the system resolution to the application layer and the system layer.

5. The method according to any one of claims 1 to 4, characterized in that, The determination of the target processing parameter of the second processor in the electronic device based on the state parameter of the electronic device comprises: based on the state parameter of the electronic device, adjusting a sub-parameter corresponding to the type of the state parameter in the target processing parameter; wherein the sub-parameter corresponding to different types of the state parameter is different.

6. The method of claim 5, wherein, The state parameter comprises a temperature parameter and a current system resolution of the electronic device; the sub-parameter comprises: a processing range for determining each detail area to be enhanced in the first image by the second processor, and a gain value for determining the intensity of gain processing of each detail area by the second processor; The adjustment of the sub-parameter corresponding to the type of the state parameter in the target processing parameter based on the state parameter of the electronic device comprises: when the temperature parameter is greater than a preset temperature threshold and the current system resolution is greater than a preset resolution threshold, determining a second target resolution based on the current system resolution; wherein the second target resolution is less than the system resolution and the preset resolution threshold; the temperature parameter comprises a shell temperature of the electronic device and / or a chip temperature of the second processor; in response to the current system resolution being adjusted to the second target resolution, determining the processing range and the gain value based on the second target resolution.

7. The method of claim 6, wherein, the state parameter comprises a resolution of the application layer; determining a target processing parameter of a second processor in the electronic device based on a state parameter of the electronic device, comprising: when the temperature parameter is less than or equal to the preset temperature threshold, determining the target processing parameter based on the resolution of the application layer; or, when the system resolution is less than or equal to the preset resolution threshold, determining the target processing parameter based on the resolution of the application layer.

8. The method of claim 5, wherein, the state parameter comprises: an application type of the application program and a scene type corresponding to the application layer; the sub parameter comprises a jaggies parameter of an anti aliasing filter in the second processor; determining a target processing parameter of a second processor in the electronic device based on a state parameter of the electronic device, comprising: determining the jaggies parameter according to the application type and / or the scene type.

9. An image processing apparatus characterized by comprising: comprising: an acquisition module configured to acquire an application layer of an application program and a system layer of an electronic device in response to the application program entering a running state in the electronic device; a synthesis module configured to perform synthesis processing on the application layer and the system layer based on a first processor in the electronic device to obtain a first image; a determination module configured to determine a target processing parameter of a second processor in the electronic device based on a state parameter of the electronic device; wherein the target processing parameters corresponding to different state parameters are different; a super resolution processing module configured to perform super resolution processing on the first image based on the target processing parameter to obtain a second image, and perform picture display based on the second image.

10. An electronic device, comprising: comprising: a processor; a memory for storing computer programs or instructions; wherein the processor executes the computer programs or instructions to implement the steps of the method of any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing a computer program or instructions, wherein, when the computer programs or instructions in the storage medium are executed by the processor, the steps of the method of any one of claims 1 to 8 are implemented.

12. A computer program product comprising computer programs or instructions, characterized in that, when the computer programs or instructions are executed by the processor, the steps of the method of any one of claims 1 to 8 are implemented.

Images