Image processing method, electronic equipment, storage medium and program product

Abstract

The invention relates to the technical field of terminals, and discloses an image processing method, electronic equipment, a storage medium and a program product. The image processing method comprises the steps that the electronic equipment firstly processes first data through a first processing module, and when the electronic equipment detects that second data comprises the first data and other layer data which cannot be processed by the first processing module, the electronic equipment is switched to the second processing module to process the first data and other layer information at the same time. Wherein when the electronic equipment is switched from using the first processing module to using the second processing module, the second processing module can obtain a processing result of the first processing module on the first data, and the processing result and a processing result of the second processing module on other layers are sent to the screen together to be displayed. Therefore, when different processing modules of the electronic equipment are switched, and the image processing results of different processing modules on the same data are displayed on the screen, the same display effect can be achieved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and particularly relates to an image processing method, an electronic device, a storage medium, and a program product. Background Art

[0002] To improve the user experience of watching videos, electronic devices such as mobile phones support playing high dynamic range (HDR) videos. Among them, HDR videos achieve a larger exposure dynamic range (that is, a larger difference between light and dark) compared to ordinary videos, making the picture colors more vivid, the picture more vivid, and closer to what the human eye sees.

[0003] However, the original brightness of HDR videos is usually different from the brightness range that the electronic device screen can display. For example, the peak brightness of a frame of an image in an HDR video can be 10,000 nits (nit), while the highest peak brightness that the electronic device screen can display is 4,000 nit. Therefore, the electronic device needs to perform tone mapping on the HDR video, that is, adjust the brightness of the image to the brightness range that the electronic device can display, so as to be able to display the HDR video on the screen.

[0004] For example, the display processor unit (DPU) and the graphics processing unit (GPU) of some electronic devices can both perform tone mapping. Since the DPU is a dedicated image processing element with high performance and fast processing speed, the electronic device usually processes HDR videos through the DPU so that the HDR videos are displayed on the electronic device screen.

[0005] However, in some cases, when the screen simultaneously displays the HDR video playback interface and other windows (such as pop-up windows, sidebars, etc.), since the display effects of some windows can only be achieved through the GPU, the electronic device will switch from using the DPU to process the HDR video to using the GPU to process the layer to be displayed. For example, the GPU is used to simultaneously process the HDR video and other windows such as pop-up windows and sidebars. For example, when the user swipes the sidebar while playing an HDR video, since the GPU can simultaneously achieve tone mapping and the animation effects of the sidebar sliding in and out, the electronic device will switch from using the DPU to using the GPU to process the HDR video and the sidebar layer so that the HDR video and the sidebar are displayed on the screen together.

[0006] Since the hardware structures and processing algorithms in the DPU and the GPU are different, when the electronic device switches from using the DPU to using the GPU to perform tone mapping on the HDR video, the display effect of the HDR video on the screen will show obvious differences before and after the switch. Summary of the Invention

[0007] To solve the above problems, embodiments of the present application provide an image processing method, an electronic device, a storage medium, and a program product.

[0008] In a first aspect, the present application provides an image processing method applied to an electronic device. The electronic device includes a first processing module and a second processing module. The method includes: performing image processing on first data to be processed through the first processing module, where the image processing includes a first image operation; corresponding to the second data to be processed satisfying a processing conversion condition, performing image processing on the second data to be processed through the second processing module, where the image processing on the second data to be processed includes a first image operation and a second image operation, and the image processing of the second data to be processed by the second processing module includes: obtaining a first operation result of the first processing module performing the first image operation on the second data to be processed, and performing the second image operation on the second data to be processed to obtain a second operation result, and completing the image processing of the second data to be processed based on the first operation result and the second operation result.

[0009] In the present application, the first processing module may be a processor such as a DPU and / or GPU mentioned later; the second processing module may be a processor such as a DPU and / or GPU mentioned later; the first data to be processed may be data processed by the first processing module mentioned later, for example, HDR video data, image data for performing screen scaling processing, and image data for performing color space conversion, etc.; the first image operation may be an image operation corresponding to the first data to be processed mentioned later, for example, tone mapping, screen scaling processing, color space conversion, etc.; the first operation result may be a processing result of the first processing module processing the first data to be processed; the second data to be processed may include the first data to be processed mentioned later and other layer data such as pop-up windows and sidebars; the second image operation may be an image operation on layers such as pop-up windows and sidebars; the second operation result may be a processing result of other layers such as pop-up windows and sidebars mentioned later.

[0010] It can be understood that in embodiments of the present application, the electronic device first performs image processing on the first data to be processed through the first processing module. When the electronic device detects that the second data to be processed includes the first data to be processed and other layer data that the first processing module cannot process, the electronic device will process the first data to be processed by the first processing module and switch to the second processing module to process the first data to be processed and other layer information simultaneously. Among them, when the electronic device switches from using the first processing module to using the second processing module, the second processing module may obtain the processing result of the first processing module on the first data to be processed, and send the processing result and the processing result of the second processing module on other layers to the screen for display.

[0011] Thus, based on the above method, when the electronic device switches between different processing modules and displays the image processing results of the same data by different processing modules on the screen, the same display effect will be obtained.

[0012] In a possible implementation of the above first aspect, the first processing module includes a display processing unit, and the second processing module includes a graphics processor.

[0013] In the embodiments of the present application, the first processing module may be the DPU mentioned later, and the second processing module may be the GPU mentioned later.

[0014] In some embodiments, the first processing module and the second processing module may also be any other processor units that can implement the same functions, and the present application does not limit this.

[0015] In a possible implementation of the above first aspect, the first image operation includes one or more of the following: performing tone mapping on a high-dynamic-range video; scaling or sharpening the display screen of the electronic device; converting the image data from the first color value in the first color space to the second color value in the second color space; and the second image operation includes window rotation, window scaling, and a blur effect, where the blur effect includes but is not limited to: sliding the sidebar, displaying a pop-up window, displaying a multitasking interface, displaying the system volume keys, etc.

[0016] In the present application, the first color space may be the color space format before the color space conversion of the image data mentioned later; the first color value may be the color value of the image data in the first color space mentioned later; the second color space may be the color space format after the color space conversion of the image data mentioned later; the second color value may be the color value of the image data in the second color space mentioned later.

[0017] In the embodiments of the present application, the first image operation may include performing tone mapping on an HDR video. For example, adjusting the brightness of the HDR video to the brightness range that the electronic device can display, so that the HDR video can be displayed on the screen of the electronic device.

[0018] In some embodiments, the first image operation may also include scaling or sharpening the display screen, for example, adjusting the resolution of the video to the resolution size that the electronic device screen can display; or performing a sharpening operation such as adjusting the edge contrast of the image to enhance the edge details of the image.

[0019] In some embodiments, the first image operation may further include color space conversion, that is, converting color values in one color space into color values in another color space. For example, converting a video in the luminance chrominance (YUV) color space format into a video in the red green blue (RGB) color space format that can be displayed on the screen of an electronic device.

[0020] In the embodiments of the present application, the second image operation may be image operations such as window rotation, window scaling, and blur effects that the DPU cannot handle. For example, displaying a sidebar, displaying a pop-up window, displaying a multitasking interface, and displaying blur effect operations such as the system volume keys. Among them, window scaling may include floating window scaling, picture-in-picture scaling, etc.

[0021] In a possible implementation of the first aspect above, the processing conversion condition includes at least one of the following: the first processing module does not support the second image operation, and the second processing module supports the second image operation; the number of layers in the second data to be processed is greater than the number of layers that the first processing module can process.

[0022] In the embodiments of the present application, when the electronic device performs image processing on the second data to be processed, if there is a second image operation that cannot be completed by the first processing module, the electronic device will switch from processing the first data to be processed by the first processing module to processing the second data to be processed by the second processing module, that is, processing the first data to be processed and other layers by the second processing module at the same time.

[0023] In some embodiments, the number of layers that the DPU can process is lower than the number of layers that the GPU can process, and the processing conversion condition may also be that the number of layers is greater than the number of layers that the DPU can process. When the electronic device performs image processing on the second data to be processed, if the number of layers in the second data to be processed is greater than the number of layers that the DPU can process, the electronic device will switch from processing the first data to be processed by the DPU to processing the second data to be processed by the GPU, that is, processing the first data to be processed and other layers by the GPU at the same time.

[0024] In a possible implementation of the first aspect above, the second processing module obtains the first operation result of the first processing module performing the first image operation on the second data to be processed, including: the second processing module obtains the first operation result stored by the first processing module in the shared storage space.

[0025] In the embodiments of the present application, the first processing module and the second processing module may share a part of the memory space in the memory. The first processing module may store the first operation result in the shared memory space. When the electronic device switches from using the first processing module to using the second processing module, the second processing module may directly read the first operation result stored in this memory space. In this way, when the second processing module displays the first operation result as the processing result of the first image operation on the screen, the display effect of the data to be processed on the screen before and after the processing module switch is the same.

[0026] In a possible implementation of the foregoing first aspect, the second processing module obtains the first operation result of the first processing module performing a first image operation on the second data to be processed, including: the second processing module receives the first operation result sent by the first processing module.

[0027] In the embodiments of the present application, the second processing module may also obtain the first operation result of the first processing module through data transfer. Specifically, when the electronic device switches from using the first processing module to using the second processing module, the first processing module may directly send the first operation result to the second processing module. In this way, when the second processing module displays the first operation result as the processing result of the first image operation on the screen, the display effect of the data to be processed on the screen before and after the processing module switch is the same.

[0028] In a possible implementation of the foregoing first aspect, the electronic device further includes a layer compositor; corresponding to the first image operation being tone mapping of a high-dynamic range video, the method includes: after the second processing module receives the second data to be processed from the layer compositor, it obtains the first operation result in the shared storage space.

[0029] In the embodiments of the present application, when playing an HDR video on an electronic device, tone mapping needs to be performed on the HDR video. When the layer compositor (surfaceflinger, SF) determines that the second data to be processed meets the processing conversion condition, for example, the second data to be processed includes layer data such as pop-up windows and sidebars, SF will send the HDR video data and data of other layers such as pop-up layers and sidebars to the GPU. After the GPU receives the HDR video data and the data of each layer, the GPU will first obtain the first operation result of the DPU performing tone mapping on the HDR video from the memory space shared with the DPU, and then send the first operation result and the processing results of other layers to the screen for display. In this way, when the electronic device switches from using the DPU to using the GPU, the HDR video will have the same display effect on the screen.

[0030] In a possible implementation of the above first aspect, the electronic device further includes a layer compositor; corresponding to the first image operation being tone mapping of a high-dynamic range video, the method includes: the layer compositor sending second data to be processed to a second processing module, and sending a processing conversion instruction to a first processing module; the first processing module, in response to the processing conversion instruction, sending a first operation result of the first image operation to the second processing module.

[0031] In an embodiment of the present application, when the electronic device plays an HDR video, tone mapping needs to be performed on the HDR video. When the SF determines that the second data to be processed meets the processing conversion condition, for example, the second data to be processed includes layer data such as pop-up windows and sidebars, the SF will send the second data to be processed to the GPU and send a processing conversion instruction to the DPU. After receiving the processing conversion instruction, the DPU will send a first operation result of tone mapping the HDR video to the GPU, and the GPU will send the first operation result and the processing results of other layers to the screen for display. In this way, when the electronic device switches from using the DPU to using the GPU, the HDR video will have the same display effect on the screen.

[0032] In a possible implementation of the above first aspect, the electronic device further includes a layer compositor; corresponding to the first image operation being tone mapping of a high-dynamic range video, the method includes: the layer compositor detecting that the second data to be processed meets the processing conversion condition, and sending a processing conversion instruction to the first processing module; the first processing module, in response to the processing conversion instruction, sending a first operation result of the first image operation to the layer compositor; the layer compositor sending the first operation result and the second data to be processed to the second processing module.

[0033] In an embodiment of the present application, when the electronic device plays an HDR video, tone mapping needs to be performed on the HDR video. When the SF determines that the second data to be processed meets the processing conversion condition, for example, the second data to be processed includes layer data such as pop-up windows and sidebars, the SF will send a processing conversion instruction to the DPU. After receiving the processing conversion instruction, the DPU will send a first operation result of tone mapping the HDR video to the SF, and then the SF will send the first operation result and the second data to be processed to the GPU, and the GPU will send the first operation result and the processing results of other layers to the screen for display. In this way, when the electronic device switches from using the DPU to using the GPU, the HDR video will have the same display effect on the screen.

[0034] In a second aspect, the present application provides an electronic device, including: a layer synthesizer, a first processing module, and a second processing module; wherein, the layer synthesizer is configured to detect whether the data to be processed meets the processing conversion condition, and when the data to be processed meets the processing conversion condition and the first processing module performs image processing, control the second processing module to perform image processing; the first processing module is configured to perform image processing on the data to be processed, and the image processing includes a first image operation; the second processing module is configured to perform image processing on the data to be processed, wherein the image processing of the second processing module on the data to be processed includes: obtaining the operation result of the first image operation performed by the first processing module on the data to be processed, and performing a second image operation on the data to be processed.

[0035] In the present application, the first processing module may be a processor such as the DPU and / or GPU mentioned later; the second processing module may be a processor such as the DPU and / or GPU mentioned later; the first image operation may be a tone mapping, a screen scaling process, a color space conversion, etc. mentioned later; the second image operation may be an image operation on layers such as pop-up windows and sidebars mentioned later.

[0036] In an embodiment of the present application, when the electronic device first performs a first image operation on the data to be processed by the first processing module, when the SF determines that the next data to be processed meets the processing conversion condition, for example, when the data to be processed includes layer data such as pop-up windows and sidebars, the electronic device will switch from using the first processing module to using the second processing module. At the same time, the second processing module can obtain the processing result of the first image operation performed by the first processing module, and send the processing result and the processing results of other layers to the screen for display together.

[0037] In this way, when the electronic device switches different processing modules and displays the image processing results of different processing modules on the same data on the screen, there will be the same display effect.

[0038] In a possible implementation of the above second aspect, the first processing module includes a display processing unit, and the second processing module includes a graphics processor.

[0039] In an embodiment of the present application, the first processing module may be the DPU mentioned later, and the second processing module may be the GPU mentioned later.

[0040] In some embodiments, the first processing module and the second processing module may also be other processor units that can implement the same function, and the present application does not limit this.

[0041] In a possible implementation of the second aspect described above, the first image operation includes one or more of the following: performing tone mapping on a high-dynamic range video; scaling the display screen of the electronic device; converting the image data from a first color value in a first color space to a second color value in a second color space; and the second image operation includes window rotation, window scaling, and blur effects, where the blur effects include, but are not limited to: sliding the sidebar, displaying a pop-up window, displaying a multitasking interface, displaying the system volume keys, etc.

[0042] In this application, the first color space may be the color space format before the color space conversion of the image data mentioned later; the first color value may be the color value of the image data in the first color space mentioned later; the second color space may be the color space format after the color space conversion of the image data mentioned later; the second color value may be the color value of the image data in the second color space mentioned later.

[0043] In the embodiments of this application, the first image operation may include performing tone mapping on an HDR video. For example, compressing the HDR video from a high brightness to a low brightness range so that the HDR video can be displayed on the screen of the electronic device.

[0044] In some embodiments, the first image operation may also include scaling or sharpening the display screen. For example, adjusting the resolution of the video to the resolution size that can be displayed on the screen of the electronic device; or performing a sharpening operation such as adjusting the edge contrast of the image to enhance the edge details of the image.

[0045] In some embodiments, the first image operation may also include color space conversion, that is, converting the color value in one color space to the color value in another color space. For example, converting a video in the YUV color space format to a video in the RGB color space format that can be displayed on the screen of the electronic device.

[0046] In the embodiments of this application, the second image operation may be image operations such as window rotation, window scaling, and blur effects that the DPU cannot handle, such as blur effect operations like displaying the sidebar, displaying a pop-up window, displaying a multitasking interface, displaying the system volume keys, etc. Among them, window scaling may include floating window scaling, picture-in-picture scaling, etc.

[0047] In a possible implementation of the second aspect described above, the processing conversion conditions include at least one of the following: the first processing module does not support the second image operation, and the second processing module supports the second image operation; the number of layers in the data to be processed is greater than the number of layers that the first processing module can handle.

[0048] In the embodiments of the present application, when an electronic device performs image processing on data to be processed, if there is a second image operation that cannot be completed by the first processing module, the electronic device will switch from processing the data to be processed through the first processing module to processing the data to be processed through the second processing module, that is, the second processing module processes the data to be processed and other layers simultaneously.

[0049] In some embodiments, the number of layers that the DPU can process is lower than the number of layers that the GPU can process. The processing conversion condition can also be that the number of layers is greater than the number of layers that the DPU can process. When an electronic device performs image processing on data to be processed, if the number of layers in the data to be processed is greater than the number of layers that the DPU can process, the electronic device will switch from processing the data to be processed through the DPU to processing the data to be processed through the GPU, that is, the GPU processes the data to be processed and other layers simultaneously.

[0050] In a third aspect, the present application provides a readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the image processing method mentioned in the present application.

[0051] In a fourth aspect, the present application provides a computer program product, including: computer instructions. When the computer instructions run on an electronic device, the electronic device executes the image processing method mentioned in the present application.

[0052] For the beneficial effects of the above third aspect to the fourth aspect, reference can be made to the relevant descriptions in the above first aspect and various possible implementations of the first aspect, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 According to some embodiments of the present application, a schematic flowchart of an electronic device playing an HDR video is shown;

[0054] Figure 2 According to some embodiments of the present application, a schematic diagram of a scenario of switching from DPU to GPU is shown;

[0055] Figure 3 According to some embodiments of the present application, a schematic diagram of a scenario where the GPU obtains the processing result of the DPU is shown;

[0056] Figure 4 According to some embodiments of the present application, a schematic diagram of an application scenario of the first image processing method is shown;

[0057] Figure 5 According to some embodiments of the present application, a schematic diagram of an application scenario of the second image processing method is shown;

[0058] Figure 6According to some embodiments of the present application, a schematic diagram of the application scenario of the third image processing method is shown;

[0059] Figure 7 According to some embodiments of the present application, a schematic diagram of the application scenario of the fourth image processing method is shown;

[0060] Figure 8 According to some embodiments of the present application, a schematic diagram of the system architecture of an electronic device is shown;

[0061] Figure 9A According to some embodiments of the present application, a schematic diagram of the process of an image processing method is shown;

[0062] Figure 9B According to some embodiments of the present application, a schematic diagram of the interaction process of an image processing method is shown;

[0063] Figure 10 According to some embodiments of the present application, a schematic diagram of the first scenario for layer composition processing is shown;

[0064] Figure 11 According to some embodiments of the present application, a schematic diagram of the second scenario for layer composition processing is shown;

[0065] Figure 12 According to some embodiments of the present application, a schematic diagram of the hardware structure of an electronic device is shown. Detailed implementation manners

[0066] The illustrative embodiments of the present application include but are not limited to an image processing method, an electronic device, a storage medium, and a program product.

[0067] To more clearly understand the solution of the present application, first, the relevant field terms involved in the present application are explained.

[0068] (1) Layer: Each application program can correspond to one or more graphical interfaces, which can also be called windows, and each graphical interface can be called a surface. Each graphical interface can be a layer. For example, in a video application program, it can include multiple layers such as a video layer, a progress bar layer, a selection box layer, and a bullet screen layer.

[0069] (2) Layer composition: Integrate the drawing contents of each layer, and stack, combine, and adjust each layer according to its hierarchical relationship on the screen. Among them, layer composition is divided into hardware composer (HWC) composition and GPU composition. Among them, HWC composition refers to layer composition through the hardware DPU. Therefore, layer composition can also be divided into DPU composition and GPU composition.

[0070] (3) HDR: A processing technology that improves video brightness and contrast. Compared with ordinary videos, HDR videos can provide more image details, richer colors, and more vivid pictures closer to what the human eye sees. Among them, the peak brightness of HDR videos can reach 10000 nits, and the peak brightness that the screen of an electronic device can display can reach 4000 nits.

[0071] (4) Tone mapping: Adjust the brightness of the HDR video to the brightness that the electronic device can display. For example, compress the HDR video from high brightness to a low brightness range so that the HDR video can be displayed on the screen of the electronic device; or adjust the brightness of the HDR video to the high brightness range that the electronic device can display. For example, the brightness of a frame of an HDR video is 10000 nits, while the maximum brightness that the screen of the electronic device can display is 1000 nits. Then the electronic device needs to perform tone mapping to adjust the brightness of the image from 10000 nits to 0 - 1000 nits so that the screen can display the image. Among them, tone mapping can be performed in both the DPU and the GPU.

[0072] The following briefly describes the application background of the image processing method provided in the embodiments of the present application.

[0073] When an electronic device plays an HDR video, it needs to decode and perform image processing on the HDR video source, and then display the HDR video on the screen. Exemplarily, as Figure 1 shown, the process of an electronic device playing an HDR video includes:

[0074] 101: The video decoder in the electronic device decodes the HDR video source.

[0075] The user can search for HDR videos through the electronic device in applications such as websites and video players. Based on the operation of the user triggering the playback of the HDR video, the electronic device obtains the HDR video source from the database in applications such as websites and video players.

[0076] The electronic device needs to perform decoding processing on the obtained HDR video source through the video decoder. For example, decode video data in h.264 and h.265 formats into video data in luminance chrominance (YUV) format. The video decoder can also be used for audio - video synchronization calibration of HDR videos.

[0077] 102: The surfaceflinger (SF) in the electronic device determines the processing module for processing the HDR video.

[0078] The processing module for processing HDR videos includes a GPU and a DPU. After the video decoder decodes the HDR video source, the SF needs to determine whether to use the DPU or the GPU to process the HDR video based on the actual screen display scenario, such as whether there is a sidebar, a pop-up window, etc.

[0079] Among them, the SF process is used to manage the layers to be displayed on the screen. If it is determined that the display effects of all the layers to be displayed on the screen can be achieved by the DPU, the SF will pass the decoded HDR video into the DPU for tone mapping. If it is determined that the display effects of some of the layers to be displayed need to be achieved through the GPU, the SF can pass the decoded HDR video into the GPU for tone mapping.

[0080] 103: The SF processes the HDR video through the DPU.

[0081] Since the DPU has high performance and fast processing speed, the SF in the electronic device usually passes the HDR video data into the DPU to perform tone mapping, layer composition and other processing on the HDR video. Among them, the DPU has a dedicated circuit and hardware structure for image processing and display.

[0082] 104: The SF processes the HDR video through the GPU.

[0083] The GPU can achieve various image processing effects (for example, sliding animation effects, etc.). If the SF determines that the display effects of some layers to be displayed cannot be completed by the DPU, for example, when the user slides the sidebar while playing the HDR video, or when the electronic device pops up a pop-up notification while playing the HDR video, the SF will perform tone mapping and layer composition processing on the HDR video through the GPU.

[0084] Among them, compared with the DPU, there is no dedicated circuit or hardware structure for image processing and display in the GPU, resulting in different tone mapping processing effects of the GPU on the HDR video and the DPU on the HDR video. For example, the processing effect of the GPU is weaker than that of the DPU.

[0085] In addition, in some embodiments, different algorithms will be adopted in the DPU and the GPU to process the HDR video. For example, the GPU can directly perform linear compression on the pixel brightness in the HDR video to map the high brightness to the low brightness range; while the DPU can adopt a fitting curve method based on the visual curve of the human eye's perceived brightness to perform non-linear compression on the pixels in the HDR video. Due to the different processing algorithms in the DPU and the GPU, the tone mapping processing effects of the GPU on the HDR video and the DPU on the HDR video are different.

[0086] 105: The DPU of the electronic device performs layer blending processing on the HDR video after tone mapping is completed.

[0087] The DPU of the electronic device can perform layer blending processing on the HDR video after tone mapping is completed. For example, it can adjust the brightness and contrast of the human image and the environmental image in the HDR video.

[0088] 106: The electronic device inputs the processed HDR video into the buffer through the destination surface processor pipeline.

[0089] The destination surface processor pipes (DSPP) can transfer the processed HDR video data to the buffer in the direct memory access (DMA) so that the processor can deliver the HDR video data in the buffer to the screen for display.

[0090] 107: The screen of the electronic device receives the HDR video data and displays the HDR video.

[0091] The processor of the electronic device (such as the DPU or GPU) can send the HDR video data to the screen for display. Or, the DPU or GPU can also send the HDR video data to the buffer and then send it to the screen for display after receiving the central processing unit (CPU) instruction.

[0092] In this way, based on the above process, the playback of the HDR video can be realized.

[0093] As mentioned above, the electronic device usually uses the DPU to process the HDR video (such as 103 in Figure 1 ), however, in some cases, the electronic device will switch to using the GPU to process the HDR video (such as 104 in Figure 1 ). Taking the electronic device as the mobile phone 10 as an example, Figure 2 shows a scenario where an electronic device switches from using the DPU to using the GPU.

[0094] Usually, when the user uses the electronic device to play the HDR video, the sidebar, pop-up window, volume bar, etc. can also be displayed on the screen of the electronic device. Among them, the display effect of this part of the window layer needs to be realized through the GPU. For example, as shown in (a) of Figure 2 , when the mobile phone 10 is playing the HDR video normally, the mobile phone 10 performs tone mapping on the HDR video through the DPU. If the user swipes out the sidebar of the mobile phone 10 while playing the HDR video, as shown in Figure 2As shown in (b) therein, since the DPU can implement tone mapping but cannot implement the in-out animation effect of the sidebar, while the GPU can implement tone mapping and the in-out animation effect of the sidebar, therefore, in Figure 2 In (b) of Figure 2 , the mobile phone 10 will process the HDR video and the sidebar layer simultaneously through the GPU, so that the HDR video and the sidebar are displayed on the screen of the mobile phone 10 at the same time. That is, in Figure 2 In (b) of Figure 2 , the mobile phone 10 performs tone mapping on the HDR video through the GPU.

[0095] However, as described in 103 and 104 above Figure 1 Since the hardware structures or processing algorithms in the DPU and the GPU are different, when the electronic device switches from using the DPU to using the GPU, the display effect of the HDR video on the screen will have obvious differences before and after the switch, such as inconsistent brightness. For example, in the scenario shown in Figure 2 When the mobile phone 10 switches from the DPU to the GPU, Figure 2 the HDR video shown in (b) of Figure 2 is brighter than the HDR video shown in Figure 2 In (a) of Figure 2 , resulting in overexposure. The difference in the display effect before and after the switch affects the user viewing experience.

[0096] To solve the above problems, the present application provides an image processing method. In the method of the present application, when the electronic device switches from using the first processing module to using the second processing module, the second processing module can obtain the processing result of the first processing module and use this processing result as the output result of the second processing module. In this way, when the output results of different processing modules are displayed on the screen when the electronic device switches different processing modules, there will be the same display effect.

[0097] Among them, the second processing module can obtain the processing result of the first processing module in various ways. For example, sharing a mapping relationship memory: The first processing module and the second processing module can share a part of the memory space in the memory. The first processing module can store the processing result (for example, the mapping relationship) in the shared memory space. When the electronic device switches from using the first processing module to using the second processing module, the second processing module can directly read the processing result stored in this memory space and use this processing result as the output result of the second processing module to be sent to the screen for display. Among them, the mapping relationship is the corresponding relationship between the initial data and the data after being processed by the first processing module. For example, if the initial data is X and the first processing module calculates X to obtain data Y, then the first processing module can store the mapping relationship X→Y in the shared memory space, and the second processing module directly reads this mapping relationship from the memory space and displays the data Y as the output result of the second processing module on the screen.

[0098] In some other embodiments, the second processing module may also obtain the processing result of the first processing module by means of data transfer. Specifically, when the electronic device switches from using the first processing module to using the second processing module, the first processing module may directly send the processing result to the second processing module. After receiving the processing result, the second processing module may directly use the processing result as the output result of the second processing module and display it on the screen.

[0099] It should be understood that the image processing method mentioned above in the present application can be applied to two or more processing modules in the same electronic device. For example, the second processing module may obtain the processing result of the first processing module; or, multiple processing modules such as the second processing module and the third processing module may simultaneously obtain the processing result of the first processing module. In addition, the image processing method mentioned above in the present application can also be applied to two or more processing modules in different electronic devices. The present application does not make any limitation in this regard.

[0100] In some embodiments, the first processing module may be the DPU mentioned above, and the second processing module may be the GPU mentioned above. In some other embodiments, the first processing module and the second processing module may also be any other processor units that can implement the same functions. The present application does not make any limitation in this regard.

[0101] It should be understood that the image processing method mentioned above in the present application can be applied to any scenario where the use is switched from the first processing module to the second processing module. For example, in some embodiments, the image processing method mentioned in the present application can be applied to scenarios such as video resolution scaling or video image sharpening. Specifically, when the electronic device adjusts the resolution of a video to a resolution size that can be displayed on the electronic device screen; or when performing a sharpening operation such as adjusting the edge contrast of an image, the electronic device generally uses the first processing module (such as a DPU) to perform scaling processing on the picture to be adjusted. However, when the display effects of other windows (such as sidebars, pop-up windows, etc.) on the screen need to be implemented by the second processing module (such as a GPU), the electronic device will switch from using the first processing module to implement picture scaling to using the second processing module to simultaneously implement picture scaling and the display effects of other windows. Among them, when the electronic device switches from using the first processing module to using the second processing module, the second processing module directly uses the picture scaling processing result of the first processing module as the output result and displays it on the screen. In this way, when the electronic device switches different processing modules and displays the picture scaling processing results of different processing modules on the screen, there will be the same display effect.

[0102] For another example, in some other embodiments, the image processing method mentioned in the present application can be used in scenarios of color space conversion. Specifically, if different processing modules can all perform color space conversion processing operations, that is, convert color values in one color space into color values in another color space. For example, when an electronic device plays a video in the luminance chrominance (YUV) color space format, it is necessary to convert the video data into the red green blue (RGB) color space format that can be displayed on the screen of the electronic device. An electronic device usually uses a first processing module (such as a DPU) to perform color space conversion on the YUV format video. However, when the display effects of other windows (such as sidebars, pop-up windows, etc.) on the screen need to be implemented by a second processing module (such as a GPU), the electronic device will switch from using the first processing module to implement color space conversion to using the second processing module to simultaneously implement color space conversion and the display effects of other windows. Among them, when the electronic device switches from using the first processing module to using the second processing module, the second processing module directly uses the color space conversion processing result of the first processing module as the output result.

[0103] For yet another example, in some other embodiments, the image processing method mentioned above in the present application can also be applied to the scenario where an electronic device switches from using a first processing module to using a second processing module when playing an HDR video. Specifically, the first processing module can be a DPU, and the second processing module can be a GPU. As Figure 3 shown, Figure 3 shows three HDR video playback scenarios (a), (b), and (c). Among them, Figure 3 in (a) of Figure 3 , the picture 301 is the video image after the mobile phone 10 uses the DPU to perform tone mapping processing on the HDR video,

[0104] and in (b) of Figure 3As shown in (c), when the mobile phone 10 switches from using the DPU to using the GPU, the GPU can obtain the processing result of the DPU by sharing the mapped relationship memory (3011), or the DPU can directly send the processing result to the GPU (3012). After the GPU obtains the processing result of the DPU for tone mapping of the HDR video, the GPU directly uses the processing result of the DPU as the output result and displays it on the screen. At this time, the video image in the picture 303 of the HDR video tone-mapped by the DPU is the same as the display effect of the video image in the picture 304 of the HDR video tone-mapped by the switched-to GPU.

[0105] It can be understood that the electronic device applicable to the image processing method provided by the embodiments of the present application may include, but is not limited to, any electronic device such as a mobile phone, a computer, a tablet computer, an augmented reality (AR) device, a laptop computer, etc. The present application does not make any limitation in this regard.

[0106] Still taking the electronic device as the mobile phone 10 as an example below, and taking the scenario of switching from the DPU to the GPU when the electronic device plays an HDR video mentioned above as an example, based on Figures 4 - 7 A variety of application scenarios in the embodiments of the present application will be briefly described.

[0107] As mentioned above, the mobile phone 10 usually uses the DPU to perform tone mapping on the HDR video. However, in some cases, the mobile phone 10 will switch to using the GPU to perform tone mapping on the HDR video. For example, when the mobile phone 10 plays an HDR video, in scenarios such as when the mobile phone 10 pops up a pop-up notification, enters the multitasking interface, or the user swipes out the navigation bar (such as the top navigation bar, the side bar, etc.), or adjusts the volume, etc., since the GPU can implement various image processing such as tone mapping, layer composition processing, animation effects, and image rendering, the mobile phone 10 will switch from the DPU to the GPU to perform tone mapping on the HDR video.

[0108] In some embodiments, for example, as Figure 4 shown in (a), when the mobile phone 10 plays an HDR video in landscape mode, the user can rotate the mobile phone 10, as Figure 4 shown in (b), so that the mobile phone 10 plays the HDR video in portrait mode. Among them, in Figure 4In (a) of the figure, the mobile phone 10 performs tone mapping on the HDR video through the DPU. During the process of the mobile phone 10 detecting its rotation, the mobile phone 10 switches from the DPU to the GPU to process the layer to be displayed, and the GPU can directly obtain the processing result of the DPU's tone mapping of the HDR video, for example, by means of the above-mentioned shared mapping relationship memory, data transfer, etc. to obtain the processing result of the DPU's tone mapping of the HDR video. Then, the GPU processes other layers, such as the progress bar layer, the selection box layer, etc., and also sends the processing result of the DPU and the processing results of other layers to the screen of the mobile phone 10 for display.

[0109] Among them, in Figure 4 In (b) of the figure, when the mobile phone 10 is stationary, for example, when no new window pops up and only the layer corresponding to the HDR video needs to be refreshed, the mobile phone 10 can also switch back to the DPU to perform tone mapping on the HDR video.

[0110] In this way, before rotating the mobile phone 10, during the rotation of the mobile phone 10, and after rotating the mobile phone 10, the layer corresponding to the HDR video is processed by the DPU, and thus the display effect of the HDR video on the screen of the mobile phone 10 remains unchanged.

[0111] In some other embodiments, for example, as shown in (a) of Figure 5 When the mobile phone 10 plays an HDR video, the mobile phone 10 performs tone mapping on the HDR video through the DPU. Among them, when the mobile phone 10 plays an HDR video, the user can swipe out the sidebar. As shown in (b) of Figure 5 When the mobile phone 10 detects that the sidebar layer needs to be displayed, the mobile phone 10 switches from the DPU to the GPU to process the layer to be displayed, and the GPU can directly obtain the processing result of the DPU's tone mapping of the HDR video, for example, by means of the above-mentioned shared mapping relationship memory, data transfer, etc. to obtain the processing result of the DPU's tone mapping of the HDR video. And send the processing result of the DPU and the processing result of the sidebar layer to the screen of the mobile phone 10 for display.

[0112] Among them, in Figure 5 In (c) of the figure, after the sidebar leaves the video playback interface, that is, when the mobile phone 10 detects that there is no sidebar layer in the layer to be displayed and only the layer corresponding to the HDR video needs to be refreshed, the mobile phone 10 can also switch back to the DPU to perform tone mapping on the HDR video.

[0113] In this way, before swiping in the sidebar, during the display of the sidebar, and after swiping out the sidebar, the layer corresponding to the HDR video is processed by the DPU, and then the display effect of the HDR video on the screen of the mobile phone 10 remains unchanged.

[0114] For another example, in some other embodiments, as Figure 6 shown in (a) of [reference], when the mobile phone 10 plays an HDR video, the mobile phone 10 will perform tone mapping on the HDR video through the DPU. Among them, when the mobile phone 10 plays an HDR video, the user can swipe up to enter the multitasking interface of the mobile phone 10. As Figure 6 shown in (b) of [reference], when the mobile phone 10 detects that the multitasking interface needs to be displayed, the mobile phone 10 will switch from the DPU to the GPU to process the layer to be displayed, and the GPU can directly obtain the processing result of the DPU after tone mapping the HDR video. For example, the processing result of the DPU after tone mapping the HDR video can be obtained through the above-mentioned shared mapping relationship memory, data transfer, etc., and the processing result of the DPU and the processing results of other layers are sent to the screen of the mobile phone 10 for display together.

[0115] Among them, in Figure 6 (c) of [reference], when the user enters the video playback interface again, that is, when the mobile phone 10 detects that only the layer corresponding to the HDR video needs to be refreshed, the mobile phone 10 can switch back to the DPU to perform tone mapping on the HDR video.

[0116] In this way, before entering the multitasking interface, during the process of displaying the multitasking interface, and after exiting the multitasking interface, the layer corresponding to the HDR video is processed by the DPU, so the display effect of the HDR video on the screen of the mobile phone 10 remains unchanged.

[0117] For another example, in some other embodiments, as Figure 7 shown in (a) of [reference], when the mobile phone 10 plays an HDR video, the mobile phone 10 will perform tone mapping on the HDR video through the DPU. Among them, when the mobile phone 10 plays an HDR video, the user can adjust the volume. As Figure 7 shown in (b) of [reference], when the mobile phone 10 detects that the volume needs to be adjusted, the mobile phone 10 will switch from the DPU to the GPU to process the layer to be displayed, and the GPU can directly obtain the processing result of the DPU after tone mapping the HDR video. For example, the processing result of the DPU after tone mapping the HDR video can be obtained through the above-mentioned shared mapping relationship memory, data transfer, etc., and the processing result of the DPU and the processing results of the volume layer are sent to the screen of the mobile phone 10 for display together.

[0118] Among them, in Figure 7 (c) of [reference], when the user completes the operation of adjusting the volume, that is, when the mobile phone 10 detects that only the layer corresponding to the HDR video needs to be refreshed, the mobile phone 10 can switch back to the DPU to perform tone mapping on the HDR video.

[0119] Thus, before adjusting the volume, during the process of adjusting the volume, and after adjusting the volume, the layer corresponding to the HDR video is processed by the DPU, so the display effect of the HDR video on the screen of the mobile phone 10 remains unchanged.

[0120] It should be understood that in addition to the above-mentioned scenarios of the sliding sidebar, display pop-up window, display system volume keys, and entering the multi-tasking interface with various blur animation effects, the image processing method mentioned in this application can also be applied to any scenario where the DPU is switched to the GPU when playing an HDR video. For example, it can be applied to the window rotation scenario. For example, changing from playing an HDR video in landscape mode to playing it in portrait mode; or for example, window scaling scenarios such as zooming in on a floating window or zooming in on a picture-in-picture. The application scenarios of the image processing method in this application are not limited.

[0121] In addition, the image processing method mentioned in this application can also be applied to any scenario where the GPU is switched to the DPU when playing an HDR video. For example, when using the floating window function of an electronic device, the GPU is used to process the floating window layer. At this time, if an HDR video is played, the electronic device will simultaneously process the HDR video and the floating window layer through the GPU. When the electronic device detects that the floating window is closed and only needs to process the HDR video, the electronic device can switch to the DPU to perform tone mapping on the HDR video. Among them, when the electronic device switches from using the GPU to using the DPU, the DPU can obtain the processing result of the GPU by sharing the mapping relationship memory, or the GPU can directly send the processing result to the DPU. After the DPU obtains the processing result of the GPU for tone mapping of the HDR video, the DPU directly displays the processing result of the GPU as the result of tone mapping of the HDR video on the screen. At this time, the display effect of tone mapping the HDR video through the GPU is the same as the display effect of switching to the DPU for tone mapping of the HDR video.

[0122] Still taking the electronic device as the mobile phone 10 as an example below, combined with Figure 8 , a simple description of the system architecture of the electronic device will be given.

[0123] As Figure 8 shown, a system architecture block diagram of the mobile phone 10 is shown. Among them, the layered architecture divides the system into several layers, each layer has a clear role and division of labor, and the layers communicate through software interfaces.

[0124] In some embodiments, the system of the mobile phone 10 can be divided into an application layer, an application framework layer, system libraries of Android runtime ( runtime), and a kernel layer from top to bottom.

[0125] AsFigure 8 As shown, the application layer may include a series of application packages. For example, it may include applications such as a camera, a gallery, a short message, a video playback software, etc. Among them, the video playback software can be used to play the HDR video mentioned in this application.

[0126] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0127] As Figure 8 shown, the application framework layer may include a surface flinger, a notification manager, a window manager, a content provider, a view system, a resource manager, etc.

[0128] The surface flinger is a system service. The surface flinger can create corresponding layers for the applications in the application framework layer. During the running of the applications in the application framework layer, the surface flinger can synthesize the layers through the GPU and / or DPU. Among them, the surface flinger can also perform tone mapping on the HDR video through the GPU and / or DPU.

[0129] The notification manager enables the application to display notification information in the status bar. It can be used to convey message of the notification type, and it can disappear automatically after a short stay without user interaction. For example, the notification manager is used to inform that the download is completed, message reminder, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or a scroll bar text, such as the notification of a background running application, and it can also be a notification that appears on the screen in the form of a dialogue layer, such as prompting text information in the status bar, emitting a prompt sound, vibrating the electronic device, flashing the indicator light, etc.

[0130] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0131] The content provider is used to store and obtain data, and enable these data to be accessed by applications. The data may include HDR videos, images, audio, dialed and answered calls, browsing history and bookmarks, phone books, etc.

[0132] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface (i.e., the layer) can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0133] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

[0134] The system library can include multiple functional modules. For example: the surface manager, the media database, the 2D graphics engine, etc.

[0135] The kernel layer is the layer between the hardware and the software. The kernel layer at least includes driver modules such as the DPU driver, the GPU driver, and the display driver. Among them, the DPU driver is used to drive the DPU, the GPU driver is used to drive the GPU, and the display driver is used to drive the display screen.

[0136] In addition, the hardware of the mobile phone 10 includes devices such as the DPU, the GPU, and the display screen. Among them, both the DPU and the GPU can perform functions such as layer composition and tone mapping.

[0137] The display screen can be a liquid crystal display (LCD), a light-emitting diode (LED) display screen, a cathode ray tube (CRT) display screen, a projector, a holographic imaging, etc.

[0138] Next, based on Figure 9A the flowchart shown below, the processing process of the image processing method mentioned in this application will be briefly described. Among them, this method can be applied to an electronic device, such as the mobile phone 10 shown above Figure 8 as shown. Specifically, as Figure 9A shown, this method includes the following steps:

[0139] S9001: Perform a first image operation on the first data to be processed through the first processing module.

[0140] In the embodiments of this application, the first processing module can be the DPU or the GPU mentioned above. The first data to be processed can be data that can be processed by the first processing module, such as HDR video data, data for performing screen scaling processing, and data for performing color space conversion, etc. The first image operation can be operations such as tone mapping, screen scaling processing, and color space conversion.

[0141] In some embodiments, an electronic device generally uses a first processing module (e.g., DPU) to perform tone mapping processing on an HDR video. For example, the HDR video is compressed from high brightness to a low brightness range so that the HDR video can be displayed on the screen of the electronic device.

[0142] In other embodiments, the electronic device can also use the first processing module to perform color space conversion on an image or video. For example, when the electronic device plays a video in the YUV color space format, the video data needs to be converted into the RGB color space format that can be displayed on the screen of the electronic device.

[0143] In addition, the electronic device can also use the first processing module to perform scaling processing on the screen. It should be understood that this application does not limit the first data to be processed and the corresponding first image operation.

[0144] S9002: It is detected that the second data to be processed meets the processing conversion condition.

[0145] In the embodiments of this application, the second data to be processed includes the first data to be processed and other data. The image processing of the second data to be processed includes the first image operation and the second image operation mentioned above; the conversion condition can be that the second image operation cannot be completed by the first processing module and can only be completed by the second processing module.

[0146] In some embodiments, when the electronic device performs image processing on the second data to be processed, if there is a second image operation that cannot be completed by the first processing module, the electronic device will switch from processing the first data to be processed by the first processing module to processing the second data to be processed by the second processing module, that is, the second processing module processes the first data to be processed and other data at the same time.

[0147] For example, in some embodiments, the second data to be processed may include an HDR video (i.e., the first data to be processed) and data such as a sidebar and a pop-up window. If the first frame display screen of the electronic device is an HDR video playback interface and the second frame display screen is an HDR video playback interface and a sidebar layer, before the electronic device displays the second frame display screen, if the electronic device detects that the data to be processed includes sidebar data, the electronic device will switch from the DPU to the GPU and process the HDR video and the sidebar through the GPU.

[0148] S9003: Control the second processing module to obtain the first operation result of the first processing module performing the first image operation.

[0149] In the embodiments of this application, the image processing of the second data to be processed by the second processing module includes: the first image operation on the first data to be processed in the second data to be processed, and the second image operation on other data.

[0150] In an embodiment of the present application, when the electronic device switches from performing a first image operation on first data to be processed by a first processing module to performing a first image operation on the first data to be processed in second data to be processed by a second processing module, the second processing module may obtain a first operation result of the first processing module and use this first operation result as the processing result of the second processing module for performing the first image operation. For example, if the second data to be processed is an HDR video (i.e., the first data to be processed) and sidebar data, the GPU may obtain the tone mapping processing result of the DPU for the second data to be processed, that is, obtain the tone mapping processing result of the DPU for the HDR video.

[0151] In some embodiments, the second processing module may obtain the first operation result of the first processing module in various ways. For example, the first processing module and the second processing module may share the storage space of the first operation result; for another example, the first processing module may directly send the first operation result to the second processing module.

[0152] S9004: Perform a second image operation on the second data to be processed by a second data processing module.

[0153] In an embodiment of the present application, the second data processing module also needs to perform other image processing operations on the second data to be processed. For example, layer composition, and some image processing operations that the first processing module cannot complete, etc.

[0154] S9005: Complete the image processing of the second data to be processed.

[0155] In an embodiment of the present application, after the electronic device executes the above steps S9003 and S9004, that is, after completing the first image operation and the second image operation, it means that the image processing of the second data to be processed is completed.

[0156] In this way, when the electronic device switches different processing modules and displays the image processing results of different processing modules for the same data on the screen, there will be the same display effect.

[0157] The following combines the interaction process diagram. Taking the layer to be displayed including the corresponding picture of the HDR video as an example, and the first processing module being the DPU and the second processing module being the GPU as an example, the interaction process of the image processing method mentioned in the present application will be specifically described. Figure 9B This is a specific interaction flowchart of an image processing method provided by an embodiment of the present application. This method can be applied to the Figure 8 mobile phone 10 shown above. As Figure 9B shown, this method includes the following steps:

[0158] 901: SurfaceFlinger obtains the HDR video and determines the processing module for processing the HDR video.

[0159] In the embodiment of the present application, when the user plays the HDR video through the mobile phone 10, the mobile phone 10 needs to decode the obtained HDR video source. For example, the video data in h.264 or h.265 format is decoded into video data in YUV format. Then the decoded video data is passed into SurfaceFlinger, and SurfaceFlinger determines whether to use the DPU or GPU to process the HDR video based on the actual screen display scenario. For example, whether the layer to be displayed includes a sidebar, a pop-up window, etc.

[0160] In some embodiments, the SurfaceFlinger process is used to manage the layers to be displayed on the screen. If it is determined that the display effects of all the layers to be displayed on the screen can be achieved by the DPU, then SF will pass the HDR video into the DPU for tone mapping. If it is determined that the display effects of some of the layers to be displayed need to be achieved through the GPU, then SurfaceFlinger can pass the HDR video into the GPU for tone mapping.

[0161] 902: SurfaceFlinger sends the HDR video to the DPU, and the DPU processes the HDR video.

[0162] In the embodiment of the present application, since the DPU has high performance and fast processing speed, SurfaceFlinger in the mobile phone 10 usually passes the HDR video data into the DPU, and the DPU performs tone mapping on the HDR video. Among them, the DPU has a dedicated circuit and hardware structure responsible for image processing and display.

[0163] 903: The DPU performs tone mapping on the HDR video and stores the processing result in the shared memory space.

[0164] In the embodiment of the present application, after receiving the HDR video data, the DPU will perform tone mapping on the HDR video so that the HDR video can be displayed on the screen of the mobile phone 10. For example, the brightness of a frame of image in the HDR video is 10000 nit, while the maximum brightness that the screen of the mobile phone 10 can display is 1000 nit. Then the mobile phone 10 needs to perform tone mapping to adjust the brightness of the image from 10000 nit to 0-1000 nit so that the screen can display the image.

[0165] In the embodiment of the present application, the DPU and the GPU can share a part of the memory space in the memory. After the DPU performs tone mapping processing on the HDR video, the DPU can store the processing result in the shared memory space.

[0166] In some embodiments, the DPU may store the mapping relationship between the original HDR video and the tone-mapped HDR video in a memory space. For example, as shown in Table 1, the mapping relationship between the frames of each HDR video may be stored in a mapping table, and the mapping table (e.g., Table 1) may be stored in a shared memory space.

[0167] Table 1

[0168] HDR video image Initial image data Image data after tone mapping First frame image X1 Y1 Second frame image X2 Y2 … … … nth frame image Xn Yn

[0169] In Table 1, the original image data may include the original pixel brightness of the image, and the image data after tone mapping may include the adjusted brightness. Among them, the image brightness in the tone-mapped HDR video is lower than that in the original HDR video.

[0170] 904: The DPU sends the tone-mapped HDR video to the screen.

[0171] In the embodiments of the present application, after the DPU performs tone mapping on the HDR video, the HDR video data may be sent to the screen of the mobile phone 10, so as to display the HDR video on the screen of the electronic device.

[0172] In some embodiments, after the DPU performs tone mapping on the HDR video, it may also perform layer blending processing on the HDR video. For example, adjust the brightness contrast of the human image and the environmental image in the HDR video, and then transmit the HDR video to the buffer through DSPP. The screen may read the HDR video data in the buffer and display the HDR video on the screen. The embodiments of the present application do not specifically limit the process of the DPU sending the tone-mapped HDR video to the screen.

[0173] In other embodiments, the tone-mapped HDR video data may also be stored in the buffer, and the HDR video data in the buffer may be sent to the screen for display by the processor of the electronic device (e.g., DPU or GPU).

[0174] 905: The screen displays the HDR video.

[0175] In the embodiments of the present application, after the screen receives the HDR video data sent by the DPU, or after the processor of the mobile phone 10 sends the HDR video in the buffer to the screen, the screen may display the HDR video.

[0176] 906: surfaceflinger obtains the HDR video data and the information of each layer.

[0177] In some embodiments, when the user plays an HDR video using the mobile phone 10, sidebars, progress bars, selection boxes, and other layers can also be displayed on the screen of the mobile phone 10.

[0178] In the embodiments of the present application, surfaceflinger obtains the layer information in the next frame of the screen display. The layer information may include data such as layer attributes or display content. Among them, the layer attributes may include the display effects of the layers on the screen, for example, may include the size and position area of the layer (such as the position coordinates on the screen), or may also include animation information such as fade-in / fade-out, rotation, and scaling of the layer.

[0179] It should be understood that when an HDR video is displayed on the screen, the HDR video is also a layer. Therefore, the layer information obtained by surfaceflinger includes the layer information of the HDR video layer and the layer information of other display content.

[0180] In some embodiments, the screen of the mobile phone 10 first displays the first frame of the picture. Before the second frame of the picture is displayed on the screen, all the layer information of the second frame needs to be passed into surfaceflinger, and surfaceflinger determines to perform layer processing through the DPU or GPU.

[0181] 907: surfaceflinger determines that the layer information meets the processing conversion condition.

[0182] In the embodiments of the present application, the processing conversion condition may be data related to layer display effects that cannot be achieved by the DPU in the layer information, for example, animation information such as fade-in / fade-out and rotation.

[0183] In some embodiments, the number of layers that the DPU can process is lower than the number of layers that the GPU can process. The processing conversion condition may also be that the number of layers is greater than the number of layers that the DPU can process.

[0184] 908: surfaceflinger sends the HDR video data and the layer information to the GPU.

[0185] In the embodiments of the present application, when surfaceflinger determines that the layer information meets the processing conversion condition, it means that the DPU cannot simultaneously process the display effects of the HDR video and other layers. Then surfaceflinger needs to pass the layer information and HDR data into the GPU, and the GPU performs tone mapping on the HDR video and realizes the display effects of the layers, so that other layers and the HDR video can be displayed on the screen at the same time.

[0186] 909: The GPU obtains the processing result of tone mapping the HDR video stored in the shared memory space.

[0187] In the embodiment of the present application, after performing the above step 908, that is, after the GPU receives the HDR video data and each layer information sent by surfaceflinger, the GPU will first obtain from the memory space shared with the DPU the processing result of the DPU for tone mapping the HDR video.

[0188] In some embodiments, the DPU may store the mapping relationship between each frame image in the HDR video in a mapping table, and store the mapping table (such as the above Table 1) in the shared memory space, and the GPU reads the mapping table from the shared memory space.

[0189] In addition, in some embodiments, after the GPU obtains the HDR video data sent by surfaceflinger and the mapping table obtained from the shared memory space, data matching is also required. If it is determined that the HDR video data corresponding to the mapping table is consistent with the HDR video data sent by surfaceflinger, the GPU can directly adjust the brightness of the HDR video to the brightness that can be displayed on the screen of the mobile phone 10 corresponding to the mapping table.

[0190] In other embodiments, in the above step 903, after the DPU performs tone mapping on the HDR video, the processing result may not be stored in the shared memory space. In the above step 908, when surfaceflinger sends the HDR video data and each layer information to the GPU, surfaceflinger may also send an instruction to the DPU, so that the DPU directly sends the processing result of the HDR video to the GPU.

[0191] In other embodiments, in the above step 903, after the DPU performs tone mapping on the HDR video, the processing result may not be stored in the shared memory space. In the above step 907, when surfaceflinger determines that the layer information meets the processing conversion condition, surfaceflinger may also send an instruction to the DPU, so that the DPU directly sends the processing result of the HDR video to surfaceflinger. Then, in the above step 908, surfaceflinger will send the HDR video data and each layer information that have been tone mapped by the DPU to the GPU.

[0192] 910: The GPU synthesizes the tone-mapped HDR video layer and other layers into a display screen.

[0193] In the embodiments of the present application, the screen of the mobile phone 10 displays a synthesized image of multiple layers, that is, multiple layers are synthesized into one image. After the GPU obtains the HDR video data that has been tone-mapped by the DPU, the GPU synthesizes the tone-mapped HDR video layer and other layers into a display image. For example, as shown in (a) of Figure 10 , the HDR video layer 1 after tone mapping and the volume bar layer 2 are layer-synthesized. As shown in (b) of Figure 10 , the synthesized display image 1010 is obtained. Another example is shown in (a) of Figure 11. The HDR video layer 3 after tone mapping and the pop-up window layer 4 are layer-synthesized. As shown in (b) of Figure 11 , the synthesized display image 1110 is obtained.

[0194] It should be understood that the present application can perform layer synthesis through any algorithm, and the present application does not limit this.

[0195] 911: The GPU sends the display image to the screen.

[0196] In the embodiments of the present application, the GPU can send the synthesized display image to the screen for display.

[0197] 912: The screen displays the display image sent by the GPU.

[0198] In the embodiments of the present application, after the screen receives the display image sent by the GPU, it will display the image.

[0199] It should be understood that in step 910 above, the GPU synthesizes the tone-mapped HDR video layer and other graphics into a display image. Therefore, the image displayed on the screen includes the HDR video.

[0200] In addition, since the GPU obtains the HDR video data that has been tone-mapped by the DPU and then performs layer synthesis processing with other layers. Therefore, the display effect of the HDR video displayed on the screen in step 905 is the same as that of the HDR video displayed on the screen in step 912.

[0201] In this way, based on the above interaction process, when the mobile phone 10 switches from using the DPU to using the GPU, the HDR video will have the same display effect on the screen.

[0202] In addition, the present application also provides another image processing method to solve the problem mentioned above that when playing an HDR video on an electronic device and switching from the DPU to the GPU, the display effect of the HDR video on the screen of the electronic device is different. Among them, in this image processing method, different processing modules, such as the DPU and the GPU, can adopt the same tone mapping processing algorithm to perform tone mapping on the HDR video, so that when the electronic device switches different processing modules and the output results of different processing modules are displayed on the screen, there will be the same display effect.

[0203] Among them, this image processing method can be executed by an electronic device. Specifically, this image processing method may include the following:

[0204] Performing image processing on the first data to be processed by a first processing module; among them, the image processing includes a first image operation, and the first processing module executes the first image operation using a first processing algorithm; detecting that the second data to be processed meets the processing conversion condition, where the image processing of the second data to be processed includes a first image operation and a second image operation; through a second processing module, performing image processing on the second data to be processed, where the image processing of the second data to be processed by the second processing module includes: executing the first image operation on the second data to be processed using the first processing algorithm to obtain a first operation result, and performing a second image operation on the second data to be processed to obtain a second operation result, and completing the image processing of the second data to be processed based on the first operation result and the second operation result.

[0205] It should be understood that if both the first processing module and the second processing module execute the first image operation using the first processing algorithm, the same first image operation processing result will be obtained. Therefore, when the electronic device switches from the first processing module to using the second processing module, there will be the same display effect of the first image operation on the screen of the electronic device.

[0206] In some embodiments, if both the first processing module and the second processing module can obtain the input data of the first processing algorithm, indicating that the first processing algorithm can be used in both the first processing module and the second processing module, then different processing modules have the same processing result when performing the first image operation on the same data.

[0207] In some embodiments, for example, when an electronic device plays an HDR video, in a scenario where the electronic device switches from using a DPU to using a GPU. For example, the first tone mapping algorithm needs to receive ambient light information, the maximum brightness information that the electronic device screen can display, and the brightness information of the HDR video as input data. The first tone mapping algorithm performs tone mapping on the HDR video based on these input data to output a processing result. If both the DPU and the GPU can obtain the above three types of information as the input data of the first tone mapping algorithm, it means that both the DPU and the GPU can use the first tone mapping algorithm to perform tone mapping on the HDR video. Then, when the electronic device switches from using the DPU to using the GPU, the display effect of the HDR video on the electronic device screen is the same. If the DPU can obtain the above three types of information as the input data of the first tone mapping algorithm, while the GPU cannot obtain the three types of information (for example, can only obtain two of them) as the input data of the first tone mapping algorithm, it means that the DPU can use the first tone mapping algorithm, and only other tone mapping algorithms can be used in the GPU.

[0208] In some embodiments, when different tone mapping algorithms are used in the DPU and the GPU. For example, if the DPU uses the first tone mapping algorithm and other tone mapping algorithms are used in the GPU, then when the electronic device switches from using the DPU to using the GPU, the GPU can obtain the processing result of the HDR video by the DPU through the first tone mapping algorithm through the method Figure 9B shown above and display the processing result on the screen of the electronic device, so that the display effect of the HDR video on the electronic device screen is the same before and after the switch.

[0209] In some embodiments, if the DPU and the GPU can execute the same tone mapping algorithm, then when the electronic device switches from using the DPU to using the GPU to perform tone mapping on the HDR video, the GPU directly uses the same algorithm as in the DPU for tone mapping processing. However, if the DPU and the GPU cannot execute the same tone mapping algorithm, then when the electronic device switches from using the DPU to using the GPU to perform tone mapping on the HDR video, the GPU needs to obtain the processing result of the DPU for tone mapping the HDR video to achieve the same display effect. For example, execute the Figure 9B method shown above.

[0210] An embodiment of the present application provides a readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the image processing method mentioned in the present application.

[0211] An embodiment of the present application further provides a computer program product, including: computer instructions. When the computer instructions run on an electronic device, the electronic device executes the image processing method mentioned in the present application.

[0212] In addition, an embodiment of the present application further provides an electronic device, including: a layer synthesizer, a first processing module, and a second processing module; wherein, the layer synthesizer is configured to detect whether the data to be processed meets the processing conversion condition, and when the data to be processed meets the processing conversion condition and the first processing module performs image processing, control the second processing module to perform image processing; the first processing module is configured to perform image processing on the data to be processed, and the image processing includes a first image operation; the second processing module is configured to perform image processing on the data to be processed, wherein the image processing of the data to be processed by the second processing module includes: obtaining the operation result of the first processing module performing the first image operation on the data to be processed, and performing a second image operation on the data to be processed.

[0213] The image processing method provided by the embodiment of the present application can be applied to an electronic device, and the electronic device can be any electronic device such as a mobile phone, a computer, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, etc., and the embodiment of the present application does not make any limitation thereto.

[0214] Still taking the electronic device as the mobile phone 10 as an example below, an exemplary schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application is described.

[0215] As Figure 12 shown, an exemplary schematic diagram of the hardware structure of the mobile phone 10 according to an embodiment of the present application is described. As Figure 12 shown, the mobile phone 10 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0216] It can be understood that the structure schematically shown in the embodiment of the present application does not constitute a specific limitation on the mobile phone 10. In other embodiments of the present application, the mobile phone 10 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0217] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a controller, a digital signal processor (DSP), a baseband processor, a display processor unit (DPU), and a graphics processing unit (GPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0218] In this application, the DPU and GPU can be used to execute the image processing methods mentioned in this application. Among them, both the DPU and GPU can perform tone mapping on HDR videos and implement layer composition processing.

[0219] The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0220] A memory may also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0221] In the embodiments of this application, the screen 194 can be used to display HDR videos and the display images after synthesizing each layer.

[0222] The wireless communication function of the mobile phone 10 can be implemented through the antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor, etc.

[0223] The antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone 10 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0224] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the mobile phone 10. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter, amplify, and process the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be provided in the same device.

[0225] The wireless communication module 160 can provide solutions for wireless communications such as wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), etc. applied to the mobile phone 10. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves through the antenna 2, performs frequency modulation and filtering on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be transmitted from the processor 110, perform frequency modulation and amplification on it, and convert it into electromagnetic waves through the antenna 2 for radiation.

[0226] In some embodiments, the antenna 1 of the mobile phone 10 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone 10 can communicate with the network and other devices through wireless communication technologies.

[0227] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc. The data storage area can store data created during the use of the mobile phone 10 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the mobile phone 10 by running the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor 110.

[0228] The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0229] The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation from the mobile phone 10. The mobile phone 10 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The mobile phone 10 interacts with the network through the SIM card to implement functions such as calls and data communication.

[0230] The charging management module 140 is used to receive a charging input from a charger. Among them, the charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive the charging input from the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the mobile phone 10. While charging the battery 142, the charging management module 140 can also supply power to the mobile phone 10 through the power management module 141.

[0231] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the screen 194, the camera 193, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery charge cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0232] The embodiments disclosed in the present application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memories and / or storage elements), at least one input device, and at least one output device.

[0233] The program code can be applied to the input instructions to perform the various functions described in the present application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of the present application, the processing system includes any system having a processor such as, for example, a digital signal processor, a microcontroller, an application processor, or a microprocessor.

[0234] The program code can be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system. When needed, the program code can also be implemented in assembly language or machine language. In fact, the mechanisms described in the present application are not limited to the scope of any specific programming language. In any case, the language can be a compiled language or an interpreted language.

[0235] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more transient or non-transitory machine-readable storage media (e.g., computer-readable storage media), which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable media. Thus, machine-readable media may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to, floppy disks, optical disks, optical discs, magneto-optical discs, read-only memory (ROM), random access memory (RAM), magnetic or optical cards, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using electrical, optical, acoustic, or other forms of propagated signals via the Internet. Thus, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0236] In the drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or ordering may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0237] It should be noted that each unit / module mentioned in the device embodiments of this application is a logical unit / module. Physically, a logical unit / module may be a physical unit / module, a part of a physical unit / module, or may be implemented as a combination of multiple physical units / module. The physical implementation manner of these logical units / modules themselves is not the most important. The combination of the functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. In addition, in order to highlight the innovative part of this application, the above device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that there are no other units / modules in the above device embodiments.

[0238] It should be noted that in the examples and the description of this application, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one" does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0239] Although this application has been illustrated and described by reference to certain preferred embodiments thereof, those of ordinary skill in the art should understand that various changes may be made therein in form and detail without departing from the scope of this application.

Claims

1. An image processing method, applied to an electronic device, the electronic device including a first processing module and a second processing module, characterized in that, The method includes: Performing image processing on first data to be processed by the first processing module, where the image processing includes a first image operation; Corresponding to the second data to be processed satisfying a processing conversion condition, performing image processing on the second data to be processed by the second processing module, where the image processing on the second data to be processed includes a first image operation and a second image operation, and The image processing of the second data to be processed by the second processing module includes: Obtaining a first operation result of the first image operation performed by the first processing module on the second data to be processed, and performing a second image operation on the second data to be processed to obtain a second operation result, and completing the image processing of the second data to be processed based on the first operation result and the second operation result.

2. The method according to claim 1, characterized in that, The first processing module includes a display processing unit, and the second processing module includes a graphics processor.

3. The method according to claim 2, wherein The first image operation includes one or more of the following: Performing tone mapping on a high-dynamic range video; Scaling or sharpening the display screen of the electronic device; Converting image data from a first color value in a first color space to a second color value in a second color space; and The second image operation includes window rotation, window scaling, and a blur effect, where the blur effect includes one or more of: sliding a sidebar, displaying a pop-up window, displaying a multitasking interface, and displaying a system volume key.

4. The method according to claim 3, characterized in that, The processing conversion condition includes at least one of the following: The first processing module does not support the second image operation, and the second processing module supports the second image operation; The number of layers in the second data to be processed is greater than the number of layers that the first processing module can process.

5. The method according to claim 4, characterized in that The second processing module obtaining the first operation result of the first image operation performed by the first processing module on the second data to be processed includes: The second processing module obtaining the first operation result stored by the first processing module in a shared storage space.

6. The method according to claim 4, characterized in that, The second processing module obtaining the first operation result of the first image operation performed by the first processing module on the second data to be processed includes: The second processing module receiving the first operation result sent by the first processing module.

7. The method according to claim 5, characterized in that, The electronic device further includes a layer compositor; Corresponding to the first image operation being performing tone mapping on a high-dynamic range video, the method includes: After receiving the second data to be processed from the layer compositor, the second processing module obtaining the first operation result in the shared storage space.

8. The method according to claim 6, wherein The electronic device further includes a layer compositor; Corresponding to the first image operation being performing tone mapping on a high-dynamic range video, the method includes: The layer compositor sending the second data to be processed to the second processing module, and sending a processing conversion instruction to the first processing module; The first processing module, in response to the processing conversion instruction, sending the first operation result of the first image operation to the second processing module.

9. The method according to claim 6, wherein The electronic device further includes a layer compositor; Corresponding to the first image operation for tone mapping a high-dynamic range video, the method includes: The layer compositor detects that the second data to be processed meets the processing conversion condition, and sends a processing conversion instruction to the first processing module; In response to the processing conversion instruction, the first processing module sends the first operation result of the first image operation to the layer compositor; The layer compositor sends the first operation result and the second data to be processed to the second processing module.

10. An electronic device, characterized in that, Including: A layer compositor, a first processing module, and a second processing module; wherein, The layer compositor is configured to detect whether the data to be processed meets the processing conversion condition, and control the second processing module to perform image processing when the data to be processed meets the processing conversion condition and the first processing module performs image processing; The first processing module is configured to perform image processing on the data to be processed, and the image processing includes a first image operation; The second processing module is configured to perform image processing on the data to be processed. Among them, the image processing of the data to be processed by the second processing module includes: obtaining the operation result of the first image operation performed by the first processing module on the data to be processed, and performing a second image operation on the data to be processed.

11. The electronic device according to claim 10, wherein The first processing module includes a display processing unit, and the second processing module includes a graphics processor.

12. The electronic device according to claim 11, wherein The first image operation includes one or more of the following: Tone mapping a high-dynamic range video; Scaling the display screen of the electronic device; Converting the image data from a first color value in a first color space to a second color value in a second color space; And The second image operation includes window rotation, window scaling, and a blur effect. Among them, the blur effect includes one or more of: sliding the sidebar, displaying a pop-up window, displaying a multitasking interface, and displaying the system volume keys.

13. The electronic device according to claim 12, characterized in that, The processing conversion condition includes at least one of the following: The first processing module does not support the second image operation, and the second processing module supports the second image operation; The number of layers in the data to be processed is greater than the number of layers that the first processing module can process.

14. A readable storage medium, characterized in that, Instructions are stored on the readable storage medium, and when the instructions are executed on the electronic device, the electronic device executes the image processing method according to any one of claims 1-9.

15. A computer program product, characterized in that, Including: Computer instructions, when the computer instructions run on the electronic device, cause the electronic device to execute the image processing method according to any one of claims 1-9.

Images