Image processing method, apparatus and system, electronic device and medium

Abstract

Disclosed in the present application are an image processing method, apparatus and system, an electronic device and a storage medium, which belong to the technical field of image processing. The method comprises: by means of a main control chip and on the basis of interface data of a first interface, acquiring a graphics layer, a user interface layer and a target system interface layer corresponding to the first interface, wherein the resolution of the target system interface layer is less than the resolution of a screen; by means of the main control chip, obtaining a first superimposed image on the basis of the graphics layer and an application interface layer, and transmitting a target image to an image processing chip, wherein the resolution of the first superimposed image is less than the resolution of the target system interface layer, the target image comprises a second superimposed image, or the first superimposed image and the target system interface layer, and the second superimposed image is a superimposed image obtained by the main control chip on the basis of the first superimposed image and the target system interface layer; and processing the target image by means of the image processing chip to obtain an interface image corresponding to the first interface, wherein the resolution of the interface image is equal to the resolution of the screen.

Description

Image processing methods, apparatus, systems, electronic devices and media

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202410802653.5, filed in China on June 20, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of image processing technology, and more specifically, to an image processing method, apparatus, system, electronic device, and medium. Background Technology

[0004] A discrete graphics processing chip (hereinafter referred to as a discrete graphics chip) has significant advantages in improving the display effect of electronic display devices. By processing display frame rate, image resolution, and color adjustment through the discrete graphics chip, the visual experience can be significantly enhanced. Generally, the discrete graphics chip is placed between the main control chip and the screen. The main control chip draws the various layers corresponding to the interface on the screen to obtain the corresponding interface image. Then, it transmits the interface image to the image processing chip for further processing, and finally displays the processed interface image on the screen.

[0005] Currently, discrete graphics chips do not process images in a pipelined manner. Therefore, multiple frames of images need to be stored in the random access memory (RAM) within the discrete graphics chip. However, to display higher-quality images on the screen, the screen resolution can already reach 1440P. Thus, high-resolution images need to be processed within the discrete graphics chip. However, due to the large data volume of high-resolution images and the limited RAM size within the discrete graphics chip, in related technologies, after receiving a high-resolution image, the discrete graphics chip needs to downsample the received high-resolution image into a low-resolution image, or crop a portion of the image from the high-resolution image, in order to reduce the amount of data processed by the discrete graphics chip.

[0006] However, whether downsampling a high-resolution image into a low-resolution image or cropping a portion of a high-resolution image, both will result in a loss of image quality. Summary of the Invention

[0007] The purpose of this application is to provide an image processing method, apparatus, system, electronic device, and medium that can improve image quality.

[0008] In a first aspect, embodiments of this application provide an image processing method applied to an image processing apparatus, the image processing apparatus including a main control chip and an image processing chip, comprising:

[0009] Based on the interface data of the first interface, the main control chip obtains the screen layer, user interface layer and target system interface layer corresponding to the first interface. The resolution of the target system interface layer is smaller than the screen resolution.

[0010] The main control chip obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to the image processing chip; the resolution of the first overlay image is less than the resolution of the target system interface layer; the target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer; the second overlay image is an overlay image obtained by the main control chip based on the first overlay image and the target system interface layer;

[0011] The target image is processed by an image processing chip to obtain an interface image corresponding to the first interface, wherein the resolution of the interface image is equal to the resolution of the screen.

[0012] Secondly, embodiments of this application provide an image processing apparatus, including a main control chip and an image processing chip, comprising:

[0013] The main control chip is used to obtain the screen layer, user interface layer and target system interface layer corresponding to the first interface based on the interface data of the first interface. The resolution of the target system interface layer is smaller than the screen resolution.

[0014] The main control chip is also used to overlay a first superimposed image based on the screen layer and the application interface layer, and transmit the target image to the image processing chip; the resolution of the first superimposed image is less than the resolution of the target system interface layer; the target image includes a second superimposed image, or the target image includes the first superimposed image and the target system interface layer; the second superimposed image is a superimposed image obtained by the main control chip based on the first superimposed image and the target system interface layer;

[0015] An image processing chip is used to process the target image to obtain the interface image corresponding to the first interface. The resolution of the interface image is equal to the resolution of the screen.

[0016] Thirdly, embodiments of this application provide an image processing system, including a main control chip and an image processing chip. The main control chip includes an image processor GPU and a display processor DPU, and the DPU is connected to the GPU and the image processing chip respectively.

[0017] The GPU is used to obtain the screen layer, user interface layer and target system interface layer corresponding to the first interface based on the interface data of the first interface. The resolution of the target system interface layer is smaller than the screen resolution.

[0018] The GPU is also used to obtain the first overlay image based on the screen layer and the application interface layer, and to transfer the target image to the DPU;

[0019] The DPU is used to transmit the target image to the image processing chip; the resolution of the first overlay image is smaller than the resolution of the target system interface layer; the target image includes the second overlay image, or the target image includes the first overlay image and the target system interface layer; the second overlay image is an overlay image obtained by the DPU based on the first overlay image and the target system interface layer.

[0020] An image processing chip is used to process a target image to obtain an interface image corresponding to a first interface, wherein the resolution of the interface image is equal to the resolution of the screen.

[0021] Fourthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0022] Fifthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0023] In a sixth aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0024] In a seventh aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0025] In this embodiment, the main control chip obtains the screen layer, user interface layer, and target system interface layer corresponding to the first interface based on the interface data of the first interface. The resolution of the target system interface layer is less than the screen resolution of the electronic device. The main control chip obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to the image processing chip. The resolution of the first overlay image is less than the resolution of the target system interface layer. The target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer. The second overlay image is the overlay image obtained by the main control chip based on the first overlay image and the target system interface layer. The image processing chip processes the target image to obtain the interface image corresponding to the first interface, and the resolution of the interface image is equal to the screen resolution. In this solution, when the main control chip in the image processing device overlays the screen layer and the application interface layer corresponding to the first interface, it controls the resolution of the overlay image to be less than the resolution of the target system interface layer corresponding to the first interface, so that the resolution of the target image transmitted by the main control chip to the image processing chip is less than the screen resolution. This enables the image processing chip to process low-resolution target images. Therefore, the image processing chip processes target images with a smaller data volume, and does not need to downsample or crop the target image, thereby improving the image quality of the interface image. Attached Figure Description

[0026] Figure 1 is a schematic diagram of the structure of an electronic device provided in some embodiments of this application;

[0027] Figure 2 is a schematic diagram of one of the image processing procedures provided in some embodiments of this application;

[0028] Figure 3 is a flowchart of an image processing method provided in some embodiments of this application;

[0029] Figure 4 is a second schematic diagram of the image processing process provided in some embodiments of this application;

[0030] Figure 5 is a schematic diagram of the image processing process provided in some embodiments of this application.

[0031] Figure 6 is a schematic diagram of the image processing apparatus provided in an embodiment of this application;

[0032] Figure 7 is a schematic diagram of the image processing system provided in an embodiment of this application;

[0033] Figure 8 is a second schematic diagram of the structure of an electronic device provided in some embodiments of this application;

[0034] Figure 9 is a third schematic diagram of the structure of an electronic device provided in some embodiments of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0036] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0037] The terms "at least one," "at least one," etc., in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included objects. For example, "at least one of a, b, and c" can mean "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more, and its meaning is similar to that of "at least one."

[0038] It should be noted that the image processing method provided in this application can be executed by electronic devices such as mobile phones, tablets, laptops, PDAs, and in-vehicle electronic devices. Some embodiments of this application use electronic devices as the executing entity to illustrate the image processing method provided in this application.

[0039] The following will explain the terminology used in the embodiments of this application.

[0040] A system-on-a-chip (SoC), also known as a system-level chip or main control chip, is a system composed of multiple integrated circuits with specific functions. It typically includes modules such as a central processing unit (CPU), a graphics processing unit (GPU), a display processing unit (DPU), a digital signal processor (DSP), random access memory (RAM), and a modem.

[0041] The Mobile Industry Processor Interface (MIPI) is an alliance established by mobile industry manufacturers to standardize hardware and software interfaces.

[0042] Display Serial Interface (DSI) is a protocol interface under the MIPI protocol specifically used for displaying data.

[0043] General Purpose Input / Output (GPIO) is used to transmit control commands.

[0044] The receiver (RX) is the receiving end of the discrete graphics chip, which receives the content to be displayed.

[0045] The transmitting end (Transport, TX) is the transmitting end of the discrete graphics chip, which sends the content that has been processed by the discrete graphics chip.

[0046] Motion estimation and motion compensation (MEMC) can insert intermediate frames into low frame rate videos to make the video smoother, based on the continuity of dynamic images.

[0047] Artificial Intelligence Super Resolution (AISR) uses neural networks to restore low-resolution images to high-resolution images.

[0048] Image quality (PQ) refers to color adjustments made to achieve a desired visual effect.

[0049] User interface (UI) design (or interface design) refers to the overall design of software's human-computer interaction, operational logic, and aesthetics. UI design is divided into physical UI and virtual UI; the UI design discussed on the internet refers to virtual UI.

[0050] The image processing method and electronic device provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0051] The image processing method provided in this application embodiment can be applied to interface display scenarios, which can be video interfaces, game interfaces, or other interfaces. This embodiment does not impose specific limitations here.

[0052] The following explanation uses a game interface display scenario as an example. When a user wants to view the game application interface, they can trigger an action by interacting with the game application icon displayed on the electronic device screen. The main control chip in the electronic device responds to the user's trigger action on the game application icon, acquiring the interface data of the game interface in real time. This interface data includes image data of the game screen layer and the UI layer of the game interface, which the main control chip obtains in real time from the game application's backend server, as well as image data of the system layer obtained from the main interface of the game application.

[0053] Furthermore, as shown in Figure 2, the GPU within the main control chip will use image data from the game screen layer to draw, for example, a game screen layer layer1 with a resolution of 720P, and use image data from the UI layer of the game interface to draw, for example, a UI layer layer2 of the game interface with a resolution of 810P, and use image data from the system layer to draw a system layer layer3 with the same resolution as the screen at 1440P.

[0054] Furthermore, the GPU overlays the game screen layer (layer1) and the UI layer (layer2) to obtain a first overlaid image (layer4) with a resolution of 810P. Then, the DPU overlays this first overlaid image (layer4) with the system layer (layer3) to obtain a game interface overlaid image with a resolution of 1440P, corresponding to the game interface. Further still, the DPU transmits this game interface overlaid image to the DSI RX interface of the discrete graphics chip 12 via DSI in MIPI format, while simultaneously transmitting image processing instructions from the main control chip's GPIO to the image processing chip's GPIO. Upon receiving the image processing instructions and the game interface overlaid image, the image processing chip processes the overlaid image according to the instructions to obtain the interface image corresponding to the game interface, and then transmits the interface image to the screen for display.

[0055] Currently, image processing chips do not use a pipelined approach for image processing. Therefore, multiple frames of interface images need to be stored in the RAM of the image processing chip. However, the data volume of a 1440P resolution interface image is very large. Due to the limitation of the RAM size within the discrete graphics chip, currently, after receiving a 1440P resolution interface image, the image processing chip needs to downsample the received interface image into a low-resolution downsampled image, or crop a portion of the image from the 1440P resolution interface image. Then, the internal MEMC performs motion estimation and motion compensation on the downsampled image or the portion of the image, and uses AISR to restore the downsampled image or the portion of the image to a 1440P resolution interface image. PQ is then used to adjust the colors of the interface image. Finally, the image processing chip transmits the interface image to the screen in MIPI format via DSI TX for display.

[0056] However, when an image processing chip receives a 1440P resolution interface image and downsamples it to a lower resolution, it loses a corresponding proportion of image data. When the AISR restores the downsampled image to 1440P resolution, it only linearly enlarges the interface image, failing to recover the lost image data and thus degrading image quality. Similarly, when restoring a cropped area to 1440P resolution, the remaining cropped areas are not recovered, resulting in further image quality loss.

[0057] The image processing method, image processing apparatus, electronic device, and medium provided in this application embodiment obtain a screen layer, a user interface layer, and a target system interface layer corresponding to the first interface based on interface data of the first interface by a main control chip. The resolution of the target system interface layer is less than the resolution of the screen of the electronic device. The main control chip obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to an image processing chip. The resolution of the first overlay image is less than the resolution of the target system interface layer. The target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer. The second overlay image is the overlay image obtained by the main control chip based on the first overlay image and the target system interface layer. The image processing chip processes the target image to obtain an interface image corresponding to the first interface, the resolution of which is equal to the resolution of the screen. In this solution, when the main control chip in the electronic device overlays the screen layer and the application interface layer corresponding to the first interface, it controls the resolution of the overlay image to be less than the resolution of the target system interface layer corresponding to the first interface, so that the resolution of the target image transmitted by the main control chip to the image processing chip is less than the resolution of the screen. This enables the image processing chip to process low-resolution target images. Therefore, the image processing chip processes target images with a smaller data volume, and does not need to downsample or crop the target image, thereby improving the image quality of the interface image.

[0058] The image processing method provided in this application can be executed by an electronic device, or at least one of the functional modules and physical modules within the electronic device capable of implementing the image processing method. The specific implementation can be determined based on actual usage requirements, and this invention does not impose limitations. The following explanation uses an electronic device executing the image processing method as an example to illustrate the image processing method provided in this application.

[0059] This application provides an image processing method applied to an image processing device. Figure 3 is a flowchart illustrating the image processing method provided in this application. As shown in Figure 3, the image processing method provided in this application may include the following steps:

[0060] Step 301: The image processing device obtains the screen layer, user interface layer and target system interface layer corresponding to the first interface based on the interface data of the first interface through the main control chip.

[0061] The resolution of the target system interface layer is smaller than the resolution of the electronic device screen.

[0062] For example, when the screen resolution is 1440P, the resolution of the target system interface layer can be 1080P. It should be noted that this embodiment only provides illustrative examples of the screen resolution and the target system interface layer resolution. The screen resolution of 1440P and the target system interface layer resolution can also be other values, which are not specifically shown in this embodiment.

[0063] In some embodiments of this application, the interface data of the first interface includes image data of the screen layer corresponding to the first interface, image data of the user interface layer corresponding to the first interface, and image data of the target system interface layer corresponding to the first interface.

[0064] In some embodiments of this application, the image data of the screen layer corresponding to the first interface and the image data of the user interface layer corresponding to the first interface are obtained in real time by the image processing device from the background server of the application corresponding to the first interface through the main control chip.

[0065] In some embodiments of this application, the image data of the target system interface layer corresponding to the first interface is obtained by the image processing device from the main interface of the first application through the main control chip.

[0066] For example, when the first interface is a game interface, the interface data of the game interface includes the image data of the game screen layer and the image data of the UI layer of the game interface, which are obtained in real time from the background server of the game application by the image processing device through the main control chip, as well as the image data of the system layer obtained by the main control chip from the main interface of the game application.

[0067] In some embodiments of this application, the image processing device draws a screen layer based on the image data of the screen layer corresponding to the first interface through the main control chip, draws a UI layer of the game interface based on the image data of the UI layer of the game interface corresponding to the first interface, and draws a target system interface layer based on the image data of the system layer.

[0068] In some embodiments of this application, as shown in FIG1, the image processing device can draw a screen layer based on the image data of the screen layer corresponding to the first interface through the GPU 111 in the main control chip 11, draw a UI layer of the game interface based on the image data of the UI layer of the game interface corresponding to the first interface, and draw a target system interface layer based on the image data of the system layer.

[0069] Step 302: The image processing device obtains the first superimposed image based on the screen layer and the application interface layer through the main control chip, and transmits the target image to the image processing chip.

[0070] The resolution of the first overlaid image is lower than that of the target system interface layer. The target image includes a second overlaid image, or the target image includes the first overlaid image and the target system interface layer; the second overlaid image is an overlaid image obtained by the main control chip based on the first overlaid image and the target system interface layer.

[0071] In some embodiments of this application, after obtaining the first overlay image through the main control chip, the image processing device can directly transmit the first overlay image and the target system interface layer to the image processing chip for processing.

[0072] In some embodiments of this application, as shown in FIG1, the image processing device generates a first overlay image through the GPU 111 within the main control chip 11. Then, the image processing device can control the DPU 112 to transmit the first overlay image and the target system interface layer to the image processing chip 12 via the DSI 113.

[0073] In some embodiments of this application, as shown in FIG1, the image processing device controls the DPU 112 to transmit the first overlay image and the target system interface layer to the DSI RX 121 of the image processing chip 12 through the DSI interface 113, while simultaneously controlling the DPU 112 to transmit image processing instructions to the GPIO 122 of the image processing chip 12 through the GPIO 114.

[0074] It is understandable that after the image processing device obtains the first overlay image through the main control chip, it can directly transmit the first overlay image and the target system interface layer to the image processing chip for processing. This allows the image processing chip to process the low-resolution first overlay image and the target system interface layer. Therefore, the image processing chip processes the first overlay image and the target system interface layer with a smaller data volume. The image processing chip does not need to downsample or crop the first overlay image and the target system interface layer, thereby improving the image quality of the interface image.

[0075] In some embodiments of this application, after obtaining the first superimposed image through the main control chip, the image processing device can further obtain a second superimposed image based on the first superimposed image and the target system interface layer, and then directly transmit the second superimposed image to the image processing chip for processing.

[0076] In some embodiments of this application, as shown in FIG1, the image processing device generates a first overlay image through the GPU 111 within the main control chip 11, and transmits the generated first overlay image to the DPU 112 through the GPU 111. Then, the DPU 112 overlays the first overlay image and the target system interface image to obtain a second overlay image. Further, the image processing device can control the DPU 112 to transmit the target image to the image processing chip 12 through the DSI 113.

[0077] In some embodiments of this application, as shown in FIG1, the image processing device controls the DPU 112 to transmit the second superimposed image to the DSI RX 121 of the image processing chip 12 via DSI 113, while simultaneously controlling the DPU 112 to transmit image processing instructions to the GPIO 122 of the image processing chip 12 via GPIO 114.

[0078] It is understandable that after obtaining the first superimposed image through the main control chip, the image processing device can further obtain a second superimposed image based on the first superimposed image and the target system interface layer. Then, the second superimposed image is directly transmitted to the image processing chip for processing, so that the image processing chip can process the low-resolution second superimposed image. Therefore, the image processing chip processes the second superimposed image and the target system interface layer with a smaller data volume, and the image processing chip does not need to downsample or crop the second superimposed image, thereby improving the image quality of the interface image.

[0079] Step 303: The image processing device processes the target image through the image processing chip to obtain the interface image corresponding to the first interface.

[0080] The resolution of the interface image corresponding to the first interface is equal to the screen resolution.

[0081] In some embodiments of this application, as shown in FIG1, the image processing device can control the image processing chip 12 to receive the target image through the DSI RX 121 within the image processing chip 12.

[0082] In some embodiments of this application, when the target image includes a first overlay image and a target system interface layer, the above step 303 can be implemented by the following steps 303a and 303b.

[0083] Step 303a: The image processing device uses an image processing chip to overlay the first overlay image and the target system interface layer to obtain a third overlay image.

[0084] In some embodiments of this application, as shown in FIG1, the image processing device can directly overlay the first overlay image and the target system interface layer through the GPU 112 of the image processing chip 11 to obtain the third overlay image, or it can upsample the first overlay image through the GPU 112 and overlay the obtained image with the target system interface layer to obtain the third overlay image.

[0085] Step 303b: The image processing device processes the third superimposed image using an image processing chip, according to the screen resolution, to obtain the interface image.

[0086] In the embodiments of the application, the image processing device can use an image processing chip to upsample the third superimposed image according to the screen resolution to obtain an interface image with the same resolution as the screen.

[0087] In some embodiments of this application, since the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer, the resolution of the upsampled image is the same as that of the target system interface layer. Therefore, the resolution of the third overlay image obtained by superimposing the first upsampled image and the target system interface layer is also the same as that of the target system interface layer. Consequently, the resolution of the third overlay image is also less than the screen resolution. Thus, the image processing chip does not need to downsample the third overlay image; it can directly upsample the third overlay image to obtain an interface image with the same resolution as the screen.

[0088] In some embodiments of this application, as shown in FIG1, the image processing device overlays the first upsampled image with the target system interface layer using an image processing chip to obtain a third overlaid image. The image processing device then performs motion estimation and motion compensation on the third overlaid image using the MEMC 123 within the image processing chip 12, upsamples the third overlaid image using AISR 124 to obtain a 1440P resolution interface image, and adjusts the colors of the interface image using PQ 125. Finally, the image processing chip transmits the 1440P resolution interface image to the screen 23 for display via DSI TX 126.

[0089] Thus, the image processing device overlays the first overlay image and the target system interface layer using the image processing chip to obtain a third overlay image. Since the resolution of the third overlay image is the same as that of the target system interface layer, both being smaller than the screen resolution, when the image processing chip processes the third overlay image to generate the interface image according to the screen resolution, it is not necessary to downsample the third overlay image. Instead, the image processing chip directly upsamples the third overlay image to obtain an interface image with the same resolution as the screen, thereby improving the image quality of the interface image.

[0090] In some embodiments of this application, step 303a can be implemented by steps 303a1 to 303a2.

[0091] Step 303a1: The image processing device upsamples the first superimposed image using an image processing chip to obtain a first upsampled image.

[0092] In some embodiments of this application, the resolution of the first overlay image is lower than the resolution of the target system interface layer. Therefore, the image processing device needs to upsample the first overlay image using a DPU to increase the resolution of the first overlay image, so that the resolution of the obtained first upsampled image is consistent with the resolution of the target system interface layer, and the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer.

[0093] For example, if the resolution of the first overlay image is 810P and the resolution of the target system interface layer is 1080P, the image processing device can first upsample the 810P first overlay image through the DPU in the main control chip to obtain the 1080P first upsampled image.

[0094] Step 303a2: The image processing device overlays the first upsampled image with the target system interface layer through the image processing chip to obtain a third overlaid image.

[0095] In some embodiments of this application, the image processing device can use a main control chip to superimpose a first pixel in a first upsampled image with a second pixel in a target system interface layer to obtain the pixel value of a pixel in a third superimposed image, thereby obtaining a third superimposed image. Here, the first pixel is a pixel in the first upsampled image, and the second pixel is the pixel in the target system interface layer corresponding to the first pixel.

[0096] In some embodiments of this application, both the first upsampled image and the target system interface layer include an R channel, a G channel, a B channel, and an Alpha channel. The Alpha channel is used to represent the transparency of a color. In the RGBA model, transparent color is represented by setting the values ​​of all three RGB channels to 0, and simultaneously setting the value of the Alpha channel to 0, indicating complete transparency.

[0097] In some embodiments of this application, the image processing device can superimpose the first pixel in the first upsampled image and the second pixel in the target system interface layer by performing a weighted operation through the main control chip.

[0098] For example, the image processing device can use the image processing chip to use the Alpha parameter value of the first pixel in the Alpha channel as the first weight of the second pixel, and the 1-Alpha parameter value as the second weight of the second pixel. Then, the first weight and the second weight are used to perform a weighted operation on the pixel value of the second pixel and the pixel value of the second target pixel on the target system interface layer to obtain the weighted operation result. The weighted operation result is the pixel value of a pixel in the third overlay image.

[0099] In one example, the image processing device can use an image processing chip to calculate a first weighted value of the R color value of a first pixel in the R channel and the R color value of a second pixel in the R channel, using a first weight and a second weight respectively; calculate a second weighted value of the G color value of a second pixel in the G channel and the G color value of a second pixel in the G channel, using the first weight and the second weight respectively; and calculate a third weighted value of the B color value of a second pixel in the B channel and the B color value of a second pixel in the B channel, using the first weight and the second weight respectively. Then, the first weighted value, the second weighted value, and the third weighted value are used as the R color value in the R channel, the G color value in the G channel, and the B color value in the B channel of a pixel in the third overlay image, respectively.

[0100] Thus, the resolution of the first upsampled image obtained by upsampling the first superimposed image from the image processing chip is consistent with the resolution of the target system interface layer. Therefore, the resolution of the third superimposed image obtained by superimposing the first upsampled image with the target system interface layer is also consistent with the resolution of the target system interface layer. Therefore, the image processing chip can generate a third superimposed image with a resolution lower than the screen resolution, thereby reducing the amount of data processing within the image processing chip and improving its image processing efficiency.

[0101] In some embodiments of this application, step 303 described above can be implemented by step 303c.

[0102] Step 303c: The image processing device processes the second superimposed image according to the screen resolution using an image processing chip to obtain an interface image.

[0103] In embodiments of this application, the target image includes a second overlay image processing step. The image processing device can upsample the second overlay image according to the screen resolution using an image processing chip to obtain an interface image with the same resolution as the screen.

[0104] In some embodiments of this application, as shown in FIG1, when the image processing chip 12 receives an image processing instruction through GPIO 122 and receives a second superimposed image through DSI RX 121, the image processing device can upsample the second superimposed image according to the screen resolution based on the image processing instruction through the image processing chip to obtain an interface image with the same resolution as the screen.

[0105] For example, when the resolution of the second overlay image is 1080P, the image processing chip 12 does not need to downsample the second overlay image. Instead, it upsamples the second overlay image using AISR 124 to obtain a 1440P resolution interface image, and then uses PQ 125 to adjust the colors of the interface image. Finally, the image processing chip transmits the 1440P resolution interface image to the screen for display via DSI TX 126.

[0106] Thus, since the resolution of the second superimposed image is smaller than the screen resolution, when the image processing device processes the second superimposed image according to the screen resolution using the image processing chip to obtain the interface image, the image processing chip does not need to downsample the second superimposed image, thereby improving the image quality of the interface image.

[0107] The image processing method provided in this application embodiment involves a main control chip in an image processing device controlling the resolution of the superimposed first superimposed image to be lower than the resolution of the target system interface layer corresponding to the first interface when superimposing the screen layer and the application interface layer. This ensures that the resolution of the target image transmitted by the main control chip to the image processing chip is lower than the screen resolution. Consequently, the image processing chip can process low-resolution target images, resulting in a smaller data volume when processing the target image. The image processing chip does not need to downsample or crop the target image, thus improving the image quality of the interface image.

[0108] In some embodiments of this application, when the target system interface layer includes a first system interface layer, the above step 301 can be implemented by the following step 301a.

[0109] Step 301a: The image processing device draws the first system interface layer according to the screen resolution using the main control chip.

[0110] In some embodiments of this application, as shown in FIG1, the main control chip 11 can draw a first system interface layer with a resolution lower than the screen resolution using system interface layer data according to the screen resolution.

[0111] For example, when the screen resolution is 1440P, as shown in Figure 1, the image processing device can use the main control chip 11 to draw a first system interface layer with a resolution of 1080P using system interface layer data.

[0112] It should be noted that this embodiment only provides an example of the screen resolution and the resolution of the first system interface layer. The screen resolution and the resolution of the first system interface layer can also be other values, and this embodiment does not impose any specific restrictions.

[0113] In this way, the image processing device can draw the first system interface layer according to the screen resolution through the main control chip, and can generate a first system interface layer with a resolution smaller than the screen resolution. When the interface image is obtained by using the first system interface layer, the amount of image data processing can be reduced and the image processing efficiency can be improved.

[0114] In this embodiment of the application, when the target system interface layer includes a second system interface layer, the above step 301 can be implemented by the following steps 301b1 to 301b3.

[0115] Step 301b1: Draw the first system interface layer using the main control chip at the screen resolution.

[0116] In this embodiment of the application, as shown in FIG1, the main control chip 11 can draw a first system interface layer with a resolution equal to the screen resolution using system interface layer data.

[0117] In the embodiments of this application, the image processing device can use the GPU 111 in the main control chip 11 to draw a first system interface layer with a resolution equal to the screen resolution using system interface layer data.

[0118] For example, when the screen resolution is 1440P, as shown in Figure 1, the image processing device can use the GPU 111 of the main control chip 11 to draw a first system interface layer with a resolution of 1440P using system interface layer data.

[0119] Step 301b2: The main control chip downsamples the first system interface layer to obtain the downsampled layer;

[0120] Step 301b3: Use the downsampling layer as the second system interface layer through the main control chip.

[0121] In some embodiments of this application, as shown in FIG1, the image processing device can use the GPU 111 within the main control chip 11 to downsample the first system interface layer according to the size to be reduced in the first system interface layer. Suitable pixels are selected from the first system interface layer, and the first system interface layer is reduced in size to obtain a downsampled layer. This downsampled layer retains the original image features of the first system interface layer, and then this downsampled layer is used as the second system interface layer. Here, equal-interval sampling and local averaging algorithms are used as examples to illustrate how the first system interface layer is downsampled.

[0122] In this embodiment, the image downsampling algorithm using equal-interval sampling selects pixels at regular intervals in the first system interface layer as pixels for the downsampling layer. While the equal-interval sampling downsampling method is simple to implement, unselected pixel information in the first system interface layer is lost in the downsampling layer.

[0123] In this embodiment, the local mean downsampling method is improved. When obtaining the pixels of the downsampling layer, it does not simply obtain the pixels of the sampling points in the first system interface layer, but divides the first system interface layer into sub-blocks by using two adjacent sampling points as the dividing point, and the pixels of the downsampling layer are taken as the mean of the pixels of the corresponding sub-block.

[0124] For example, as shown in Figure 1, the image processing device can use the GPU 111 in the main control chip 11 to downsample the first system interface layer with a resolution of 1440P to obtain a downsampled layer with a resolution of 1080P.

[0125] In one embodiment of this application, the image processing device uses the downsampling layer as the second system interface layer through the main control chip. Then, the main control chip overlays the second system interface layer with the first overlay image to obtain the second overlay image, and transmits the second overlay image to the image processing chip.

[0126] In this embodiment, the first system interface layer is downsampled by the main control chip to obtain a downsampled layer; the first system interface layer is downsampled by the main control chip to obtain a downsampled layer; the downsampled layer is used as the second system interface layer by the main control chip, which can generate a second system interface layer with a resolution smaller than the screen resolution. When the interface image is obtained using the second system interface layer, the amount of image data processing can be reduced and the image processing efficiency can be improved.

[0127] In some embodiments of this application, the step 302 above, "the image processing device obtains a first superimposed image based on the screen layer and the application interface layer through the main control chip", can be implemented by the following steps 302a to 302b.

[0128] Step 302a: The image processing device upsamples the image layer through the main control chip to obtain the upsampled layer.

[0129] In some embodiments of this application, as shown in FIG1, the image processing device upsamples the image layer through the GPU 111 in the main control chip 11 to obtain the upsampled layer.

[0130] In some embodiments of this application, the resolution of the image layer is lower than that of the application interface layer. Therefore, the image processing device needs to upsample the image layer using the GPU to increase its resolution, so that the resolution of the resulting upsampled layer is consistent with the resolution of the application interface layer.

[0131] For example, if the resolution of the screen layer is 720P and the resolution of the application interface layer is 810P, the image processing device can first upsample the 720P upsampled layer through the DPU in the main control chip to obtain the 810P upsampled layer.

[0132] In some embodiments of this application, the image processing device may use existing algorithms such as nearest neighbor interpolation, single linear interpolation, bilinear interpolation, and bicubic interpolation to upsample the image layer. This embodiment does not impose specific limitations here. For example, nearest neighbor interpolation assigns the gray value of the nearest neighboring pixel to the pixel to be determined among the four neighboring pixels of the pixel to be determined.

[0133] Step 302b: The image processing device overlays the upsampled layer and the application interface layer through the main control chip to obtain the first overlaid image.

[0134] In some embodiments of this application, as shown in FIG1, the image processing device overlays the upsampling layer and the application interface layer through the GPU 111 in the main control chip 11 to obtain a first overlaid image.

[0135] In some embodiments of this application, the upsampled image layer and the user interface layer have the same resolution; therefore, the pixels on the upsampled image layer correspond one-to-one with the pixels on the user interface layer. The image processing device can use the GPU to superimpose the third pixel in the upsampled image layer with the fourth pixel in the user interface layer to obtain the pixel value of a pixel in the first superimposed image, thus obtaining the first superimposed image. Here, the third pixel is a pixel in the upsampled image layer, and the fourth pixel is superimposed to the pixel in the user interface layer corresponding to the third pixel.

[0136] It should be noted that the specific generation process of the first superimposed image is similar to that of the third superimposed image, and will not be repeated here in this embodiment.

[0137] In this way, the image processing device upsamples the screen layer through the main control chip to obtain an upsampled layer and an application interface layer. The main control chip then overlays the upsampled layer and the application interface layer to obtain a first overlaid image. Therefore, the image processing chip can generate a first overlaid image with a resolution lower than the screen resolution, enabling it to process low-resolution images. On the one hand, this reduces the amount of data processing within the image processing chip, thereby improving its image processing efficiency. On the other hand, since the image processing chip processes a smaller amount of data when processing the first overlaid image, it does not need to downsample or crop the image, thus improving the image quality of the interface.

[0138] The image processing method provided in this application will be illustrated by specific embodiments below. For example, in a game interface display scenario, as shown in FIG4, in one embodiment, the image processing method may include steps 31 to 36.

[0139] Step 31: Control the GPU in the main control chip to draw the game screen layer1 (i.e. the above screen layer) (720P), the game UI (i.e. the above application interface layer) layer2 (810P), and the system UI layer3 (1440P) (i.e. the system interface layer).

[0140] In some embodiments of this application, the main control chip can obtain in real time the image data of the game screen layer1 corresponding to the first interface, the image data of the game UI, and the image data of the system UI layer3 obtained from the main interface of the electronic device screen.

[0141] Furthermore, the GPU within the main control chip will use the image data of game screen layer1 to draw a game screen layer1 with a resolution of 720P, use the image data of game UI to draw a game UI layer2 with a resolution of 810P, and use the system interface layer data to draw a system interface layer3 with a resolution of 1440P.

[0142] Step 32: Control the GPU to overlay the game screen layer1 (720P) and the game UI layer2 (810P) to generate the game image layer4 (i.e. the first overlaid image above) (810P).

[0143] The implementation process of step 32 is similar to that of step 302 above, and will not be repeated here.

[0144] Step 33: Control the GPU to redraw the system UI layer 3 (1080P).

[0145] In some embodiments of this application, UI layer 3 is generated by default according to the screen resolution of 1440P. After the GPU generates the game image layer 4 (810P), it will pass the game image layer 4 (810P) and the system UI layer 3 (1440P) to the DPU. The DPU receives layer 4 (810P), first scales it to 1440P, then overlays it with layer 3 (1440P) to generate layer (1440P), and finally sends it to the image processing chip through the DSI port.

[0146] When the user enables the dedicated graphics function, that is, when DPU pass-through is enabled, the main control chip is first instructed to redraw the system UI layer 3 (1080P) of the GPU. The GPU then draws the game content and the game UI layer 4 (810P) and sends it to the DPU.

[0147] Step 34: Control the DPU to upsample the game screen of layer 4 (810P) to obtain an upsampled image of up to 1080P.

[0148] In some embodiments of this application, the DPU receives the game screen layer4 (810P), performs upsampling, and obtains the upsampled game screen layer4 to enlarge the game screen to 1080P instead of 1440P.

[0149] It should be noted that the upsampling process for the game screen of layer 4 (810P) can refer to the above steps, and will not be repeated here.

[0150] Step 35: Control the DPU to overlay the upsampled game screen layer 4 with the system UI layer 3 (1080P) to obtain a 1080P image (i.e. the second overlaid image mentioned above), and transmit the 1080P image to the image processing chip.

[0151] The implementation process of step 35 is similar to that of step 302 above, and will not be repeated here.

[0152] Step 36: Control the image processing chip to upsample the 1080P image to obtain a 1440P image (i.e., the interface image mentioned above).

[0153] It should be noted that the upsampling process for 1080P images can be referenced from the upsampling process for the aforementioned image layers, and will not be repeated here.

[0154] In some embodiments of this application, after receiving a 1080P image via DSI RX, the image processing chip does not perform downsampling but directly sends it to MEMC. MEMC performs motion estimation and motion compensation through frame interpolation, and then upsampling is performed via AISR to improve the image resolution. The AISR output is a 1440P high-resolution image. In some embodiments of this application, AISR can utilize neural networks to achieve image upsampling. AISR processing of the image can improve the clarity of detailed images and texture details. Furthermore, PQ is used for color adjustment, and finally the 1440P image is transmitted to the screen for display via DSI TX inside the discrete graphics chip.

[0155] In some embodiments of this application, PQ can adjust the color of an image using a look-up table (LUT). In the film industry, due to differences in playback devices, mapping between different color spaces is necessary. 3D LUTs, a technology widely used in the film and display industries, are used for precise color correction. The principle of a 3D LUT is to combine three 1D LUTs (RGB) into a 3D LUT. The input RGB color values ​​are mapped according to the three look-up tables of the 3D LUT to obtain the converted color.

[0156] In this way, by controlling the DPU to merge all layers into a single image and scaling it to the resolution supported by the image processing chip (such as 1080P), the image processing chip no longer downsamples high-resolution images. This avoids the image quality loss caused by downsampling of discrete graphics chips, while also reducing the bandwidth of data transmission from the main control chip to the image processing chip and effectively reducing power consumption.

[0157] The image processing method provided in this application will be exemplarily described below through a specific embodiment two. For example, referring to FIG5, the image processing method may include steps 31 to 35.

[0158] Step 31: Control the GPU in the main control chip to draw the game screen layer1 (i.e. the above screen layer) (720P), the game UI (i.e. the above application interface layer) layer2 (810P), and the system UI layer3 (1440P) (i.e. the system interface layer).

[0159] Step 32: Control the GPU to overlay the game screen layer1 (720P) and the game UI layer2 (810P) to generate the game image layer4 (i.e. the first overlaid image above) (810P).

[0160] Step 33: Control the GPU to redraw the system UI layer 3 (1080P).

[0161] In some embodiments of this application, UI layer 3 is generated by default according to the screen resolution of 1440P. After the GPU generates the game image layer 4 (810P), it will pass the game image layer 4 (810P) and the system UI layer 3 (1440P) to the DPU. The DPU receives layer 4 (810P), first scales it to 1440P, and then overlays it with layer 3 (1440P) to generate layer (1440P), which is finally sent to the discrete graphics chip via the DSI TX port. When the user enables the discrete graphics function, that is, the GPU pass-through is enabled. First, an instruction is sent to the main control chip to notify the GPU to redraw the system UI layer 3 (1080P).

[0162] Step 34: Control the GPU to transfer the game image layer4 (810P) and system UI layer3 (1080P) to the DPU112, and control the DPU112 to transfer the game image layer4 (810P) and system UI layer3 (1080P) to the image processing chip 12.

[0163] In some embodiments of this application, after the DPU receives the game image layer4 (810P) and UI layer3 (1080P), it does not perform layer blending and scaling operations, but instead sends the game image layer4 (810P) and system UI layer3 (1080P) respectively through the two DSI interfaces in the main control chip.

[0164] Step 35: Control the image processing chip to overlay the game image layer4 (810P) and the system UI layer3 (1080P) to obtain a third overlaid image (1080P), and upsample the third overlaid image (1080P) to a 1440P image.

[0165] Accordingly, the graphics processing chip needs to be configured with an extra DSI RX port in its hardware architecture to receive the game image layer 4 (810P) and the system UI layer 3 (1080P) respectively. For example, as shown in Figure 5, the discrete graphics chip receives the game image layer 4 (810P) through DSI RX_0 and receives the system UI image layer 3 (1080P) through DSI RX_1.

[0166] In some embodiments of this application, by configuring two DSI RX channels in the image processing chip, not only can the game screen content be transmitted, but also information such as game motion vectors and depth can be transmitted to achieve better AISR and MEMC effects, thereby improving the image quality of the interface image.

[0167] Furthermore, the game image layer 4 (810P) is input into the MEMC, where motion estimation and compensation are performed through frame interpolation. Then, the AISR overlays the game image layer 4 (810P) and the system UI layer 3 (1080P) to obtain a third overlaid image (1080P). This third overlaid image (1080P) is then upsampled to a resolution of 1440P, and color adjustment is performed using PQ. Finally, the 1440P image is transmitted to the screen for display via the DSI TX within the discrete graphics chip.

[0168] In this way, by controlling the DPU to transmit game images and system UI separately, the image processing chip can process the game's native content independently. This allows for the configuration of larger neural networks with controllable hardware costs and power consumption when using AISR for layer processing, resulting in clearer image quality. In addition, MEMC can interpolate only game content, supporting frame interpolation even when there are system windows or other system UI elements. It also separates game content from system content, preventing them from interfering with each other, thereby improving image processing efficiency.

[0169] The above-described method embodiments, or various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0170] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0171] Figure 6 is a schematic diagram of the image processing device provided in an embodiment of this application. The image processing device includes a main control chip 601 and an image processing chip 602. The main control chip 601, based on the interface data of a first interface, acquires a screen layer, a user interface layer, and a target system interface layer corresponding to the first interface. The resolution of the target system interface layer is less than the resolution of the screen of the electronic device. The main control chip 601 is also used to obtain a first overlay image based on the screen layer and the application interface layer, and transmit the target image to the image processing chip. The resolution of the first overlay image is less than the resolution of the target system interface layer. The target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer. The second overlay image is an overlay image obtained by the main control chip based on the first overlay image and the target system interface layer. The image processing chip is used to process the target image to obtain an interface image corresponding to the first interface. The resolution of the interface image is equal to the resolution of the screen.

[0172] In some embodiments of this application, the main control chip 601 is specifically used to: draw a first system interface layer according to the screen resolution; the target system interface layer includes the first system interface layer.

[0173] In some embodiments of this application, the main control chip 601 is specifically used for: drawing a first system interface layer at the screen resolution; downsampling the first system interface layer through the main control chip to obtain a downsampled layer; using the downsampled layer as a second system interface layer through the main control chip; and the target system interface layer includes the second system interface layer.

[0174] In some embodiments of this application, the target image includes a first overlay image and a target system interface layer; the image processing chip 602 is further specifically used to: obtain a third overlay image based on the first overlay image and the target system interface layer; and process the third overlay image according to the screen resolution using the image processing chip to obtain an interface image.

[0175] In some embodiments of this application, the image processing chip 602 is further specifically used to: upsample the first overlay image to obtain a first upsampled image, wherein the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer; and overlay the first upsampled image and the target system interface layer by the image processing chip to obtain the third overlay image.

[0176] In some embodiments of this application, the image processing chip 602 is also specifically used to: process the second superimposed image according to the screen resolution to obtain an interface image.

[0177] The image processing apparatus provided in this application embodiment obtains a screen layer, a user interface layer, and a target system interface layer corresponding to the first interface based on interface data of the first interface by a main control chip. The resolution of the target system interface layer is less than the resolution of the screen of the electronic device. The main control chip obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to the image processing chip. The resolution of the first overlay image is less than the resolution of the target system interface layer. The target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer. The second overlay image is the overlay image obtained by the main control chip based on the first overlay image and the target system interface layer. The image processing chip processes the target image to obtain an interface image corresponding to the first interface, and the resolution of the interface image is equal to the resolution of the screen. In this solution, when the main control chip in the electronic device overlays the screen layer and the application interface layer corresponding to the first interface, it controls the resolution of the overlay image to be less than the resolution of the target system interface layer corresponding to the first interface, so that the resolution of the target image transmitted by the main control chip to the image processing chip is less than the resolution of the screen. This enables the image processing chip to process low-resolution target images. Therefore, the image processing chip processes target images with a smaller data volume, and does not need to downsample or crop the target image, thereby improving the image quality of the interface image.

[0178] The image processing device in this application embodiment can be an electronic device or a component on an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit the device.

[0179] The image processing apparatus in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0180] The image processing apparatus provided in this application embodiment can implement the various processes implemented in the method embodiments of Figures 3 to 5. To avoid repetition, these processes will not be described again here.

[0181] The image processing method provided in this application can be executed by an image processing system. This application uses an image processing system executing the image processing method as an example to illustrate the image processing system provided in this application.

[0182] Figure 7 is a schematic diagram of the structure of the image processing system provided in the embodiment of this application. The image processing device includes a main control chip 701 and an image processing chip 702. The main control chip 701 includes an image processor (GPU) 7011 and a display processor (DPU) 7012. The DPU 7011 is connected to both the GPU 7012 and the image processing chip 702. The GPU 7011 is used to obtain a screen layer, a user interface layer, and a target system interface layer corresponding to the first interface based on the interface data of the first interface. The resolution of the target system interface layer is less than the screen resolution. The GPU 7011 is also used to obtain a first overlay image based on the screen layer and the application interface layer, and transmit the target image to the DPU 7012. The DPU 7012 is used to transmit the target image to the image processing chip. The resolution of the first overlay image is less than the resolution of the target system interface layer. The target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer. The second overlay image is an overlay image obtained by the DPU 7012 based on the first overlay image and the target system interface layer. The image processing chip 702 is used to process the target image to obtain an interface image corresponding to the first interface. The resolution of the interface image is equal to the screen resolution.

[0183] In some embodiments of this application, GPU 7011 is specifically used to: draw a first system interface layer according to the screen resolution; wherein, the target system interface layer is the first system interface layer.

[0184] In some embodiments of this application, the GPU 7011 is specifically used to: draw a first system interface layer at the screen resolution; downsample the first system interface layer to obtain a downsampled layer; and use the downsampled layer as a second system interface layer; wherein the target system interface layer is the second system interface layer.

[0185] In some embodiments of this application, the target image includes the first overlay image and the target system interface layer; the image processing chip 702 is further specifically used to: obtain a third overlay image based on the first overlay image and the target system interface; and process the third overlay image according to the screen resolution to obtain the interface image.

[0186] In some embodiments of this application, the image processing chip 702 is specifically used to: upsample the first overlay image to obtain a first upsampled image, wherein the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer; and overlay the first upsampled image and the target system interface layer to obtain a third overlay image.

[0187] In some embodiments of this application, the image processing chip 702 is specifically used to: process the second superimposed image according to the screen resolution to obtain an interface image.

[0188] The image processing system provided in this application embodiment obtains a screen layer, a user interface layer, and a target system interface layer corresponding to the first interface based on the interface data of the first interface using a GPU. The resolution of the target system interface layer is less than the resolution of the electronic device's screen. The GPU obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to a DPU. The DPU transmits the target image to an image processing chip. The resolution of the first overlay image is less than the resolution of the target system interface layer. The target image includes a second overlay image, or the first overlay image and the target system interface layer. The second overlay image is an overlay image obtained by a main control chip based on the first overlay image and the target system interface layer. The image processing chip processes the target image to obtain an interface image corresponding to the first interface, the resolution of which is equal to the screen resolution. In this solution, when the GPU in the image processing system overlays the screen layer and the application interface layer corresponding to the first interface, it controls the resolution of the overlay image to be less than the resolution of the target system interface layer corresponding to the first interface, so that the resolution of the target image transmitted by the DPU to the image processing chip is less than the screen resolution. This enables the image processing chip to process low-resolution target images. Therefore, the image processing chip processes target images with a smaller data volume, and does not need to downsample or crop the target image, thereby improving the image quality of the interface image.

[0189] The image processing system provided in this application embodiment can implement the various processes implemented in the method embodiments of Figures 3 to 5. To avoid repetition, these processes will not be described again here.

[0190] Optionally, as shown in FIG8, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0191] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0192] Figure 9 is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of this application.

[0193] The electronic device 900 includes, but is not limited to, components such as: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910. The electronic device also includes a main control chip and an image processing chip.

[0194] Those skilled in the art will understand that the electronic device 900 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 910 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The electronic device structure shown in Figure 9 does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0195] The processor 910 is configured to: acquire, via the main control chip, the screen layer, user interface layer, and target system interface layer corresponding to the first interface based on the interface data of the first interface; obtain a first overlay image via the main control chip based on the screen layer and the application interface layer; and transmit the target image to the image processing chip; the resolution of the first overlay image is less than the resolution of the target system interface layer; the target image includes a second overlay image, or the first overlay image and the target system interface layer; the second overlay image is the overlay image obtained by the main control chip based on the first overlay image and the target system interface layer; and process the target image via the image processing chip to obtain an interface image corresponding to the first interface, the resolution of which is equal to the resolution of the screen.

[0196] In some embodiments of this application, the processor 910 is specifically used to: draw a first system interface layer according to the screen resolution using the main control chip; the target system interface layer includes the first system interface layer.

[0197] In some embodiments of this application, the processor 910 is specifically configured to: draw a first system interface layer at the screen resolution using a main control chip; downsample the first system interface layer using the main control chip to obtain a downsampled layer; use the downsampled layer as a second system interface layer using the main control chip; and the target system interface layer includes the second system interface layer.

[0198] In some embodiments of this application, the target image includes a first overlay image and a target system interface layer; the processor 910 is specifically used to: obtain a third overlay image by overlaying the first overlay image and the target system interface layer using an image processing chip; and process the third overlay image according to the screen resolution using an image processing chip to obtain an interface image.

[0199] In some embodiments of this application, the processor 910 is specifically used to: upsample the first overlay image using an image processing chip to obtain a first upsampled image, wherein the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer; and overlay the first upsampled image and the target system interface layer using an image processing chip to obtain the third overlay image.

[0200] In some embodiments of this application, the processor 910 is specifically used to: process the second superimposed image according to the resolution of the screen using an image processing chip to obtain an interface image.

[0201] The electronic device provided in this application embodiment obtains a screen layer, a user interface layer, and a target system interface layer corresponding to the first interface based on interface data of the first interface through a main control chip. The resolution of the target system interface layer is smaller than the resolution of the electronic device's screen. The main control chip obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to an image processing chip. The resolution of the first overlay image is smaller than the resolution of the target system interface layer. The target image includes a second overlay image, or the first overlay image and the target system interface layer. The second overlay image is the overlay image obtained by the main control chip based on the first overlay image and the target system interface layer. The image processing chip processes the target image to obtain an interface image corresponding to the first interface, and the resolution of the interface image is equal to the resolution of the screen. In this solution, when the main control chip in the electronic device overlays the screen layer and the application interface layer corresponding to the first interface, it controls the resolution of the overlay image to be smaller than the resolution of the target system interface layer corresponding to the first interface, so that the resolution of the target image transmitted by the main control chip to the image processing chip is smaller than the screen resolution. This enables the image processing chip to process low-resolution target images. Therefore, the image processing chip processes target images with a smaller data volume, and does not need to downsample or crop the target image, thereby improving the image quality of the interface image.

[0202] It should be understood that, in this embodiment, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042. The GPU 9041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0203] The memory 909 can be used to store software programs and various data. The memory 909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 909 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0204] Processor 910 may include one or more processing units; optionally, processor 910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, UI, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 910.

[0205] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0206] The processor is the processor on the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0207] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0208] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0209] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0210] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0211] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0212] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method applied to an image processing device, the image processing device comprising a main control chip and an image processing chip, comprising: The main control chip obtains the screen layer, user interface layer and target system interface layer corresponding to the first interface based on the interface data of the first interface, wherein the resolution of the target system interface layer is smaller than the screen resolution. The main control chip obtains a first overlay image based on the screen layer and the application interface layer, and transmits the target image to the image processing chip; the resolution of the first overlay image is less than the resolution of the target system interface layer; the target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer; the second overlay image is the overlay image obtained by the main control chip based on the first overlay image and the target system interface layer; The target image is processed by the image processing chip to obtain an interface image corresponding to the first interface, wherein the resolution of the interface image is equal to the resolution of the screen.

2. The method according to claim 1, wherein, The main control chip obtains the target system interface layer based on the interface data of the first interface, including: The main control chip draws the first system interface layer according to the screen resolution; The target system interface layer is the first system interface layer.

3. The method according to claim 1, wherein, The target system interface layer is obtained through the main control chip, including: The main control chip draws the first system interface layer at the resolution of the screen. The main control chip is used to downsample the first system interface layer to obtain a downsampled layer. The main control chip uses the downsampling layer as the second system interface layer. The target system interface layer is the second system interface layer.

4. The method according to any one of claims 1 to 3, wherein, The target image includes the first overlaid image and the target system interface layer; the processing of the target image to obtain the interface image corresponding to the first interface includes: The image processing chip generates a third overlay image based on the first overlay image and the target system interface. The image processing chip processes the third overlay image according to the screen resolution to obtain the interface image.

5. The method according to claim 4, wherein, The step of obtaining a third overlay image based on the first overlay image and the target system interface through the image processing chip includes: The first superimposed image is upsampled by the image processing chip to obtain a first upsampled image, and the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer. The image processing chip overlays the first upsampled image with the target system interface layer to obtain the third overlaid image.

6. The method according to claim 1, wherein, The process of processing the target image to obtain the interface image corresponding to the first interface includes: The second overlay image is processed by the image processing chip according to the screen resolution to obtain the interface image.

7. An image processing apparatus, the image processing apparatus comprising a main control chip and an image processing chip; The main control chip is used to obtain the screen layer, user interface layer and target system interface layer corresponding to the first interface based on the interface data of the first interface, wherein the resolution of the target system interface layer is smaller than the resolution of the screen of the electronic device. The main control chip is further configured to obtain a first overlay image based on the screen layer and the application interface layer, and transmit the target image to the image processing chip; the resolution of the first overlay image is less than the resolution of the target system interface layer; the target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer; the second overlay image is an overlay image obtained by the main control chip based on the first overlay image and the target system interface layer; The image processing chip is used to process the target image to obtain an interface image corresponding to the first interface, wherein the resolution of the interface image is equal to the resolution of the screen.

8. An image processing system, comprising a main control chip and an image processing chip, wherein the main control chip includes an image processor (GPU) and a display processor (DPU), and the DPU is connected to the GPU and the image processing chip respectively; The GPU is used to obtain the screen layer, user interface layer and target system interface layer corresponding to the first interface based on the interface data of the first interface, wherein the resolution of the target system interface layer is smaller than the screen resolution. The GPU is also used to obtain a first overlay image based on the screen layer and the application interface layer, and to transmit the target image to the DPU; The DPU is used to transmit the target image to the image processing chip; the resolution of the first overlay image is smaller than the resolution of the target system interface layer; the target image includes a second overlay image, or the target image includes the first overlay image and the target system interface layer; the second overlay image is an overlay image obtained by the DPU based on the first overlay image and the target system interface layer; The image processing chip is used to process the target image to obtain an interface image corresponding to the first interface, wherein the resolution of the interface image is equal to the resolution of the screen.

9. The system according to claim 8, wherein, The GPU is specifically used for: Draw the first system interface layer according to the screen resolution; The target system interface layer is the first system interface layer.

10. The system according to claim 8, wherein, The GPU is specifically used for: Draw the first system interface layer according to the resolution of the screen; The first system interface layer is downsampled to obtain a downsampled layer; The downsampling layer is used as the second system interface layer; The target system interface layer is the second system interface layer.

11. The system according to any one of claims 8 to 10, wherein, The target image includes the first overlaid image and the target system interface layer; the image processing chip is further specifically used for: A third overlay image is obtained based on the first overlay image and the target system interface; The interface image is obtained by processing the third overlay image according to the screen resolution.

12. The system according to claim 11, wherein, The image processing chip is specifically used for: The first overlay image is upsampled to obtain a first upsampled image, and the pixels of the first upsampled image correspond one-to-one with the pixels of the target system interface layer. The first upsampled image is superimposed on the target system interface layer to obtain the third superimposed image.

13. The system according to claim 8, wherein, The image processing chip is specifically used for: The second overlay image is processed according to the screen resolution to obtain the interface image.

14. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the image processing method as claimed in any one of claims 1 to 6.

15. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the image processing method as claimed in any one of claims 1 to 6.

16. A chip comprising a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the image processing method according to any one of claims 1 to 6.

17. A computer program product stored in a storage medium, the program product being executed by at least one processor to implement the image processing method as claimed in any one of claims 1 to 6.

Images