Image processing circuit, image processing method, and electronic device

Abstract

The application discloses an image processing circuit, an image processing method and an electronic device. The image processing circuit comprises an application processing chip and an image processing chip. The image processing chip comprises a noise reduction processing module and a super-resolution processing module. The noise reduction processing module is used for noise reduction processing, and the super-resolution processing module is used for super-resolution processing. The noise reduction processing module and the super-resolution processing module are connected. The image processing chip is used for processing an image output by the application processing chip under the control of the application processing chip.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image processing circuit, an image processing method and an electronic device. BACKGROUND

[0002] Light tracing is an important technical development trend in the game industry in recent years. Light tracing technology can create realistic and moving gloss and lighting effects, and improve the game experience of players. At present, game console platforms and personal computer platforms both support a large number of games with light tracing technology. Light tracing has a higher performance requirement for the GPU (Graphics processing unit, image processing unit) in an application processing chip. If the light tracing function is to be implemented on a mobile platform such as a mobile phone, the power consumption and performance of the GPU face great challenges. SUMMARY

[0003] The purpose of the embodiments of the application is to provide an image processing circuit, which can reduce the rendering power consumption of the GPU in an application processing chip and improve the rendering performance.

[0004] In a first aspect, the embodiments of the application provide an image processing circuit, characterized in that the image processing circuit comprises an application processing chip and an image processing chip, the image processing chip comprises a noise reduction processing module and a super-resolution processing module, the noise reduction processing module is configured to perform noise reduction processing, and the super-resolution processing module is configured to perform super-resolution processing.

[0005] The noise reduction processing module and the super-resolution processing module are connected.

[0006] The image processing chip is configured to process an image output by the application processing chip under the control of the application processing chip.

[0007] In a second aspect, the embodiments of the application provide an image processing method applied to the image processing circuit of the first aspect, and the method comprises the following steps.

[0008] The application processing chip renders an original image according to a first parameter and / or a second parameter, and transmits a rendered image to the image processing chip, wherein the first parameter is the number of light rays, and the second parameter is the resolution.

[0009] The application processing chip determines whether to control the image processing chip to perform noise reduction processing and / or super-resolution processing according to the first parameter and / or the second parameter.

[0010] The image processing chip processes an image output by the application processing chip under the control of the application processing chip.

[0011] In this embodiment, the image processing circuit includes an application processing chip and an image processing chip. The image processing chip includes a noise reduction module and a super-resolution processing module. The noise reduction module performs noise reduction processing, and the super-resolution processing module performs super-resolution processing. The noise reduction module and the super-resolution processing module are connected. The image processing chip processes the image output by the application processing chip under the control of the application processing chip. In other words, through this embodiment, for example, when the electronic device has ray tracing enabled, the application processing chip performs simple image rendering, and the image processing chip, under the control of the application processing chip, processes the image output by the application processing chip, such as performing noise reduction processing and / or super-resolution processing. This effectively reduces the power consumption of the application processing chip and improves the image rendering effect. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the image processing circuit provided in an embodiment of this application;

[0013] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 1 ;

[0014] Figure 3 This is a flowchart illustrating an example of an image processing method according to an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 ;

[0016] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 3 ;

[0017] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 4 ;

[0018] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 5 ;

[0019] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 6 ;

[0020] Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 7 ;

[0021] Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 8;

[0022] Figure 11 is a flowchart of an image processing method provided by an embodiment of the present application;

[0023] Figure 12 is a structural diagram of an electronic device provided by an embodiment of the present application Figure 9 . DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0025] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be exchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", and the like are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0026] The image processing circuit provided by the embodiments of the present application will be described in detail below in combination with the drawings, through specific embodiments and their application scenarios.

[0027] Please refer to Figure 1 , which is an image processing circuit provided by an embodiment of the present application, the image processing circuit includes an application processing chip 10 and an image processing chip 20, the image processing chip 20 includes a noise reduction processing module 210 and a super resolution processing module 220, the noise reduction processing module 210 is used for noise reduction processing, and the super resolution processing module 220 is used for super resolution processing; the noise reduction processing module 210 and the super resolution processing module 220 are connected.

[0028] Reference Figure 2The CPU of the application processing chip 10 is connected with the GPU of the application processing chip 10, and the GPU of the application processing chip 10 is connected with the second output interface of the application processing chip 10. The CPU of the application processing chip 10 generates a raw image, renders the raw image through the GPU of the application processing chip 10, and sends the rendered image to the image processing chip 20 through the second output interface of the application processing chip 10. It can be understood that in the ray tracing scene, the GPU can render the raw image based on the number of rays and / or resolution.

[0029] The noise reduction processing module 210 can be an artificial intelligence noise reduction module (AINR). When the GPU of the application processing chip 10 outputs an image rendered with a low number of rays, for example, less than 20 rays, there is usually a large amount of noise in the image. The noise reduction processing module 210 can effectively remove the noise in the image.

[0030] The super resolution processing module 220 can be an artificial intelligence super resolution processing module (AISR). When the GPU of the application processing chip 10 outputs an image rendered with a low resolution, the image is usually blurred. The super resolution processing module 220 can improve the resolution of the image.

[0031] The image processing chip 20 is configured to process the image output by the application processing chip 10 under the control of the application processing chip 10.

[0032] In this embodiment, referring to Figure 2 The CPU of the application processing chip 10 is connected with the GPU of the application processing chip 10, and the GPU of the application processing chip 10 is connected with the second output interface of the application processing chip 10. The CPU generates a raw image and sends it to the GPU. The GPU renders the raw image, for example, based on ray data and / or resolution, and sends the rendered image and a control signal to the image processing chip 20 through the second output interface of the application processing chip 10. The control signal is used to instruct the image processing chip 20 to start the noise reduction processing module 210 and / or the super resolution processing module 220.

[0033] Referring to Figure 2The second input interface 230 of the image processing chip 20 is connected with the second output interface of the application processor 10. The rendered image and the control signal are received through the second input interface 230 of the image processing chip 20, and the required noise reduction processing module 210 and / or super resolution processing module 220 are determined according to the control signal, so as to perform noise reduction processing on the rendered image through the noise reduction processing module 210 and / or perform super resolution processing on the rendered image through the super resolution processing module 220, thereby reducing the power consumption of the GPU in the application processing chip 10 and improving the image rendering effect.

[0034] In the embodiment of the present application, the image processing circuit includes an application processing chip and an image processing chip, the image processing chip includes a noise reduction processing module and a super resolution processing module, the noise reduction processing module is used for noise reduction processing, the super resolution processing module is used for super resolution processing, the noise reduction processing module and the super resolution processing module are connected, and the image processing chip is used for processing the image output by the application processing chip under the control of the application processing chip. That is, through the embodiment of the present application, when the electronic device starts the light ray tracking function, the application processing chip performs simple rendering on the image, the image processing chip can process the image output by the application processing chip under the control of the application processing chip, for example, perform noise reduction processing and / or super resolution processing on the image output by the application processing chip, which can effectively reduce the power consumption of the application processing chip and improve the rendering effect of the image.

[0035] In one embodiment, the application processing chip 10 is configured to render the original image according to the first parameter and / or the second parameter, and transmit the rendered image to the image processing chip 20, wherein the first parameter is the number of light rays, and the second parameter is the resolution.

[0036] The application processing chip 10 is configured to determine whether to control the noise reduction processing module 210 to perform noise reduction processing and / or control the super resolution processing module 220 to perform super resolution processing according to the first parameter and / or the second parameter.

[0037] Exemplarily, with reference to Figure 2, the GPU of the application processing chip 10 renders the original image with a lower number of light rays and a lower resolution, and outputs the rendered image to the image processing chip 20 through the second output interface of the application processing chip 10. The second input interface 230 of the image processing chip 20 is connected with the noise reduction processing module 210, and the noise reduction processing module 210 is connected with the super-resolution processing module 220. The second input interface 230 of the image processing chip 20 receives the rendered image, and transmits the received rendered image to the noise reduction processing module 210. The noise reduction processing module 210 performs noise reduction processing on the rendered image, which can effectively remove the noise in the image. The noise reduction processing module 210 transmits the image after noise reduction processing to the super-resolution processing module 220, and the super-resolution processing module 220 performs super-resolution processing on the image after noise reduction processing to improve the image resolution.

[0038] On the one hand, since the light ray tracing function consumes a large amount of performance of the GPU of the application processing chip 10, in the embodiment, the GPU of the application processing chip 10 can render with a lower number of light rays and a lower resolution, and the rendering power consumption of the GPU is reduced. When the power consumption increment of the image processing chip 20 is less than the reduced rendering power consumption of the GPU of the application processing chip 10, the system power consumption is reduced.

[0039] On the other hand, since the light ray tracing function consumes a large amount of performance of the GPU of the application processing chip 10, based on the current scheduling strategy such as the temperature rise control strategy of the mobile phone system, when the light ray tracing function is turned on, the GPU of the application processing chip 10 directly performs high-resolution picture output, which is easy to be locked or have frame drop phenomenon, affecting the rendering performance. In the embodiment, since the GPU of the application processing chip 10 renders with a lower number of light rays, the introduced noise will be large, and the noise reduction processing module 210 can perform noise reduction processing on the image. When the GPU of the application processing chip 10 renders with a lower resolution, the image will be relatively blurred, and the super-resolution processing module 220 can perform super-resolution processing on the image to improve the image resolution, thereby improving the performance of the light ray tracing function.

[0040] It can be understood that when the GPU of the application processing chip 10 only renders the image with a lower number of light rays, only the noise reduction processing module 210 is needed to perform noise reduction processing on the image to remove the noise in the image. When the GPU of the application processing chip 10 only renders the image with a lower resolution, only the super-resolution processing module 220 is needed to perform super-resolution processing on the image to improve the image resolution.

[0041] In one embodiment, referring to Figure 2The image processing chip 20 further comprises an image storage module 250 connected with at least one of the noise reduction processing module 210 and the super resolution processing module 220.

[0042] The image storage module 250 is connected with the noise reduction processing module 210 and the super resolution processing module 220. Figure 2 The super resolution processing module 220 is configured to read the previous frame image after super resolution processing from the image storage module 250, perform super resolution processing on the current frame image based on the previous frame image after super resolution processing, and store the processed current frame image into the image storage module 250.

[0043] The noise reduction processing module 210 is configured to read the previous frame image after noise reduction processing or super resolution processing from the image storage module 250, perform noise reduction processing on the current frame image based on the previous frame image after noise reduction processing or super resolution processing.

[0044] The image storage module 250 is connected with the noise reduction processing module 210 and the super resolution processing module 220. Figure 2 The noise reduction processing module 210 is configured to read the previous frame image after super resolution processing from the image storage module 250, perform noise reduction processing on the current frame image based on the previous frame image after super resolution processing.

[0045] The image storage module 250 is connected with the noise reduction processing module 210. Figure 8 The noise reduction processing module 210 is configured to read the previous frame image after noise reduction processing from the image storage module 250, perform noise reduction processing on the current frame image based on the previous frame image after noise reduction processing, and store the processed current frame image into the image storage module 250.

[0046] It should be noted that the working mechanism of the image storage module 250 is a first-in first-out mechanism. After the current frame image is processed, the previous frame image after processing is deleted, and the processed current frame image is stored into the image storage module 250.

[0047] In one embodiment, referring to Figure 2 The image processing chip 20 further comprises a motion compensation processing module 260. The motion compensation processing module 260 is connected with at least one of the noise reduction processing module 210, the super-resolution processing module 220 and the image storage module 250.

[0048] The motion compensation processing module 260 can be a motion estimation and motion compensation module. The motion compensation processing module 260 is configured to read a processed previous frame image from the image storage module 250 and perform an interpolation processing on a current frame image based on the processed previous frame image. The processed previous frame image can be one of a super-resolution processed previous frame image and an interpolation processed previous frame image. In this embodiment, the interpolation processing is performed on the current frame image based on the processed previous frame image, that is, the motion trajectory of an object is estimated based on the relationship between two adjacent frames, and an intermediate frame is inserted to improve the video frame rate and the visual fluency.

[0049] Referring to Figure 2 The motion compensation processing module 260 is connected with the image storage module 250 and the super-resolution processing module 220, respectively. The motion compensation processing module 260 can obtain a super-resolution processed previous frame image from the image storage module 250 and perform an interpolation processing on a current frame image based on the super-resolution processed previous frame image.

[0050] Referring to Figure 7 The motion compensation processing module 260 is connected with the image storage module 250 and the noise reduction processing module 210, respectively. The motion compensation processing module 260 can obtain an interpolation processed previous frame image from the image storage module 250 and perform an interpolation processing on a current frame image based on the interpolation processed previous frame image, and store the interpolation processed current frame image into the image storage module 250.

[0051] In one embodiment, referring to Figure 2 The image processing chip 20 further comprises a layer combination module 270.

[0052] The layer combination module 270 is connected with the image storage module 250 and the super-resolution processing module 220, respectively. The layer combination module 270 can obtain a super-resolution processed previous frame image from the image storage module 250 and combine the super-resolution processed previous frame image with a super-resolution processed current frame image to obtain a combined image. Figure 2The layer combination module 270 is connected with the first input interface 240 of the image processing chip 20, so as to receive the user interface data output by the application processing chip 10 through the first input interface 240, and combine the user interface data and the processed image to obtain a combined image. The processed image includes one of the image after noise reduction processing, the image after super-resolution processing, and the image after frame interpolation processing. Here, the video picture layer data and the UI layer data can be superimposed by the layer combination module 270, so as to improve the display diversity of the video picture.

[0053] It can be understood that the layers required to be displayed by the display panel include a video picture layer and a UI layer. Here, the above user interface data is UI layer data, for example but not limited to including a barrage, a message, and a like button.

[0054] Referring to Figure 2 The layer combination module 270 is connected with the motion compensation processing module 260 and the first input interface 240 of the image processing chip 20. The application processing chip 10 sends the user interface data to the image processing chip 20 through the first output interface. The first input interface 240 of the image processing chip 20 receives the user interface data and inputs the received user interface data to the layer combination module 270. The layer combination module 270 combines the user interface data and the image after frame interpolation processing by the motion compensation processing module 260 to obtain a combined image.

[0055] Referring to Figure 5 The layer combination module 270 is connected with the super-resolution processing module 220 and the first input interface 240 of the image processing chip 20. The application processing chip 10 sends the user interface data to the image processing chip 20 through the first output interface. The first input interface 240 of the image processing chip 20 receives the user interface data and inputs the received user interface data to the layer combination module 270. The layer combination module 270 combines the user interface data and the image after super-resolution processing by the super-resolution processing module 220 to obtain a combined image.

[0056] Referring to Figure 8 The layer combination module 270 is connected with the noise reduction processing module 210 and the first input interface 240 of the image processing chip 20. The application processing chip 10 sends the user interface data to the image processing chip 20 through the first output interface. The first input interface 240 of the image processing chip 20 receives the user interface data and inputs the received user interface data to the layer combination module 270. The layer combination module 270 combines the user interface data and the image after noise reduction processing by the noise reduction processing module 210 to obtain a combined image.

[0057] In one embodiment, referring to Figure 2 , the image processing chip 20 further comprises a gamut control module 280.

[0058] The gamut control module 280 is connected with the layer combination module 270, and the gamut control module 280 is connected with the output interface 290 of the image processing chip 20 to control the display panel of the electronic device to display the combined image. Here, the gamut control module 280 outputs the combined image to the corresponding gamut of the display panel through the output interface 290 for display, ensuring the accuracy of the picture color.

[0059] Referring to Figure 2 , the layer combination module 270 combines the user interface data and the image processed by the motion compensation processing module 260, obtains a combined image, and sends the combined image to the gamut control module 280. The gamut control module 280 will send the combined image to the display panel of the electronic device through the output interface 290 for display.

[0060] Referring to Figure 5 , the layer combination module 270 combines the user interface data and the image processed by the super-resolution processing module 220, obtains a combined image, and sends the combined image to the gamut control module 280. The gamut control module 280 will send the combined image to the display panel of the electronic device through the output interface 290 for display.

[0061] Referring to Figure 8 , the layer combination module 270 combines the user interface data and the image processed by the noise reduction processing module 210, obtains a combined image, and sends the combined image to the gamut control module 280. The gamut control module 280 will send the combined image to the display panel of the electronic device through the output interface 290 for display.

[0062] In one embodiment, the image processing chip 20 comprises a controller (not shown in the figure). The controller is used to determine a target processing module in the noise reduction processing module 210, the super-resolution processing module 220, and the motion compensation processing module 260 and set the processing order between the target processing modules under the control of the application processing chip 10, and control the target processing modules to process the image output by the application processing chip 10 according to the processing order.

[0063] Application scenario 1: referring to Figure 2 , the GPU of the application processing chip 10 renders the original image with a lower number of light rays and a lower resolution, and the frame rate of the picture is lower. At this time, the image processing chip comprises a target function module, and the target function module comprises: a noise reduction processing module 210, a super-resolution processing module 220, and a motion compensation processing module 260.

[0064] The noise reduction processing module 210 is connected with the super-resolution processing module 220, the image storage module 250, and the second input interface 230 of the image processing chip 20 respectively, so as to receive the rendered image through the second input interface 230 of the image processing chip 20, perform noise reduction processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and send the image after noise reduction processing to the super-resolution processing module 220.

[0065] The super-resolution processing module 220 is connected with the image storage module 250 and the motion compensation processing module 260 respectively, so as to receive the image after noise reduction processing, perform super-resolution processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and send the image after super-resolution processing to the motion compensation processing module 260.

[0066] The motion compensation processing module 260 is connected with the image storage module 250 and the layer combination module 270 respectively, so as to receive the image after super-resolution processing, perform frame interpolation processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and send the image after frame interpolation processing to the layer combination module 270.

[0067] The layer combination module 270 is connected with the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively, so as to receive the user interface data through the first input interface 240 of the image processing chip 20, and combine the user interface data and the image after frame interpolation processing to obtain a combined image.

[0068] The color gamut control module 280 is connected with the output interface 290 of the image processing chip 20, so as to control the display panel of the electronic device to display the combined image.

[0069] In this scenario, since the GPU of the application processing chip 10 renders the image with a lower number of light rays and a lower resolution, the power consumption of the GPU is saved the most at this time.

[0070] Application scenario 2: refer to Figure 5 The GPU of the application processing chip 10 renders the original image with a lower number of light rays and a lower resolution, and the frame rate of the picture is lower. At this time, the target function modules include: the noise reduction processing module 210, the motion compensation processing module 260, and the super-resolution processing module 220.

[0071] The noise reduction processing module 210 is connected with the motion compensation processing module 260, the image storage module 250 and the second input interface 230 of the image processing chip 20 respectively, so as to receive the rendered image through the second input interface 230 of the image processing chip 20, perform noise reduction processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and send the image after noise reduction processing to the motion compensation processing module 260.

[0072] The motion compensation processing module 260 is connected with the super-resolution processing module 220 and the image storage module 250 respectively, receives the current frame image after noise reduction processing, performs frame interpolation processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after frame interpolation processing to the super-resolution processing module 220.

[0073] The super-resolution processing module 220 is connected with the image storage module 250 and the layer combination module 270 respectively, receives the image after frame interpolation processing, performs super-resolution processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after super-resolution processing to the layer combination module 270.

[0074] The layer combination module 270 is connected with the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively, receives the user interface data through the first input interface 240 of the image processing chip 20, and combines the user interface data and the image after super-resolution processing to obtain a combined image.

[0075] The color gamut control module 280 is connected with the output interface 290 of the image processing chip 20, so as to control the display panel of the electronic device to display the combined image.

[0076] In this scenario, the power consumption saving of the GPU is similar to that in the application scenario 1, but since the super-resolution processing module 220 operates later, it affects the operation of the noise reduction processing module 210 on the next frame image, thereby causing a large time delay.

[0077] Application scenario 3: refer to Figure 6 , the GPU of the application processing chip 10 renders the image with a lower number of light rays and a lower resolution, and the frame rate of the picture is higher. At this time, the target function modules include the noise reduction processing module 210 and the super-resolution processing module 220.

[0078] The noise reduction processing module 210 is connected to the super-resolution processing module 220, the image storage module 250, and the second input interface 230 of the image processing chip 20, respectively. It receives the rendered image through the second input interface 230 of the image processing chip 20, performs noise reduction processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the noise-reduced image to the super-resolution processing module 220.

[0079] The super-resolution processing module 220 is connected to the image storage module 250 and the layer combination module 270 respectively. The super-resolution processing module 220 receives the image after noise reduction processing, performs super-resolution processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the super-resolution processed image to the layer combination module 270.

[0080] The layer combination module 270 is connected to the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively. The layer combination module 270 receives user interface data through the first input interface 240 of the image processing chip 20, and combines the user interface data and the super-resolution processed image into layers to obtain a combined image.

[0081] The color gamut control module 280 is connected to the output interface 290 of the image processing chip 20 to control the display panel of the electronic device to display the combined image.

[0082] Application Scenario 4: Reference Figure 7 The GPU of the application processing chip 10 renders the original image at high resolution, but the frame rate is low. At this time, the target functional modules include: noise reduction processing module 210 and motion compensation processing module 260.

[0083] The noise reduction processing module 210 is connected to the motion compensation processing module 260, the image storage module 250, and the second input interface 230 of the image processing chip 20, respectively. It receives the rendered image through the second input interface 230 of the image processing chip 20, performs noise reduction processing on the current frame image based on the previous frame image after frame interpolation obtained from the image storage module 250, and sends the noise-reduced image to the motion compensation processing module 260.

[0084] The motion compensation processing module 260 is connected with the image storage module 250 and the layer combination module 270 respectively. The motion compensation processing module 260 receives the image after the noise reduction processing, performs the frame interpolation processing on the current frame image based on the previous frame image after the frame interpolation processing obtained from the image storage module 250, and sends the image after the frame interpolation processing to the layer combination module 270.

[0085] The layer combination module 270 is connected with the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively. The layer combination module 270 receives the user interface data through the first input interface 240 of the image processing chip 20, and combines the user interface data and the image after the frame interpolation processing to obtain a combined image.

[0086] The color gamut control module 280 is connected with the output interface 290 of the image processing chip 20, so as to control the display panel of the electronic device to display the combined image.

[0087] Application scenario 5: refer to Figure 8 The GPU of the application processing chip 10 renders the image at a high resolution, and the frame rate of the picture is high. At this time, the target function module includes the noise reduction processing module 210.

[0088] The noise reduction processing module 210 is connected with the layer combination module 270, the image storage module 250 and the second input interface 230 of the image processing chip 20 respectively. The noise reduction processing module 210 receives the rendered image through the second input interface 230 of the image processing chip 20, performs the noise reduction processing on the current frame image based on the previous frame image after the noise reduction processing obtained from the image storage module 250, and sends the image after the noise reduction processing to the layer combination module 270.

[0089] The layer combination module 270 is connected with the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively. The layer combination module 270 receives the user interface data through the first input interface 240 of the image processing chip 20, and combines the user interface data and the image after the noise reduction processing to obtain a combined image.

[0090] The color gamut control module 280 is connected with the output interface 290 of the image processing chip 20, so as to control the display panel of the electronic device to display the combined image.

[0091] Application scenario 6: refer to Figure 9 The GPU of the application processing chip 10 renders the image at a low resolution, and the frame rate of the picture is low. At this time, the target function module includes the motion compensation processing module 260 and the super resolution processing module 220.

[0092] The motion compensation processing module 260 is connected with the super-resolution processing module 220, the image storage module 250, and the second input interface 230 of the application processing chip 10 respectively. The motion compensation processing module 260 receives the image after rendering, performs frame interpolation processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after frame interpolation processing to the super-resolution processing module 220.

[0093] The super-resolution processing module 220 is connected with the image storage module 250 and the layer combination module 270 respectively. The super-resolution processing module 220 receives the image after frame interpolation processing, performs super-resolution processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after super-resolution processing to the layer combination module 270.

[0094] The layer combination module 270 is connected with the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively. The layer combination module 270 receives the user interface data through the first input interface 240 of the image processing chip 20, and performs layer combination on the user interface data and the image after super-resolution processing to obtain a combined image.

[0095] The color gamut control module 280 is connected with the output interface 290 of the image processing chip 20, so as to control the display panel of the electronic device to display the combined image.

[0096] Application scenario 7: refer to Figure 10 The GPU of the application processing chip 10 renders the image at a low resolution, and the frame rate of the picture is low. At this time, the target function module includes the motion compensation processing module 260 and the super-resolution processing module 220.

[0097] The super-resolution processing module 220 is connected with the motion compensation processing module 260, the image storage module 250, and the second input interface 230 of the application processing chip 10 respectively. The super-resolution processing module 220 receives the image after rendering, performs frame interpolation processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after frame interpolation processing to the motion compensation processing module 260.

[0098] The motion compensation processing module 260 is connected with the image storage module 250 and the layer combination module 270 respectively. The motion compensation processing module 260 receives the image after frame interpolation processing, performs frame interpolation processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after frame interpolation processing to the layer combination module 270.

[0099] The layer combination module 270 is connected with the color gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively, and the layer combination module 270 receives the user interface data through the first input interface 240 of the image processing chip 20, and combines the user interface data and the image processed by the super-resolution to obtain a combined image.

[0100] The color gamut control module 280 is connected with the output interface 290 of the image processing chip 20 to control the display panel of the electronic device to display the combined image.

[0101] Based on the above, the embodiment of the present application further provides an electronic device, which comprises the image processing circuit 10 of the first aspect.

[0102] In one embodiment, referring to Figure 2 , the electronic device further comprises a display panel 30, and the display panel 30 is connected with the output interface 290 of the image processing chip 20.

[0103] Please refer to Figure 11 , which is an image processing method provided by the embodiment of the present application, applied to the image processing circuit as described in any embodiment of the present application, which can include the following steps 1101-1103:

[0104] Step 1101, the application processing chip renders the original image according to the first parameter and / or the second parameter, and transmits the rendered image to the image processing chip, wherein the first parameter is the number of light rays, and the second parameter is the resolution.

[0105] Step 1102, the application processing chip determines whether to control the image processing chip to perform noise reduction processing and / or super-resolution processing according to the first parameter and / or the second parameter.

[0106] Step 1103, the image processing chip processes the image output by the application processing chip under the control of the application processing chip.

[0107] In this embodiment, the image processing chip in this step 1103 processes the image output by the application processing chip under the control of the application processing chip can further include: the image processing chip determines the target processing module from the noise reduction processing module, the super resolution processing module and the motion compensation processing module and sets the processing order between the target processing modules under the control of the application processing chip, and controls the target processing modules to process the image output by the application processing chip according to the processing order. Here, it can control only the corresponding modules of the image processing chip to work based on the image output by the application processing chip, and can further reduce the power consumption of the application processing chip.

[0108] Specifically, reference can be made to the above descriptions of the application scenarios 1 to 7, and no further description is made in this embodiment.

[0109] In the embodiments of the present application, the image processing circuit includes an application processing chip and an image processing chip, the image processing chip includes a noise reduction processing module and a super resolution processing module, wherein the noise reduction processing module is used for noise reduction processing, the super resolution processing module is used for super resolution processing, the noise reduction processing module and the super resolution processing module are connected, and the image processing chip is used for processing the image output by the application processing chip under the control of the application processing chip. That is, through the embodiments of the present application, when the light ray tracking function of the electronic device is turned on, the application processing chip performs simple rendering on the image, and the image processing chip can process the image output by the application processing chip under the control of the application processing chip, such as noise reduction processing and / or super resolution processing, which can effectively reduce the power consumption of the application processing chip and improve the rendering effect of the image.

[0110] In one embodiment, the image processing method of the embodiments of the present application further includes: the image processing chip receives the user interface data output by the application processing chip, combines the user interface data and the processed image to obtain a combined image, and controls the display panel to display the combined image.

[0111] The processed image includes one of the image processed by noise reduction, the image processed by super resolution, and the image processed by frame interpolation.

[0112] According to the present embodiment, the video picture layer data and the UI layer data can be superimposed by the layer combination module, and the display diversity of the video picture is improved.

[0113] Next, the processing process of the image processing chip 20 shown in Figure 2 and Figure 3 is described based on the application scenario of playing a non-full-screen video or a game (with a UI interface). Figure 2 ​

[0114] 1. When playing a video or a game, in the case that the game ray tracing function is turned on, the electronic device enters the ray tracing mode, the GPU of the application processing chip 10 renders the original image with a lower number of rays, for example, less than 20 rays, and a lower resolution, for example, 540P, and transmits the rendered image to the image processing chip 20 through the second output interface of the application processing chip 10. The second input interface 230 of the image processing chip 20 receives the rendered image. At the same time, the application processing chip 10 transmits the user interface data to the image processing chip 20 through the first output interface, and the first input interface 240 of the image processing chip 20 receives the user interface data.

[0115] Referring to Figure 3 In the case that the game ray tracing function is turned off, the GPU of the application processing chip 10 renders the original image with a preset number of rays and a preset resolution, and directly outputs the rendered image to the display panel 30 of the electronic device for display.

[0116] 2. The noise reduction processing module 210 is connected with the image storage module 250, the super-resolution processing module 220, and the second input interface 230 of the image processing chip 20, respectively. The noise reduction processing module 210 receives the rendered image, and performs noise reduction processing on the current frame image based on the previous frame image that has been subjected to super-resolution processing and is obtained from the image storage module 250, and sends the image that has been subjected to noise reduction processing to the super-resolution processing module 220.

[0117] 3. The super-resolution processing module 220 is connected with the image storage module 250 and the motion compensation processing module 260, respectively. The super-resolution processing module 220 receives the image that has been subjected to noise reduction processing, and performs super-resolution processing on the current frame image based on the previous frame image that has been subjected to super-resolution processing and is obtained from the image storage module 250, and sends the image that has been subjected to super-resolution processing to the motion compensation processing module 260.

[0118] 4. The motion compensation processing module 260 is connected with the image storage module 250 and the layer combination module 270, respectively. The motion compensation processing module 260 receives the image that has been subjected to super-resolution processing, and performs frame interpolation processing on the current frame image based on the previous frame image that has been subjected to super-resolution processing and is obtained from the image storage module 250, and sends the image that has been subjected to frame interpolation processing to the layer combination module 270.

[0119] 5. The layer combination module 270 is connected with the gamut control module 280 and the first input interface 240 of the image processing chip 20 respectively, the layer combination module 270 receives the user interface data, and combines the user interface data and the image after the interpolation processing into a combined image, and transmits the combined image to the gamut control module 280.

[0120] 6. The gamut control module 280 is connected with the output interface 290 of the image processing chip 20, to control the display panel 30 of the electronic device to display the combined image.

[0121] Next, the processing process of the image processing chip 20 is described based on the application scenario of playing a full-screen video or playing a game (without a UI interface) in combination with Figure 3 and Figure 4 . Figure 4

[0122] 1. When playing a video or playing a game, in the case that the game ray tracing function is turned on, the electronic device enters a ray tracing mode, the GPU of the application processing chip 10 renders the original image with a lower number of rays, for example, less than 20 rays, and a lower resolution, for example, 540P, and transmits the rendered image to the image processing chip 20 through the second output interface of the application processing chip 10. The second input interface 230 of the image processing chip 20 receives the rendered image.

[0123] Referring to Figure 3 , in the case that the game ray tracing function is turned off, the GPU of the application processing chip 10 renders the original image with a preset number of rays and a preset resolution, and directly outputs the rendered image to the display panel 30 of the electronic device for display.

[0124] 2. The noise reduction processing module 210 is connected with the image storage module 250, the super-resolution processing module 220, and the second input interface 230 of the image processing chip 20 respectively. The noise reduction processing module 210 receives the rendered image, and performs noise reduction processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after noise reduction processing to the super-resolution processing module 220.

[0125] 3. The super-resolution processing module 220 is connected with the image storage module 250 and the motion compensation processing module 260 respectively, the super-resolution processing module 220 receives the image after noise reduction processing, and performs super-resolution processing on the current frame image based on the previous frame image after super-resolution processing obtained from the image storage module 250, and sends the image after super-resolution processing to the motion compensation processing module 260.

[0126] ​4. The motion compensation processing module 260 is connected with the image storage module 250 and the layer combination module 270 respectively. The motion compensation processing module 260 receives the image after the super-resolution processing, performs the interpolation processing on the current frame image based on the previous frame image after the super-resolution processing obtained from the image storage module 250, and sends the image after the interpolation processing to the gamut control module 280.

[0127] 5. The gamut control module 280 is connected with the output interface 290 of the image processing chip 20, to control the display panel 30 of the electronic device to display the combined image.

[0128] The image processing circuit in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than the terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The electronic device can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiment of the present application is not limited in this regard.

[0129] The electronic device in the embodiment of the present application can be a device with an operating system. The operating system can be an Android operating system, an ios operating system, or other possible operating systems. The embodiment of the present application is not limited in this regard.

[0130] The electronic device provided in the embodiment of the present application can implement the method Figure 11 The method embodiment implements various processes, which will not be repeated here to avoid repetition.

[0131] Figure 12 A hardware structure diagram of an electronic device according to an embodiment of the present application.

[0132] The electronic device 1000 includes, but is not limited to, a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and an image processing chip, etc.

[0133] Those skilled in the art can understand that the electronic device 1000 can also include a power supply (such as a battery) for powering various components, and the power supply can be logically connected to the processor 1010 through a power management system, so that the power management system can realize functions such as management of charging, discharging, and power consumption management. Figure 12 The electronic device structure shown in the figure is not a limitation on the electronic device, and the electronic device can include more or fewer components than shown, or combine certain components, or different component arrangements, which will not be described here.

[0134] The processor 1010 is configured to render the original image according to the first parameter and / or the second parameter, and transmit the rendered image to the image processing chip, wherein the first parameter is the number of light rays, and the second parameter is the resolution.

[0135] The processor 1010 is configured to render the original image according to the first parameter and / or the second parameter, and transmit the rendered image to the image processing chip, wherein the first parameter is the number of light rays, and the second parameter is the resolution.

[0136] The image processing chip is configured to process the image output by the processor under the control of the processor.

[0137] According to the embodiments of the present application, the image processing circuit includes a processor (application processing chip) and an image processing chip, the image processing chip includes a noise reduction processing module and a super-resolution processing module, wherein the noise reduction processing module is configured to perform noise reduction processing, the super-resolution processing module is configured to perform super-resolution processing, the noise reduction processing module and the super-resolution processing module are connected, and the image processing chip is configured to process the image output by the processor under the control of the processor. That is, through the embodiments of the present application, for example, when the electronic device starts the light ray tracking function, the processor performs simple rendering on the image, and the image processing chip can process the image output by the processor under the control of the processor, for example, perform noise reduction processing and / or super-resolution processing on the image output by the processor, which can effectively reduce the power consumption of the processor and improve the rendering effect of the image.

[0138] In one embodiment, the image processing chip is configured to determine a target processing module from the noise reduction processing module, the super resolution processing module and the motion compensation processing module and set a processing sequence between the target processing modules under the control of the processor, and control the target processing modules to process the image output by the processor according to the processing sequence.

[0139] In one embodiment, the image processing chip receives user interface data output by the processor, combines the user interface data and the processed image to obtain a combined image, and controls the display panel to display the combined image.

[0140] In one embodiment, the processed image includes one of a noise-reduced image, a super-resolution-processed image and an interpolated image.

[0141] It should be understood that in the embodiments of the present application, the input unit 1004 can include a graphics processing unit (GPU) 10041 and a microphone 10042. The graphics processing unit 10041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1006 can include a display panel 10061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 can include a touch detection device and a touch controller. The other input devices 10072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, an operation lever, and the like, which will not be described here.

[0142] The memory 1009 can be used to store software programs and various data. The memory 1009 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), etc. In addition, the memory 1009 can include a volatile memory or a non-volatile memory, or the memory 1009 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 1009 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0143] The processor 1010 can include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1010.

[0144] The embodiments of the present application also provide a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above-mentioned image processing method embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0145] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0146] The embodiment of the present application provides a computer program product, which is stored in a storage medium, and is executed by at least one processor to realize each process of the above image processing method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be repeated here.

[0147] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the present application is not limited to the order of the functions shown or discussed, but also includes the functions performed in a substantially simultaneous manner or in the opposite order, for example, the described method can be performed in a different order from the described order, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0148] From the above description of the embodiments, those skilled in the art can clearly understand that the above embodiment method can be realized by software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method described in each embodiment of the present application.

[0149] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative, not limiting, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

Claims

1. An image processing circuit, characterized in that, It includes an application processing chip and an image processing chip. The image processing chip includes a noise reduction module and a super-resolution processing module. The noise reduction module is used to perform noise reduction processing, and the super-resolution processing module is used to perform super-resolution processing. The noise reduction module and the super-resolution processing module are connected; The image processing chip is used to process the image output by the application processing chip under the control of the application processing chip; When ray tracing is enabled, the application processing chip renders the original image according to a first parameter and / or a second parameter, and transmits the rendered image to the image processing chip. The first parameter is the number of rays, and the second parameter is the resolution. When the application processing chip renders the original image according to the first parameter and / or the second parameter, the first parameter is less than 20 rays, and the second parameter is not greater than 540P resolution. The application processing chip is used to determine, based on the first parameter and / or the second parameter, whether to control the noise reduction processing module to perform noise reduction processing and / or control the super-resolution processing module to perform super-resolution processing.

2. The image processing circuit according to claim 1, characterized in that, The image processing chip also includes an image storage module; The image storage module is connected to at least one of the noise reduction processing module and the super-resolution processing module; The super-resolution processing module is used to read the previous frame image after super-resolution processing from the image storage module, perform super-resolution processing on the current frame image based on the previous frame image after super-resolution processing, and store the processed current frame image in the image storage module. The noise reduction processing module is used to read the previous frame image after noise reduction processing or super-resolution processing from the image storage module, and to perform noise reduction processing on the current frame image based on the previous frame image after noise reduction processing or super-resolution processing.

3. The image processing circuit according to claim 2, characterized in that, The image processing chip also includes a motion compensation processing module; The motion compensation processing module is connected to at least one of the noise reduction processing module, the super-resolution processing module, and the image storage module; The motion compensation processing module is used to read the processed previous frame image from the image storage module and perform frame interpolation processing on the current frame image based on the processed previous frame image; the processed previous frame image includes either the previous frame image after super-resolution processing or the previous frame image after frame interpolation processing.

4. The image processing circuit according to claim 3, characterized in that, The image processing chip also includes a layer combination module; The layer combination module is connected to the first input interface of the image processing chip to receive user interface data output by the application processing chip through the first input interface, and combines the user interface data and the processed image into layers to obtain a combined image; the processed image includes one of the following: an image after noise reduction processing, an image after super-resolution processing, and an image after frame interpolation processing.

5. The image processing circuit according to claim 4, characterized in that, The image processing chip also includes a color gamut control module; The color gamut control module is connected to the layer combination module and the output interface of the image processing chip to control the display panel of the electronic device to display the combined image.

6. An electronic device, characterized in that, Includes the image processing circuit according to any one of claims 1-5.

7. The electronic device according to claim 6, characterized in that, The electronic device also includes a display panel; The display panel is connected to the output interface of the image processing chip.

8. An image processing method, characterized in that, The method, applied to the image processing circuit according to any one of claims 1-5, comprises: When ray tracing is enabled, the application processing chip renders the original image according to a first parameter and / or a second parameter, and transmits the rendered image to the image processing chip. The first parameter is the number of rays, and the second parameter is the resolution. When the application processing chip renders the original image according to the first parameter and / or the second parameter, the first parameter is less than 20 rays, and the second parameter is not greater than 540P resolution. The application processing chip determines, based on the first parameter and / or the second parameter, whether to control the noise reduction processing module to perform noise reduction processing and / or control the super-resolution processing module to perform super-resolution processing. The image processing chip, under the control of the application processing chip, processes the image output by the application processing chip.

9. The method according to claim 8, characterized in that, The image processing chip, under the control of the application processing chip, processes the image output by the application processing chip, including: Under the control of the application processing chip, the image processing chip determines the target processing module from the noise reduction processing module, the super-resolution processing module, and the motion compensation processing module, sets the processing order among the target processing modules, and controls the target processing module to process the image output by the application processing chip according to the processing order.

10. The method according to claim 9, characterized in that, Also includes: The image processing chip receives user interface data output by the application processing chip, combines the user interface data and the processed image to obtain a combined image, and controls the display panel to display the combined image; The processed image includes one of the following: an image after noise reduction processing, an image after super-resolution processing, or an image after frame interpolation processing.

Images