Image processing circuit, image processing method, and electronic device

Abstract

The present application discloses an image processing circuit, an image processing method, and an electronic device, belonging to the field of image processing technology. The image processing circuit includes: a main control chip and an image processing chip, the image processing chip including a first interface, the image processing chip being connected to the main control chip via the first interface; the image processing chip is configured to generate depth information based on a first image and a second image; wherein the first image and the second image are images captured by two different image sensors; and the main control chip is configured to generate a target image based on the first image and a blurred image, wherein the blurred image is generated based on the depth information and the first image.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing circuit, an image processing method and an electronic device. Background Art

[0002] To enhance image quality, the background is often blurred during photography to highlight the outlines of people in the image. Two background blurring methods are commonly used in related technologies: First, a single camera is used to blur the background. This single-camera software algorithm produces an uneven transition between near and far focal points, and noticeable object edge outlines remain in out-of-focus images. Second, a master-slave camera is used to blur the background. However, this method produces significantly different background blur effects during preview than in the final photo, and consumes more power. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide an image processing circuit, an image processing method and an electronic device, which can improve the blurring effect and increase the image processing rate.

[0004] In a first aspect, an embodiment of the present application provides an image processing circuit, including a main control chip and an image processing chip, wherein the image processing chip includes a first interface, and the image processing chip is connected to the main control chip via the first interface;

[0005] The image processing chip is configured to generate depth information based on a first image and a second image; wherein the first image and the second image are images captured by two different image sensors;

[0006] The main control chip is used to generate a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0007] In a second aspect, an embodiment of the present application provides an image processing method, applied to the image processing circuit as described in the first aspect, the method comprising:

[0008] The image processing chip generates depth information based on the first image and the second image; wherein the first image and the second image are images captured by two different image sensors;

[0009] The main control chip generates a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0010] In a third aspect, an embodiment of the present application provides an electronic device, comprising a first image sensor, a second image sensor, a display screen, and the image processing circuit as described in the first aspect.

[0011] The first image sensor and the second image sensor are respectively connected to the image processing chip; the first image sensor is used to capture the first image;

[0012] The second image sensor is used to capture the second image;

[0013] The display screen is connected to the main control chip.

[0014] In a fourth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0015] In an embodiment of the present application, while the main control chip processes the first image, the image processing chip generates depth information, which can effectively reduce the computing pressure of the main control chip, improve the blurring effect and increase the image processing rate, while also reducing the power consumption of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is one of the structural diagrams of the image processing circuit provided in the embodiment of the present application;

[0017] Figure 2 This is the second structural diagram of the image processing circuit provided in an embodiment of the present application;

[0018] Figure 3 This is the third structural diagram of the image processing circuit provided in the embodiment of the present application;

[0019] Figure 4 This is one of the flowcharts of the image processing method provided in the embodiment of the present application;

[0020] Figure 5 This is the second flowchart of the image processing method provided in the embodiment of the present application;

[0021] Figure 6 This is the third flowchart of the image processing method provided in the embodiment of the present application;

[0022] Figure 7 is a structural diagram of an image processing device provided in an embodiment of the present application;

[0023] Figure 8 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0024] Figure 9 This is a hardware diagram of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0026] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0027] The image processing circuit, image processing method, and electronic device provided in the embodiments of the present application are described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0028] The image processing method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal. The execution subject of the image processing method can be the terminal, or a control device of the terminal, etc.

[0029] The terminal includes but is not limited to a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad) or other portable communication devices. It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0030] In the following embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0031] An embodiment of the present application provides an image processing circuit.

[0032] like Figure 1 As shown, the image processing circuit includes: a main control chip 120 and an image processing chip 110 .

[0033] In this embodiment, the image processing chip 110 includes a first interface, and the image processing chip 110 is connected to the main control chip 120 via the first interface;

[0034] The image processing chip 110 is configured to generate depth information based on a first image and a second image; wherein the first image and the second image are images captured by two different image sensors;

[0035] The main control chip 120 is configured to generate a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0036] In this embodiment, the image sensor is used to collect image information, and may be, for example, a color camera, a black and white camera, a wide-angle lens, a telephoto lens, a depth camera, etc. on a mobile phone.

[0037] There are at least two image sensors, for example, a main camera and an auxiliary camera on a mobile phone.

[0038] It is understandable that when the electronic device is working normally, the user can open a shooting APP, such as a mobile phone camera, and in shooting mode, collect image information through at least two cameras set in the mobile phone.

[0039] The image processing chip 110 is a chip independent of the main control chip 120 and the image sensor, and the first interface of the image processing chip 110 is electrically connected to the main control chip 120 .

[0040] like Figure 1 As shown, the first interface may include a MIPI TX0 interface 114 and a MIPI TX1 interface 116. TX is used to represent a transmitting port.

[0041] The image processing chip 110 transmits data with the image sensor and the main control chip 120 via the MIPI protocol.

[0042] The image processing chip 110 is configured to generate depth information based on images captured by at least two image sensors.

[0043] The depth information is the distance range of the object in front and behind measured when the image sensor can obtain a clear image.

[0044] It should be noted that if Figure 2 As shown, in the normal shooting mode, the image processing chip 110 works in the BYPASS mode; when entering the dual-camera background blur mode, the main control chip 120 controls the image processing chip 110 to switch from the BYPASS mode to the normal working mode.

[0045] The main control chip 120 may be an AP chip, including an image signal processing (ISP) module 127, a graphics processor 122 (GPU) module and a CPU module 121, as well as Figure 1 Multiple interfaces shown.

[0046] The main control chip 120 is used to generate a target image, which is the final image generated after a series of processes such as blurring the background and performing noise reduction.

[0047] For example, during a shooting process, the terminal receives a first input from a user, where the first input is used to enter a dual-camera background blur mode.

[0048] In response to the first input, the terminal controls the image processing chip 110 to enter a normal operating mode.

[0049] The first input may be in at least one of the following ways:

[0050] First, the first input may be a touch input, including but not limited to a click input, a slide input, and a press input.

[0051] In this embodiment, receiving the first input from the user may be receiving a touch operation of the user on the display area of the terminal display screen 130 .

[0052] If the target control is displayed on the current interface, the first input can be achieved by touching the target control; or the first input can be set to be a continuous multiple tapping operation on the display area within a target time interval.

[0053] Secondly, the first input may be a physical key input.

[0054] In this embodiment, the terminal body is provided with a corresponding physical button, and receiving the user's first input can be performed by receiving the first input of the user pressing the corresponding physical button; the first input can also be a combined operation of pressing multiple physical buttons at the same time.

[0055] Third, the first input may be voice input.

[0056] In this embodiment, the terminal may trigger the display of the dual-camera background blur shooting interface upon receiving a voice message such as "entering the dual-camera background blur mode".

[0057] Of course, in other embodiments, the first input may also be in other forms, including but not limited to somatosensory gesture input, etc., which can be determined according to actual needs and is not limited in this embodiment of the present application.

[0058] After the image processing chip 110 enters the normal working mode, the image processing chip 110 receives a first image captured by the main camera of the mobile phone and a second image captured by the slave camera, respectively, wherein the first image and the second image are both RAW format images.

[0059] like Figure 2As shown, the image processing chip 110 sends the first image to the main control chip 120 through the MIPI bypass, so that the main control chip 120 processes the first image.

[0060] After receiving the first image, the main control chip 120 converts the first image into a YUV format through the built-in ISP module to generate a third image.

[0061] It should be noted that the main control chip 120 processes the first image and the image processing chip 110 generates depth information based on the first image and the second image, and both can be performed simultaneously.

[0062] The main control chip 120 generates a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0063] It should be noted that the blurred image may be generated by the main control chip 120 based on the depth information and the first image, or may be generated by the image processing chip 110 based on the depth information and the first image.

[0064] During their research and development, the inventors discovered that currently commonly used dual-camera background blurring solutions use the CPU / GPU hardware of the AP main control chip 120 to run software algorithms. These algorithms typically include depth map calculation, background blurring, and fusion of the original image and background blurring. Due to performance limitations, these solutions can only blur backgrounds during photos, and cannot provide real-time preview of the background blurring during the photo capture process.

[0065] In some technologies, the main control chip 120 can be used to perform simple background blurring of the preview image. However, the background blurring effect during the preview process is quite different from the blurring effect in the photo generated by the actual photo taking, and is more power-consuming.

[0066] In the present application, an independent image processing chip 110 is set between the image sensor and the main control chip 120, and the image processing chip 110 generates depth information to realize partial background blur processing function, which can effectively reduce the computing pressure of the main control chip 120, improve the blur effect and reduce power consumption; in addition, by allowing the image processing chip 110 to generate depth information and the main control chip 120 to perform image processing at the same time, the image processing rate can be significantly improved.

[0067] According to the image processing circuit provided in the embodiment of the present application, while the main control chip processes the first image, the image processing chip generates depth information, which can effectively reduce the computing pressure of the main control chip, improve the blurring effect and increase the image processing rate, while also reducing the power consumption of the chip.

[0068] In some embodiments, as Figure 1As shown, the image processing chip 110 includes a depth calculation unit 117 , and the depth calculation unit 117 is electrically connected to the first interface.

[0069] In this embodiment, the depth calculation unit 117 is configured to generate depth information based on images captured by at least two image sensors.

[0070] During actual execution, the depth calculation unit 117 receives images captured by at least two image sensors respectively, generates depth information, and then sends the depth information to the main control chip 120 through the first interface.

[0071] For example, Figure 5 As shown, after the image processing chip 110 enters the normal operating mode, the image processing chip 110 receives a first image captured by the main camera of the mobile phone and a second image captured by the slave camera, respectively, wherein the first image and the second image are both RAW format images.

[0072] The image processing chip 110 sends the first image to the main control chip 120 . The depth calculation unit 117 generates depth information based on the first image and the second image and sends the depth information to the main control chip 120 .

[0073] The main control chip 120 performs background blurring based on the depth information to generate a target image.

[0074] In this embodiment, the depth information is generated by the depth calculation unit 117 in the image processing chip 110 to implement a partial background blur processing function, which can effectively reduce the computing pressure of the main control chip 120 .

[0075] like Figure 3 As shown, in some embodiments, the image processing chip 110 may include a first background blurring unit 118 .

[0076] In this embodiment, the first background blurring unit 118 is electrically connected to the first interface.

[0077] The image processing chip 110 can not only generate depth information, but also generate a blurred image based on the depth information and the first image.

[0078] like Figure 6 As shown, for example, in the actual execution process, after the image processing chip 110 enters the dual-camera working mode, the image processing chip 110 respectively receives a first image captured by the main camera of the mobile phone and a second image captured by the slave camera, wherein the first image and the second image are both RAW format images.

[0079] The image processing chip 110 sends the first image to the main control chip 120 through the MIPI bypass, so that the main control chip 120 processes the first image.

[0080] After receiving the first image, the main control chip 120 converts the first image into a YUV format through the built-in ISP module to generate a third image.

[0081] At the same time, the first background blur unit 118 in the image processing chip 110 generates depth information based on the first image and the second image, converts the first image into a YUV format, blurs the background of the first image in the YUV format based on the depth information, generates a blurred image, and sends the blurred image to the main control chip 120 through the MIPI TX1 interface 116.

[0082] It should be noted that the main control chip 120 processes the first image and the image processing chip 110 generates the blurred image based on the first image and the second image, and both can be performed simultaneously.

[0083] After receiving the blurred image, the main control chip 120 performs an algorithm fusion on the blurred image and the third image to generate a target image.

[0084] In some embodiments, the main control chip 120 may further include a format conversion unit and a fusion unit.

[0085] In this embodiment, the format conversion unit is connected to the image processing chip 110 and the fusion unit respectively.

[0086] The format conversion unit is used to receive the first image sent by the image processing chip 110, and perform format conversion on the first image to generate a third image;

[0087] The fusion unit is used to receive the third image sent by the format conversion unit, and to perform fusion processing on the third image and the blurred image to generate a target image.

[0088] The blurred image is generated by the first background blurring unit 118 and sent to the fusion unit.

[0089] In actual execution, the image processing chip 110 sends the first image to the format conversion unit and sends the blurred image to the fusion unit. The format conversion unit converts the first image into a YUV format to generate a third image and sends the third image to the fusion unit.

[0090] The fusion unit performs algorithmic fusion on the blurred image and the third image to generate a target image.

[0091] According to the image processing circuit provided in the embodiment of the present application, most of the background blur processing functions can be realized by generating a blurred image through the image processing chip 110, thereby further reducing the computing pressure of the main control chip 120 and improving the blur effect; in addition, the blurred image is generated by the image processing chip 110 and is performed simultaneously with the main control chip 120 processing the first image, which can significantly improve the image processing rate.

[0092] like Figure 2 As shown, in some embodiments, the main control chip 120 may further include a second background blurring unit 128 .

[0093] In this embodiment, the second background blurring unit 128 is configured to generate a blurred image based on the depth information and the first image.

[0094] The image processing chip 110 sends the first image to the main control chip 120 via MIPI bypass for processing by the main control chip 120. At the same time, the image processing chip 110 generates depth information based on the first image and the second image, and sends the depth information to the main control chip 120 via the MIPI TX1 interface 116.

[0095] After receiving the first image and the depth information, the main control chip 120 generates a blurred image based on the first image and the depth information, and generates a target image based on the blurred image and the first image.

[0096] In some embodiments, the main control chip 120 may further include a format conversion unit and a fusion unit.

[0097] In this embodiment, the format conversion unit is connected to the image processing chip 110 and the fusion unit respectively.

[0098] The format conversion unit is used to receive the first image sent by the image processing chip 110, and perform format conversion on the first image to generate a third image;

[0099] The fusion unit is configured to receive the third image sent by the format conversion unit and the blurred image sent by the second background blurring unit 128 , and to perform a fusion process on the third image and the blurred image to generate a target image.

[0100] The blurred image is generated by the second background blurring unit 128 .

[0101] During the actual execution process, after the main control chip 120 receives the first image and depth information, on the one hand, the format conversion unit converts the first image into YUV format to generate a third image; on the other hand, the second background blur unit 128 in the main control chip 120 blurs the background of the first image based on the received depth information to generate a blurred image.

[0102] After obtaining the third image and the blurred image, the fusion unit of the main control chip 120 fuses the third image and the blurred image to generate a target image.

[0103] According to the image processing circuit provided in the embodiment of the present application, depth information is generated by the image processing chip 110, and the main control chip 120 generates a blurred image based on the depth information and the first image, and generates a target image based on the blurred image. While reducing the computing pressure of the main control chip 120, the blurring effect can be significantly improved.

[0104] In some embodiments, as Figure 1 and Figure 3 As shown, the main control unit 120 may further include a storage unit 126, which is configured to receive a target image and encode and store the target image.

[0105] During actual execution, after generating the target image, the main control chip 120 may perform JPEG encoding on the target image and save it as a photo.

[0106] An embodiment of the present application provides an image processing method, and the execution subject of the image processing method can be a terminal, including but not limited to mobile terminals such as mobile phones, tablets and cameras; non-mobile terminals such as desktop computers, or terminal control devices, etc.

[0107] It should be noted that the image processing method is applied to the image processing circuit described above.

[0108] like Figure 4 As shown, the image processing method includes: step 410 and step 420.

[0109] Step 410: The image processing chip 110 generates depth information based on a first image and a second image, wherein the first image and the second image are images captured by two different image sensors;

[0110] In this step, the image processing chip 110 is a chip independent of the main control chip 120 and the image sensor, and the first interface of the image processing chip 110 is electrically connected to the main control chip 120 .

[0111] The image processing chip 110 transmits data with the image sensor and the main control chip 120 via the MIPI protocol.

[0112] The first image and the second image are both RAW format images, and are images captured and generated by different sensors in the same environment at the same time.

[0113] Depth information is the distance range in front of and behind the object measured when the image sensor can obtain a clear image.

[0114] It should be noted that, when at least two sensors capture images, different sensors capture different distances to the same object, and depth information can be determined based on the at least two distances.

[0115] When the terminal is in a normal working mode and does not perform background blurring, it is not necessary to generate depth information. In this case, the image processing chip 110 works in a BYPASS mode.

[0116] When the terminal is in the dual-camera shooting mode, depth information needs to be generated. At this time, the image processing chip 110 switches from the BYPASS mode to the dual-camera working mode.

[0117] It is understandable that, before step 410, the method may further include:

[0118] receiving a first input from a user;

[0119] In response to the first input, a dual-camera shooting mode is entered.

[0120] For example, when a user needs to take a photo of a person, the background behind the person needs to be blurred. The user clicks on the position of the person on the shooting interface to focus, thereby achieving the first input.

[0121] The terminal enters a dual-camera shooting mode in response to the first input.

[0122] In the dual-camera shooting mode, the image processing chip 110 is controlled by the main control chip 120 and switches from the BYPASS mode to the dual-camera working mode.

[0123] When the user is shooting preview, the main camera and the slave camera on the mobile phone respectively capture images of the same object, wherein the main camera captures the first image through Figure 1 The MIPI RX0 interface shown in FIG. 1 is sent to the image processing chip 110; the second image collected from the camera is transmitted through the MIPI RX0 interface shown in FIG. Figure 1 The MIPI RX1 interface shown is sent to the image processing chip 110.

[0124] The image processing chip 110 sends the first image to the main control chip 120 through one channel, and the image processing chip 110 calculates depth information based on the first image and the second image.

[0125] After the depth information is calculated, step 420 is executed.

[0126] Step 420: The main control chip generates a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0127] In this step, the target image is the final image generated for display or storage, such as a real-time preview image during the shooting process.

[0128] The target image is the image after image processing and background blurring. The target image is consistent with the final image generated by the actual shooting.

[0129] It should be noted that, in the actual execution process, the main control chip generates the target image based on the first image and the blurred image, which can be expressed in the following two implementation modes:

[0130] First, the main control chip 120 generates a blurred image;

[0131] Second, the image processing chip 110 generates a blurred image.

[0132] This application will provide a detailed description of the above two implementation methods in subsequent embodiments, which will not be elaborated here.

[0133] According to the image processing method provided in the embodiment of the present application, while the main control chip processes the first image, the image processing chip generates depth information, which can effectively reduce the computing pressure of the main control chip, improve the blurring effect and increase the image processing rate, while also reducing the power consumption of the chip.

[0134] The following describes the embodiments of the present application in detail from two different implementation perspectives.

[0135] 1. The main control chip 120 generates a blurred image.

[0136] like Figure 5 As shown, in some embodiments, step 420 may further include:

[0137] The main control chip 120 performs format conversion processing on the first image to generate a third image;

[0138] The main control chip 120 generates a blurred image based on the depth information and the first image;

[0139] The main control chip 120 fuses the blurred image and the third image to generate a target image.

[0140] In this embodiment, the first image may be sent to the main control chip 120 by the image processing chip 110 , or may be directly sent to the main control chip 120 by other components, such as an image sensor.

[0141] In the case where the image sensor directly sends the data to the main control chip 120, a direct communication connection needs to be established between the image sensor and the main control chip 120. The target format may be a YUV format.

[0142] The third image is an image generated after performing RAW domain processing, noise reduction processing, format conversion, and other processing on the first image. The third image is an image in YUV format.

[0143] During the actual execution process, the ISP module built into the main control chip 120 may be used to execute this step.

[0144] It should be noted that the step of the main control chip 120 generating the third image and step 410 can be performed simultaneously.

[0145] The depth information is sent from the image processing chip 110 to the main control chip 120 .

[0146] After receiving the depth information, the main control chip 120 combines the first image with the depth information to perform background blurring to generate a blurred image;

[0147] The blurred image is then fused with the third image to generate a target image.

[0148] In the actual implementation process, Figure 1 As shown, step 410 may be performed by the depth calculation unit 117 in the image processing chip 110 .

[0149] For example, when a user needs to take a photo of a person, the background behind the person needs to be blurred. The user clicks on the position of the person on the shooting interface to focus, which realizes the first input. In response to the first input, the terminal controls the image processing chip 110 to enter the dual-camera working mode.

[0150] After the image processing chip 110 enters the dual-camera working mode, during the preview process, the image processing chip 110 respectively receives a first image captured by the main camera of the mobile phone and a second image captured by the slave camera, wherein the first image and the second image are both RAW format images.

[0151] The image processing chip 110 sends the first image to the main control chip 120 through the MIPI bypass, so that the ISP module 127 of the main control chip 120 processes the first image.

[0152] After receiving the first image, the main control chip 120 converts the first image into YUV format to generate a third image; at the same time, the depth calculation unit 117 of the image processing chip 110 generates depth information based on the first image and the second image, and sends the depth information to the main control chip 120 through the MIPITX1 interface 116.

[0153] After receiving the depth information, the main control chip 120 fuses the depth information with the first image to blur the background of the first image and generate a blurred image; then the blurred image is fused with the third image to generate a target image, in which the person is clear and the scenery behind the person is blurred.

[0154] The main control chip 120 sends the target image to the display screen for real-time preview display. During the preview process, the user can see a clear character image and a blurred background image.

[0155] In some embodiments, after step 420 , the method may further include: the main control chip 120 encoding and storing the target image.

[0156] In this embodiment, the main control chip 120 may further include a storage unit 126 for receiving the generated target image and encoding and storing the target image.

[0157] For example, when the user inputs a shooting input, the terminal responds to the input and performs a shooting operation, and the main control chip 120 can also perform JPEG encoding on the target image and save it as a photo. The final photo generated is consistent with the photo effect seen by the user in the preview.

[0158] According to the image processing method provided in the embodiment of the present application, depth information is generated by the image processing chip 110, and the main control chip 120 generates a blurred image based on the depth information and the first image, and generates a target image based on the blurred image. While reducing the computing pressure of the main control chip 120, the blurring effect can be significantly improved.

[0159] 2. The image processing chip 110 generates a blurred image.

[0160] like Figure 6 As shown, in some embodiments, after step 420, the method may further include: the image processing chip 110 generates a blurred image based on the depth information and the first image, and sends the blurred image to the main control chip.

[0161] Step 420 may also include:

[0162] The main control chip 120 performs format conversion processing on the first image to generate a third image;

[0163] The main control chip 120 fuses the blurred image and the third image to generate a target image.

[0164] In this embodiment, the blurred image is generated by blurring the background of the first image and converting its format. The blurred image is in YUV format and is generated by the image processing chip 110 and sent to the main control chip 120. The method for acquiring the first image can be the same as in the above embodiment and will not be described in detail here. The target format can be YUV format.

[0165] The third image is an image generated after performing RAW domain processing, noise reduction processing, format conversion, and other processing on the first image. The third image is an image in YUV format.

[0166] It should be noted that the image processing chip 110 processes the first image to generate the blurred image, and the main control chip 120 processes the first image to generate the third image, which can be performed simultaneously.

[0167] In the actual implementation process, Figure 3 As shown, the corresponding steps may be performed by the first background blurring unit 118 in the image processing chip 110 .

[0168] For example, when a user wants to take a photo of a person and needs to blur the background behind the person, the user clicks on the position of the person on the shooting interface to focus, thereby implementing the first input; in response to the first input, the terminal controls the image processing chip 110 to enter the dual-camera working mode.

[0169] After the image processing chip 110 enters the dual-camera working mode, during the preview process, the first background blur unit 118 of the image processing chip 110 respectively receives the first image captured by the main camera of the mobile phone and the second image captured by the slave camera, wherein the first image and the second image are both RAW format images.

[0170] The image processing chip 110 sends the first image to the main control chip 120 through the MIPI bypass, so that the main control chip 120 processes the first image.

[0171] After receiving the first image, the main control chip 120 converts the first image into a YUV format through the built-in ISP module to generate a third image.

[0172] At the same time, the first background blur unit 118 generates depth information based on the first image and the second image, converts the first image into a YUV format, blurs the background of the first image in the YUV format based on the depth information, generates a blurred image, and sends the blurred image to the main control chip 120 through the MIPI TX1 interface 116.

[0173] After receiving the blurred image, the main control chip 120 performs algorithm fusion on the blurred image and the third image to generate a target image. In the target image, the person is clear, while the scenery behind the person is blurred.

[0174] In some embodiments, when the user inputs a shooting input, the terminal responds to the input and performs a shooting operation, and the main control chip 120 can also perform JPEG encoding on the target image and save it as a photo. The final photo generated is consistent with the photo effect seen by the user in the preview.

[0175] According to the image processing method provided in the embodiment of the present application, most background blur processing functions can be realized by generating a blurred image by the image processing chip, thereby further reducing the computing pressure of the main control chip, increasing the image processing rate and improving the blur effect.

[0176] The present application also provides an electronic device, comprising a first image sensor, a second image sensor, a display screen, and any one of the image processing circuits described above.

[0177] In this embodiment, the first image sensor and the second image sensor are respectively connected to the image processing chip; the first image sensor is used to capture the first image;

[0178] The second image sensor is used to capture a second image;

[0179] The display screen is connected to the main control chip.

[0180] In this embodiment, there are at least two image sensors, which may be Figure 1 As shown in sensor0 141 and sensor1 142 , in actual implementation, the first image sensor and the second image sensor may be a main camera and a secondary camera on a mobile phone.

[0181] The display screen is connected to the main control chip and is used to display the target image sent by the main control chip.

[0182] For example, the display screen may be Figure 1 The display screen 130 shown may be a user's mobile phone screen during actual implementation.

[0183] During the actual execution process, the main control chip sends the target image to the display screen through the MIPI DSI interface for real-time preview display. During the preview process, the user can see a clear image of the person and a blurred background image.

[0184] Of course, in other embodiments, the main control chip can also send the target image to the display screen through the MIPI DSI interface for real-time preview display, and encode the target image into JPEG and save it as a photo in another way.

[0185] In this embodiment, by displaying the target image on the display screen, the user can view the preview image with the background blurred during the shooting preview process, and the preview image is consistent with the final captured image, thereby improving the user experience.

[0186] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, the image processing device provided in the embodiment of the present application is described by taking the image processing device executing the image processing method as an example.

[0187] An embodiment of the present application also provides an image processing device.

[0188] The image processing device is applied to the image processing circuit described above.

[0189] like Figure 7 As shown, the image processing device includes: a first processing module 710 and a second processing module 720 , and the first processing module 710 is connected to the second processing module 720 .

[0190] A first processing module 710 is configured to generate depth information based on a first image and a second image; wherein the first image and the second image are images captured by two different image sensors;

[0191] The second processing module 720 is configured to generate a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0192] According to the image processing device provided in the embodiment of the present application, while the first image is processed by the second processing module, the depth information is generated by the first processing module, which can effectively reduce the computing pressure of the second processing module, improve the blurring effect and increase the image processing rate, while also reducing the power consumption of the chip.

[0193] In some embodiments, the second processing module 720 is further configured to:

[0194] performing format conversion processing on the first image to generate a third image;

[0195] generating a blurred image based on the depth information and the first image;

[0196] The blurred image and the third image are fused to generate a target image.

[0197] In some embodiments, after the first processing module 710 generates depth information based on the first image and the second image, the first processing module 710 further generates a blurred image based on the depth information and the first image, and sends the blurred image to the second processing module 720 .

[0198] In some embodiments, the second processing module 720 is further configured to:

[0199] performing format conversion processing on the first image to generate a third image;

[0200] The blurred image and the third image are fused to generate a target image.

[0201] In some embodiments, after the second processing module 720 generates the target image based on the first image and the blurred image, the second processing module 720 encodes and stores the target image.

[0202] The image processing device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or a device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0203] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0204] The image processing device provided in the embodiment of the present application can achieve Figures 4 to 6 The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.

[0205] Alternatively, as Figure 8As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and a program or instruction stored in the memory 802 and executable on the processor 801. When the program or instruction is executed by the processor 801, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0206] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0207] Figure 9 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0208] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, a processor 910 and an image processing chip.

[0209] Those skilled in the art will understand that the electronic device 900 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0210] The image processing chip is electrically connected to the sensor 905 and the processor 910 respectively, the processor 910 is electrically connected to the display unit 906 , and the memory 909 is electrically connected to the processor 910 .

[0211] The image processing chip is used to generate depth information based on the first image and the second image; wherein the first image and the second image are images captured by two different sensors 905;

[0212] The processor 910 is configured to generate a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

[0213] According to the electronic device provided in the embodiment of the present application, while the processor processes the first image, the image processing chip generates depth information, which can effectively reduce the processor's computing pressure, improve the blurring effect and increase the image processing rate, while also reducing power consumption.

[0214] Optionally, the processor 910 is further configured to:

[0215] performing format conversion processing on the first image to generate a third image;

[0216] generating a blurred image based on the depth information and the first image;

[0217] The blurred image and the third image are fused to generate a target image.

[0218] Optionally, the image processing chip is further configured to: after generating depth information based on the first image and the second image, generate a blurred image based on the depth information and the first image, and send the blurred image to the processor 910 .

[0219] Optionally, the processor 910 is further configured to:

[0220] performing format conversion processing on the first image to generate a third image;

[0221] The blurred image and the third image are fused to generate a target image.

[0222] Optionally, the memory 909 is used to encode and store the target image.

[0223] It should be understood that in an embodiment of the present application, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and at least one of other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0224] The memory 909 can be used to store software programs and various data.

[0225] The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 909 may include a volatile memory or a non-volatile memory, or the memory 909 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 909 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0226] Processor 910 may include one or more processing units. Optionally, processor 910 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 910.

[0227] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0228] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0229] An embodiment of the present application also provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0230] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0231] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application is essentially or

[0232] The part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD-ROM), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0233] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An image processing circuit, characterized in that: It includes a main control chip and an image processing chip, wherein the image processing chip includes a first interface, and the image processing chip is connected to the main control chip through the first interface; The image processing chip is configured to generate depth information based on a first image and a second image; wherein the first image and the second image are images captured by two different image sensors; The main control chip is used to generate a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image; The image processing chip further includes a first background blurring unit, which is electrically connected to the first interface and configured to generate the blurred image based on the depth information and the first image.

2. The image processing circuit according to claim 1, wherein: The image processing chip includes a depth calculation unit, which is electrically connected to the first interface and is configured to generate the depth information based on the first image and the second image.

3. The image processing circuit according to claim 1, wherein: The main control chip includes a second background blurring unit, and the second background blurring unit is used to generate the blurred image based on the depth information and the first image.

4. The image processing circuit according to claim 1, wherein: The main control chip includes a format conversion unit and a fusion unit, and the format conversion unit is connected to the image processing chip and the fusion unit respectively; The format conversion unit is used to perform format conversion processing on the first image to generate a third image; The fusion unit is used to perform fusion processing on the third image and the blurred image to generate the target image.

5. An image processing method, characterized in that: Applied to the image processing circuit according to any one of claims 1 to 4, the method comprising: The image processing chip generates depth information based on the first image and the second image; wherein the first image and the second image are images captured by two different image sensors; The main control chip generates a target image based on the first image and the blurred image, wherein the blurred image is generated based on the depth information and the first image.

6. The image processing method according to claim 5, characterized in that The main control chip generates a target image based on the first image and the blurred image, including: The main control chip performs format conversion processing on the first image to generate a third image; The main control chip generates the blurred image based on the depth information and the first image; The main control chip fuses the blurred image and the third image to generate the target image.

7. The image processing method according to claim 5, characterized in that: After the image processing chip generates depth information based on the first image and the second image, the method further includes: the image processing chip generates the blurred image based on the depth information and the first image, and sends the blurred image to the main control chip.

8. The image processing method according to claim 7, wherein: The main control chip generates a target image based on the first image and the blurred image, including: The main control chip performs format conversion processing on the first image to generate a third image; The main control chip fuses the blurred image and the third image to generate the target image.

9. The image processing method according to any one of claims 5 to 8, characterized in that: After the main control chip generates a target image based on the first image and the blurred image, the method further includes: The main control chip encodes and stores the target image.

10. An electronic device, characterized in that: comprising a first image sensor, a second image sensor, a display screen, and an image processing circuit according to any one of claims 1 to 4, The first image sensor and the second image sensor are respectively connected to the image processing chip; the first image sensor is used to capture the first image; The second image sensor is used to capture the second image; The display screen is connected to the main control chip.

Images