Image processing method and device

Abstract

The embodiment of the invention provides an image processing method and device, and relates to the technical field of terminals. The method comprises the steps of obtaining a first image; adjusting the pixel value of the first image to obtain a second image; wherein the first ratio is greater than the second ratio; the first ratio is a ratio of a pixel value of a first pixel point in a light spot area in the first image to a pixel value of a second pixel point in the light spot area in the first image, and the second ratio is a ratio of a pixel value of a first pixel point in the second image to a pixel value of the second pixel point in the second image; the first pixel points comprise pixel points at the edge of a light spot area in the image, the second pixel points comprise a pixel point corresponding to a central point of the light spot area and N pixel points around the pixel point, and N is an integer greater than 0; and filtering the second image to obtain a blurred image. In this way, the pixel value of the pixel point at the edge of the light spot area in the second image is compared with the pixel value of the pixel point at the center of the light spot area more obviously, and the light spot permeability in the blurred image is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method and apparatus. Background Technology

[0002] Mobile phones and other electronic devices not only support shooting functions, but also support blurring the image to create a bokeh effect. In a bokeh image, the foreground is sharp while the background is blurred, creating a good sense of space.

[0003] When there are bright objects in the background of an image, the electronic device can create light spots in the area corresponding to the bright objects after blurring the image. Bright objects can include: night scene lights, candlelight, or gaps between leaves under sunlight, etc.

[0004] However, the display effect of light spots in blurred images is poor. Summary of the Invention

[0005] This application provides an image processing method and apparatus, applicable to the field of terminal technology. The method adjusts the pixel values ​​of an image to make the pixel values ​​of pixels at the edges of the light spot region more distinct from those at the center of the light spot region, thereby improving the contrast between pixels at the edges and the center of the light spot region, enhancing the transparency of the light spot in the blurred image, and improving the display effect of the light spot.

[0006] In a first aspect, embodiments of this application propose an image processing method. The method includes: acquiring a first image; adjusting the pixel values ​​of the first image to obtain a second image; wherein a first ratio is greater than a second ratio; the first ratio is the ratio of the pixel value of a first pixel in a spot region of the first image to the pixel value of a second pixel in the spot region of the first image, and the second ratio is the ratio of the pixel value of a first pixel in the second image to the pixel value of a second pixel in the second image; the first pixel includes pixels at the edge of the spot region in the image, and the second pixel includes pixels corresponding to the center point of the spot region and N surrounding pixels, where N is an integer greater than 0; and filtering the second image to obtain a blurred image.

[0007] The first image can be an image captured by a camera application, an image saved by a gallery application, or an image from other scenarios, such as an image captured by a non-camera application. For example, the first image can correspond to the sharp image described below. The second image can correspond to the weighted sharp image described below. The first pixel can correspond to the pixel in region A described below; the second pixel can correspond to the pixel in region D described below.

[0008] In this way, compared with the first image, the contrast of the edge and the center of the light spot region in the second image is higher, so that the light spot in the blurred image has better permeability, and the display effect of the light spot is improved.

[0009] In a possible implementation, the adjusting the pixel value of the first image to obtain the second image includes: adjusting the brightness value of the light spot region in the first image to obtain a third image; the brightness value of the first pixel point in the third image is less than the brightness value of the first pixel point in the first image, and / or the brightness value of the second pixel point in the light spot region in the third image is greater than the brightness value of the second pixel point in the light spot region in the first image; the weight of each pixel point in the third image is obtained by using the brightness value of each pixel point in the third image and a corresponding relationship, the corresponding relationship being the relationship between the brightness value and the weight; and the pixel value of each pixel point in the first image is multiplied by the corresponding weight to obtain the second image.

[0010] The third image can correspond to the optimized brightness image in the following. The weight of each pixel point can correspond to the blurring weight in the following. The blurring weight is used to indicate the blurring degree of the pixel point in the image, the higher the blurring weight, the higher the blurring degree; the lower the blurring weight, the lower the blurring degree.

[0011] In this way, the brightness value at the edge of the light spot region in the first image is adjusted, so that the corresponding weight of the pixel point is reduced, the corresponding pixel value of the pixel point in the second image is reduced, the brightness of the light spot in the blurred image is reduced, and the permeability is improved. Moreover, the brightness of the light spot in the blurred image is reduced, the case that the brightness of the light spot is relatively high is reduced, and the superposition region of adjacent light spots is more obvious. The brightness value at the center of the light spot region in the first image is adjusted, so that the corresponding weight of the pixel point is increased, the corresponding pixel value of the pixel point in the second image is increased, the brightness of the light spot in the blurred image is increased, the case that the light spot is blurred due to the low brightness of the light spot is reduced, and the display effect of the light spot is improved.

[0012] In a possible implementation, the corresponding relationship includes the relationship among the brightness value, the depth value, and the weight; and the weight of each pixel point in the third image is obtained by using the brightness value of each pixel point in the third image and the corresponding relationship, including: the weight of each pixel point in the third image is obtained by using the brightness value of each pixel point in the third image, the depth value of each pixel point in the first image, and the corresponding relationship.

[0013] The weight in the corresponding relationship can correspond to the blurring weight in the following. In this way, the weight of the pixel point is also related to the depth value of the pixel point, so that the blurring degree of the region farther from the foreground in the blurred image is higher, and the spatial sense of the blurred image is improved.

[0014] In a possible implementation, the adjusting the luminance value of the light spot region in the first image to obtain the third image comprises: in a case where an area of the first light spot region in the first image is greater than a first threshold, adjusting the luminance value of a first pixel point in the first light spot region to a first value to reduce the area of the first light spot region; the first value is less than a light spot threshold, and the light spot threshold is used to determine the light spot region in the first image.

[0015] The first threshold can correspond to threshold B in the following, and the first value can correspond to luminance value D in the following. In this way, the luminance value of the pixel point at the edge of the light spot region is adjusted to a fixed value, which is simple in manner and small in calculation amount.

[0016] In a possible implementation, the adjusting the luminance value of the first pixel point in the first light spot region to the first value comprises: expanding the first light spot region to obtain a first region, the first region being greater than the first light spot region; counting luminance values of pixel points in the first region; and adjusting the luminance value of the first pixel point in the first light spot region to the first value, the first value being a minimum value among the luminance values of the pixel points in the first region.

[0017] The first region can correspond to region C in the following. In this way, the luminance value is selected in the vicinity of the light spot region, which can reduce the case that the luminance value of the first region does not match the luminance in the vicinity of the light spot region, reduce the case of overdarkness or overbrightness caused by the mismatch, and improve the display effect of the light spot in the subsequent blurred image.

[0018] In a possible implementation, the first pixel point is a pixel point in the first light spot region except for the pixel points in the first region, and distances of the pixel points in the first region to a center point of the first light spot region are less than or equal to a second threshold.

[0019] The second threshold can correspond to threshold D in the following.

[0020] In this way, the first region is confirmed by the second threshold and the center point of the light spot region, which is small in calculation amount and easy to implement.

[0021] In a possible implementation, in a case where a luminance value of a center point of the second light spot region in the first image is less than a third threshold, the luminance value of a second pixel point in the second light spot region is adjusted to a second value, and the second value is greater than or equal to the third threshold.

[0022] The third threshold can correspond to threshold C in the following. The second value can correspond to luminance value E in the following.

[0023] In this way, in a case where the luminance of the center of the light spot region is low, the luminance of the center of the light spot region is improved, the case of blurring caused by low luminance of the light spot in the blurred image is reduced, and the display effect of the light spot is improved.

[0024] In a possible implementation, the second value is positively correlated with the first distance, and the first distance is negatively correlated with the distance between the second pixel point and the center point in the second light spot region.

[0025] The first distance can correspond to the distance A in the following. In this way, the brightness of the light spot region is more in line with the brightness variation rule of the highlight object (point light source such as a lamp or a candle), and the display effect of the light spot is improved.

[0026] In a possible implementation, the pixel value of the first image is adjusted to obtain the second image, including: adjusting the saturation of the pixel point in the light spot region in the first image to obtain a fourth image; wherein the saturation of the pixel point in the light spot region in the fourth image is greater than the saturation of the pixel point in the light spot region in the first image; and the pixel value of the fourth image is adjusted to obtain the second image.

[0027] The fourth image corresponds to the optimized clear image in the following. In this way, by increasing the saturation corresponding to the pixel point, the color vividness of the light spot region is improved, and the color dimness of the subsequent light spot is reduced.

[0028] In a possible implementation, the coordinate point of the pixel point in the YUV space color coordinate system includes: an abscissa and an ordinate, the abscissa corresponds to the U value of the pixel point, and the ordinate corresponds to the V value of the pixel point; and the saturation of the pixel point in the light spot region in the first image is adjusted, including: adjusting the coordinate point of the pixel point in the light spot region in the YUV space color coordinate system in the first image to adjust the saturation of the pixel point; wherein the ratio of the ordinate to the abscissa of the pixel point in the light spot region in the YUV space color coordinate system in the fourth image is the same as the ratio of the ordinate to the abscissa of the pixel point in the light spot region in the YUV space color coordinate system in the first image; and the saturation of the pixel point is positively correlated with a second distance in the YUV space color coordinate system, and the second distance is the distance between the coordinate point of the pixel point in the YUV space color coordinate system and the origin.

[0029] In this way, the slope of the coordinate point of the pixel point in the YUV space color coordinate system is the same before and after the saturation adjustment; it can also be understood that the coordinate point is adjusted proportionally, and the color deviation caused by the saturation adjustment is reduced.

[0030] In a possible implementation, a third ratio value is positively correlated with a third value; the third ratio value is the ratio of the ordinate of the pixel point in the light spot region in the fourth image to the ordinate of the pixel point in the light spot region in the first image; and the third value is the minimum value between the absolute value of the abscissa and the absolute value of the ordinate of the pixel point in the YUV space color coordinate system.

[0031] In this way, the smaller value is selected, the color deviation phenomenon during the color adjustment of the pixel point is reduced, and the color optimization effect is improved.

[0032] In a possible implementation, the third ratio satisfies: C=MIN(|U N | γ ,|V N | γ ), where C is the third value, γ is a gamma coefficient, U N is a horizontal coordinate of a pixel point in the spot region in the first image in a YUV space color coordinate system, and V N is a vertical coordinate of the pixel point in the spot region in the first image in the YUV space color coordinate system.

[0033] In this way, the coordinate point of the pixel point is subjected to gamma mapping.

[0034] In a possible implementation, in a case where a saturation corresponding to the third pixel point in the spot region in the first image is less than or equal to a fourth threshold value, the saturation of the third pixel point is increased.

[0035] In this way, the pixel point with a higher saturation value can not be subjected to color optimization, so that the calculation amount of color optimization is reduced, and the color optimization speed is improved. The pixel point with a lower saturation value is subjected to color optimization, so that the phenomenon of color dimming and low saturation caused by overexposure is reduced, and the phenomenon of white light and low saturation in the subsequent blurred image is reduced.

[0036] In a possible implementation, before the first image is adjusted, the method further includes: obtaining a first spot map of the first image according to luminance values of each pixel point in the first image and a spot threshold value, a connected domain in the first spot map being used to indicate a spot region in the first image, and a pixel point in the connected domain of the first spot map corresponding to a pixel point in the first image with a luminance value greater than or equal to the spot threshold value.

[0037] The first spot map can correspond to a spot binary map A in the following description. The spot threshold value can correspond to a spot threshold value in the following description.

[0038] In this way, the spot region is determined by using the spot threshold value, and the method is simple and has a small calculation amount.

[0039] In a possible implementation, after the first spot map of the first image is obtained and before the first image is adjusted, the method further includes: adjusting the connected domain in the first spot map according to the luminance values of each pixel point in the first image, to obtain a second spot map, the connected domain of the second spot map corresponding to the spot region in the first image; and in a case where a difference between the spot threshold value and a luminance value corresponding to a fourth pixel point is less than a fifth threshold value, the fourth pixel point is located in the connected domain of the second spot map, and the fourth pixel point is a pixel point adjacent to a boundary of the connected domain of the first spot map.

[0040] The fifth threshold value can correspond to a fixed value A in the following description, and the second spot map can correspond to a spot binary map B in the following description.

[0041] In this way, the omission of the pixel corresponding to the light spot area can be reduced, the inaccurate identification of the light spot can be reduced, and the case that the area of the light spot area is small can be reduced.

[0042] In a possible implementation, the first image is acquired, including: in response to a photographing operation, the first image is captured.

[0043] In this way, the image processing can be performed on the image captured in the photographing scene in the camera application or the non-camera application, and the display effect of the light spot in the blurred image in this scene can be improved.

[0044] In a possible implementation, the method further includes: displaying a first interface, the first interface displaying a first image and a first control; and acquiring the first image, including: in response to a triggering operation on the first control, acquiring the first image.

[0045] The first control is used to instruct the first image to be blurred. The first control can correspond to the blurring control in the following description, which is not limited here. In this way, the image processing can be performed on the image displayed by the electronic device (for example, the image displayed in the gallery application), and the display effect of the light spot in the blurred image in this scene can be improved.

[0046] In a second aspect, the embodiments of the present application provide an image processing apparatus. The image processing apparatus can be an electronic device, or a chip or chip system in the electronic device. The image processing apparatus can include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit can be a display screen. The display unit is configured to perform the display step, so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is an electronic device, the processing unit can be a processor. The image processing apparatus can further include a storage unit, which can be a memory. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or chip system in the electronic device, the processing unit can be a processor. The processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit (for example, a register, a cache, etc.) in the chip, or a storage unit (for example, a read-only memory, a random access memory, etc.) in the electronic device and located outside the chip.

[0047] For example, the display unit is configured to display a blurred image. The processing unit is configured to process the steps of data processing in the image processing apparatus.

[0048] In a possible implementation, the image processing apparatus further includes a storage unit, which can include one or more memories, and the memories can be devices for storing programs or data in one or more devices or circuits.

[0049] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being configured to store code instructions, and the processor being configured to execute the code instructions to perform the method described in the first aspect or any possible implementation of the first aspect.

[0050] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program or instructions, and when the computer program or instructions are executed on a computer, the computer is caused to perform the method described in the first aspect or any possible implementation of the first aspect.

[0051] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program, and when the computer program is executed on a computer, the computer is caused to perform the method described in the first aspect or any possible implementation of the first aspect.

[0052] In a sixth aspect, the present application provides a chip or chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is configured to execute a computer program or instructions to perform the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0053] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, and the at least one memory stores instructions. The memory can be a storage unit inside the chip, such as a register, a cache, etc., or can be a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0054] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding feasible implementation are similar, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0055] FIG. 1A A scene schematic diagram provided by an embodiment of the present application;

[0056] FIG. 1B An interface schematic diagram of an electronic device provided by an embodiment of the present application in a photographing scene;

[0057] FIG. 1C An interface schematic diagram of an electronic device in a photographing scene according to an embodiment of the present application;

[0058] FIG. 2 A flowchart of an image processing method in a possible design;

[0059] FIG. 3A An interface schematic diagram of an electronic device in a gallery editing scene according to an embodiment of the present application;

[0060] FIG. 3B An interface schematic diagram of an electronic device in a gallery editing scene according to an embodiment of the present application;

[0061] FIG. 3C An interface schematic diagram of an electronic device in a gallery editing scene according to an embodiment of the present application;

[0062] FIG. 3D An interface schematic diagram of an electronic device in a gallery editing scene according to an embodiment of the present application;

[0063] FIG. 3E An interface schematic diagram of an electronic device in a gallery editing scene according to an embodiment of the present application;

[0064] FIG. 4A A structural schematic diagram of an electronic device according to an embodiment of the present application;

[0065] FIG. 4B A position schematic diagram of a camera in an electronic device according to an embodiment of the present application;

[0066] FIG. 5 A software structural schematic diagram of an electronic device according to an embodiment of the present application;

[0067] FIG. 6 A flowchart of an image processing method in a photographing scene according to an embodiment of the present application;

[0068] FIG. 7 A schematic diagram of a YUV space color coordinate system according to an embodiment of the present application;

[0069] FIG. 8 A contrast schematic diagram of a blurred image according to an embodiment of the present application;

[0070] FIG. 9 A flowchart of an image processing method in an editing scene according to an embodiment of the present application;

[0071] FIG. 10 A flowchart of an image processing method according to an embodiment of the present application;

[0072] FIG. 11 FIG. 1 is a structural schematic diagram of an image processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0073] To facilitate clear description of the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0074] 1. Preview image and photographed image

[0075] The preview image can be data collected in real time based on the camera of the electronic device and allowed to be displayed in the preview picture. For example, when the electronic device receives an operation of the user starting the camera application, the electronic device can collect the preview image captured by the camera and display it in the preview picture of the camera application in real time.

[0076] The photographed image can be data obtained based on the photographing control in the electronic device, such as the target image described in the embodiments of the present application. For example, when the electronic device receives a triggering operation of the user on the photographing control, the electronic device can obtain the photographed image based on the camera at the photographing moment.

[0077] 2. High dynamic range (HDR)

[0078] HDR is a processing technology for improving image brightness and contrast. Compared with ordinary images, HDR can provide more dynamic range and image details, and use images with optimal details corresponding to each exposure time to synthesize the final HDR image, which can better reflect the visual effect in the real environment.

[0079] One possible implementation of the electronic device determining whether the current is an HDR scene can be that the electronic device performs 4 times downsampling on the preview image to obtain a preview small image, and determines whether the proportion of the number of highlight pixels in the preview small image to the number of all pixels in the preview small image is greater than a preset pixel threshold. The preview small image can be obtained by retaining one row of pixel points in every two rows of pixel points in the picture corresponding to the preview image, and storing every row of pixel points column by column. The highlight pixel can be determined based on a gray threshold of the pixel point. The pixel threshold can be used to determine whether the current scene is a high dynamic scene. The electronic device can determine whether the current is a high dynamic scene based on one frame of data or multiple frames of data, which is not limited in the embodiments of the present application.

[0080] 3. Exposure time (or exposure duration)

[0081] Exposure time is the time for which the shutter is open in order to project light onto the photosensitive surface of the photographic photosensitive material, or can also be understood as the time interval from the opening to the closing of the shutter.

[0082] Exposure time refers to the photosensitive time of the film. The longer the exposure time, the brighter the photo generated on the film, and vice versa. In the case of relatively dark external light, the exposure time is generally required to be extended to capture a brighter image.

[0083] 4. Saturation

[0084] Saturation is one of the basic characteristics of color, which is used to describe the purity or intensity of the color seen by the human eye. The purity of color can also be referred to as the degree of brightness of color.

[0085] 5. Other terms

[0086] In the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using "first", "second", etc. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit the order. Those skilled in the art can understand that "first", "second", etc. do not limit the number and execution order, and "first", "second", etc. do not necessarily mean different.

[0087] It should be noted that in the embodiments of the present application, "exemplary" or "for example" is used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0088] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c, can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0089] 6. Electronic device

[0090] The electronic device of the embodiments of the present applicationapplicationinclude a handheld device having an image processing function, a vehicle-mounted device, etc. For example, some electronic devices are: a mobile phone, a tablet computer, a palm computer, a notebook computer, a mobile internet device (MID), a wearable device (e.g., a smart watch, smart glasses, a smart bracelet, or smart jewelry, etc.), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with a wireless communication function, a computing device, or other processing devices connected to a wireless modem, a vehicle-mounted device, a terminal device in an internet of things (IoT) system, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc., and the embodiments of the present application are not limited thereto.

[0091] The electronic device in the embodiments of the present applicationapplicationalso be referred to as: a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent, or a user apparatus, etc.

[0092] In this embodiment, the electronic device or various network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0093] Electronic devices can blur the background of images captured by the device. Specifically, they can blur bright objects in the background of an image into light spots. Bright objects can include: nighttime lights, candlelight, or gaps between leaves in sunlight, etc.

[0094] However, the display effect of light spots in blurred images is poor.

[0095] The following example uses a mobile phone as an example for illustration. This example does not constitute a limitation on the embodiments of this application.

[0096] For example, FIG. 1A This is a schematic diagram of a scenario provided for an embodiment of this application. FIG. 1B and FIG. 1C for FIG. 1A The diagram shows the interface of the electronic device corresponding to the scenario depicted. FIG. 1A The scene shown may include a person 101 in the foreground and a light 102 in the background.

[0097] In response to the camera being turned on, the electronic device can display something like... FIG. 1B The interface shown can be a preview interface for the portrait photography function. This interface may include one or more of the following: a camera control 103, a preview screen, controls for switching cameras, or controls for indicating the use of the photography function. The camera application may also include other modes besides portrait photography mode, such as aperture shooting mode, night scene mode, video recording mode, short video mode, and HDR mode.

[0098] The preview image 104 displayed in the preview screen can be obtained by the electronic device recognizing the foreground and background of the captured image and blurring the recognized background. In a possible implementation, the blurred preview image may not be displayed in the preview screen, and this embodiment of the application does not limit this.

[0099] In response to the triggering operation of the user on the photographing control 103, the electronic device photographs the scene shown in FIG. 1A and obtains the photographing image 105 in the interface shown in FIG. 1B. The photographing image 105 can be obtained by performing foreground and background recognition on the photographed image and performing blur processing on the recognized background. FIG. 1A FIG. 1C

[0100] The photographing image 105 can include a clear portrait area 106 and a light spot area 107, and the like. Referring to the scene in FIG. 1A and the photographing image shown in FIG. 1B, the color of the light spot in the light spot area 107 is relatively dark, and the shape and transparency effect of the light spot are poor. FIG. 1A FIG. 1C It can be understood that the area corresponding to the light in the image is a highlight area. When the electronic device performs filter processing on the background area of the image, in the light spot area with a large area, the brightness is relatively high, the detail clarity is relatively low, and the transparency is relatively low. For example, the brightness of the area where the adjacent light spots in the blurred image are superimposed is similar to the brightness of the area where the light spots are not superimposed, so that the user cannot distinguish the superimposed area, and the transparency is poor.

[0101] It can be understood that the area corresponding to the light in the image is a highlight area. When the electronic device performs filter processing on the background area of the image, in the light spot area with a large area, the brightness is relatively high, the detail clarity is relatively low, and the transparency is relatively low. For example, the brightness of the area where the adjacent light spots in the blurred image are superimposed is similar to the brightness of the area where the light spots are not superimposed, so that the user cannot distinguish the superimposed area, and the transparency is poor.

[0102] The following describes the blur processing of the image in the possible design in combination with FIG. 2. An exemplary, FIG. 2 is a flowchart of an image processing method in the possible design. As shown in FIG. 2, the flow includes: FIG. 2 FIG. 2 S201, in response to a photographing operation, obtaining a clear image.

[0103] The photographing operation can be a triggering operation of the user on the photographing control in the portrait photographing function, for example, a click operation on the photographing control in the photographing function. Herein, no specific limitation is made.

[0104] S202, obtaining a brightness image corresponding to the clear image.

[0105] The brightness image is used to represent the brightness information of the image. The brightness image includes the brightness value of each pixel point, which is used to indicate the brightness or brightness of the pixel point.

[0106] S203, according to the brightness value of each pixel in the brightness image, searching a weight table to obtain the weight corresponding to each pixel.

[0107] S204, multiplying the weight corresponding to each pixel with the background area in the clear image to obtain a weighted clear image.

[0108] S204, multiplying the weight corresponding to each pixel with the background area in the clear image to obtain a weighted clear image.

[0109] ​​​​S205, performing filtering calculation on the weighted clear image to obtain a blurred image.

[0110] From FIG. 2 As can be seen from the flowchart shown in the figure, in possible designs, the point light source region in the image is not adjusted and optimized, and thus the light spot corresponding to the region in the blurred image may have a dark color, poor shape, and poor transparency effect, and the effect of the blurred image is poor.

[0111] In possible designs, the electronic device can perform brightness stretching on the clear image, and perform weighted calculation on the image after brightness stretching through the light spot template to obtain a blurred image. However, the brightness of the center of the light spot in the blurred image is high, and the effect is unnatural. In addition, after brightness stretching is performed on the clear image, the highlight object in the image may have overexposure, and thus the display effect of the light spot in the blurred image is poor.

[0112] Therefore, an image processing method and related apparatus are provided in the embodiments of the present application. The electronic device can identify a light spot region in an image, and perform color optimization, transparency optimization processing, and the like on the light spot region in the image; and perform blurring processing on the image after optimization processing, to improve the display effect of the light spot in the blurred image.

[0113] An exemplary FIG. 3A to FIG. 3E Another interface schematic diagram of an electronic device in another scenario provided by the embodiments of the present application is provided. Taking an image saved by a gallery application as an example, FIG. 3A The interface shown in the figure is a gallery interface. The gallery interface can include at least one thumbnail. The at least one thumbnail can be a thumbnail of an image, or a thumbnail of a video, which is not limited here.

[0114] When the electronic device receives a triggering operation (for example, a click operation) on the thumbnail 301 of the image, the electronic device enters the image interface as shown in FIG. 3B The image interface includes: an image 302 corresponding to the thumbnail 301 and a setting item. The setting item includes but is not limited to: sharing, collecting, editing, deleting, more 303 or other types of editing selection items.

[0115] When the electronic device receives a triggering operation (for example, a click operation) on the more 303, the electronic device displays more editing items, and enters the interface as shown in FIG. 3C The interface displays: a blurred 304, details, and the like editing items.

[0116] In some embodiments, the "blurred 304" can also be set in the interface as shown in FIG. 3B In this way, the user can edit the image more conveniently, the operation is more convenient, and the user experience is improved.

[0117] When the electronic device receives a click, touch, or other trigger operation targeting the blurring 304, the electronic device enters... FIG. 3D The interface shown includes an adjustment bar 305, a slider 306, and an image 302. When the electronic device receives a sliding operation on the slider 306, it determines the background blur ratio in the image 302 based on the position of the slider 306 on the adjustment bar 305. The electronic device blurs the background in the image 302 using the image processing method provided in this embodiment and then enters... FIG. 3E The interface shown. This interface displays a blurred image 307.

[0118] In some embodiments, the electronic device may also not display anything. FIG. 3D As shown in the interface, when the electronic device receives a click, touch or other trigger operation for the blurred image 304, the electronic device blurs the background in the image 302 using the image processing method provided in this application embodiment, and obtains and displays the blurred image.

[0119] It is understood that the camera's shooting scene and the gallery's editing scene in the above embodiments are merely examples. The image processing method provided in this application embodiment can also be applied to other scenarios, such as background blurring in video recording mode, and background blurring in scenarios involving taking photos, video calls, and live streaming in third-party camera applications. Third-party camera applications may include: Applications are not specifically limited here. The scenarios in which the image processing method provided in this application is applied are not specifically limited.

[0120] To better understand the embodiments of this application, the structure of the electronic device according to the embodiments of this application is described below. For example, FIG. 4A This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0121] The electronic device can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0122] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0123] The processor 110 can include one or more processing units. Different processing units can be independent devices, or can be integrated in one or more processors. The processor 110 can also be provided with a memory for storing instructions and data. For example, the processor 110 is used to store instructions and data related to the image processing method provided by the embodiments of the present application.

[0124] The camera 193 is composed of PCB board, lens, fixer, color filter, DSP and sensor components. The imaging principle of the camera 193 is that the object passes through the lens, projects the generated optical image onto the sensor, and then the optical image is converted into an electrical signal. The electrical signal is converted into a digital signal after analog-to-digital conversion, and the digital signal is processed by the DSP, and is sent to the processor of the terminal device for processing, and finally converted into an image displayed on the display screen of the electronic device.

[0125] The camera 193 can be classified into wide-angle lens, standard lens, long-focus lens, periscope zoom lens, depth-of-field lens, etc. according to the lens type. Two important parameters of the lens of the camera 193 are aperture and focal length. The aperture is a device installed on the lens to control the amount of light reaching the sensor. In addition to controlling the amount of light, the aperture also has the function of controlling the depth of field, and the larger the aperture, the smaller the depth of field. The large-aperture effect is a manifestation of a small depth of field. The focal length refers to the distance between the center point of the lens and the clear image formed on the sensor plane. The focal length of the lens determines the size of the image of the object captured by the lens on the sensor, that is, the farther the object in the long-focus case, the larger the image. The camera 193 is an imaging sensor for taking pictures. In general, the more cameras, the clearer the image.

[0126] It can be understood that the electronic device can include one or more cameras. The number of cameras is not limited in the embodiments of the present application.

[0127] In some embodiments, the electronic device includes at least two cameras. In this way, the two cameras can be used for shooting to improve the shooting effect of the image. For example, as shown in FIG. 4A, the electronic device includes a camera 401 and a camera 402. The camera 401 and the camera 402 are used to capture images on the same side. FIG. 4B

[0128] It can be understood that the camera 401 and the camera 402 do not completely coincide, and therefore the exposure and color information of the images captured by the camera 401 and the camera 402 can be different, and the image after fusing the image captured by the camera 401 and the image captured by the camera 402 is more detailed and has better image quality.

[0129] It can be understood that when the distance between the camera 401 and the camera 402 is small, the range of the two cameras is small, and the misalignment has little effect when the images are fused. When the distance between the camera 401 and the camera 402 is large, the depth of field information can be better calculated, and the foreground and background can be easily segmented and subsequent background blurring and other processing can be facilitated. The distance and position between the camera 401 and the camera 402 are not limited in the embodiments of the present application.

[0130] It can be understood that in addition to the camera 401 and the camera 402, the electronic device shown in FIG. 4B can also include the camera shown in the virtual frame. The camera is not limited here. FIG. 4B

[0131] For example, in the embodiments of the present application, after the electronic device detects a trigger operation through the touch sensor 180K, the display screen of the electronic device can realize switching display of the corresponding interface in FIG. 4C, and / or, FIG. 1A to FIG. 1C FIG. 3A to FIG. 3E ​​​Switching display of the corresponding interface. When the electronic device can detect a trigger operation for "blurring" through the touch sensor 180K, the processor 110 can perform blurring processing on the clear image to obtain a blurred image through the image processing method provided by the embodiments of the present application.

[0132] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture, and the like, which will not be described here.

[0133] Exemplarily, FIG. 5 A software structure schematic diagram of an electronic device provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer. FIG. 5

[0134] The application layer can include a series of application packages. In the embodiments of the present application, the application package can include: a camera, a gallery, etc. The camera can realize the shooting of the image and the display of the photographed image. The gallery can also be called an album, etc. The gallery can realize the storage and access of the photographed image.

[0135] The application framework layer provides the application programming interface (API) and the programming framework for the application programs of the application layer. The application framework layer includes some pre-defined functions. In the embodiments of the present application, the application framework layer can include a camera access interface and a gallery access interface, wherein the camera access interface can include camera management and camera devices. The camera access interface is used to provide the application programming interface and the programming framework for the camera application. The gallery access interface is used to provide the application programming interface and the programming framework for the gallery application

[0136] The hardware abstraction layer is an interface layer between the application framework layer and the driver layer, which provides a virtual hardware platform for the operating system. In the embodiments of the present application, the hardware abstraction layer can include a camera hardware abstraction layer and a camera algorithm library.

[0137] The camera hardware abstraction layer can provide virtual hardware of the camera device 1, the camera device 2 or more camera devices. The camera algorithm library can include running code and data for implementing the image processing method provided by the embodiments of the present application. Exemplarily, the camera algorithm library can include a blurring algorithm module, etc. The blurring algorithm module is used to implement the blurring processing of the image.

[0138] ​The driving layer is a layer between hardware and software. The driving layer includes drivers of various hardware. The driving layer can include a camera device driver, a digital signal processor driver, and an image processor driver, etc.

[0139] The camera device driver is used to drive the sensor of the camera to collect images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process the images. The image processor driver is used to drive the image processor to process the images.

[0140] The image processing method in the embodiments of the present application is described in detail below in combination with the system structure described above:

[0141] Taking a photographing process as an example, in response to a user's operation of opening a camera application, for example, an operation of clicking a camera application icon, the camera application calls a camera access interface of the application framework layer, starts the camera application, and then sends an instruction of starting a camera through a camera device (a camera device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends the instruction to a camera device driver in the kernel layer. The camera device driver can start a corresponding camera sensor and collect image light signals through the sensor. One camera device in the camera hardware abstraction layer corresponds to one camera sensor in the hardware layer.

[0142] Then, the camera sensor can transmit the collected image light signals to the image signal processor for pre-processing to obtain an image electric signal (an original image), and transmit the original image to the camera hardware abstraction layer through the camera device driver.

[0143] The camera hardware abstraction layer can send the original image to a camera algorithm library. The camera algorithm library stores program codes for implementing the image processing method provided in the embodiments of the present application. Based on the digital signal processor and the image processor, the camera algorithm library executes the codes to implement the process of generating a blurred image in the image processing method described in the embodiments of the present application.

[0144] The camera algorithm library can send the original image collected by the camera to the camera hardware abstraction layer. Then, the camera hardware abstraction layer can display the original image.

[0145] Taking image editing in a gallery application as an example, in response to a user's operation on a "blurring" control, the gallery application calls the camera algorithm library through a gallery access interface, and implements the process of generating a blurred image in the image processing method described in the embodiments of the present application through the camera algorithm library.

[0146] It can be understood that the software architecture provided in the embodiments of the present application is only an example and cannot constitute a limitation on the embodiments of the present application.

[0147] The image processing method in the embodiments of the present application is described in detail below in combination with the system structure described above:FIG. 6 to FIG. 10 The image processing method in the camera photographing scene and the image processing method in the gallery editing scene are described.

[0148] An exemplary image processing method is shown in FIG. 1. FIG. 6 An exemplary image processing method is shown in FIG. 1. FIG. 6 In the corresponding embodiment, the electronic device can include a camera, a gallery, a camera access interface, a camera algorithm library, a camera hardware abstraction layer, and a camera device driver. The functions of any module can be referred to FIG. 5 In the corresponding embodiment, the functions of any module can be referred to FIG. 6 As shown in FIG. 1, the image processing method can include the following steps:

[0149] S601, in response to a photographing operation, the camera device driver acquires an original image.

[0150] The original image can include one or more of the following: long frames, short frames, or normal frames. The exposure time of any frame in the long frame is greater than the exposure time of any frame in the normal frame. The exposure time of any frame in the normal frame is greater than the exposure time of any frame in the short frame.

[0151] It can be understood that the original image can include at least two image frames collected at the same time. The exposure times of the at least two image frames are different, so that the electronic device can generate a photographing image with an HDR effect based on image fusion of the at least two image frames with different exposure times in an HDR photographing scene.

[0152] In response to the photographing operation, it includes: in response to the user's trigger operation for the photographing control in the portrait photographing function, or in response to the user's trigger operation for the photographing control in the HDR photographing function, or in response to the user's trigger operation for the photographing control in the large aperture photographing function, or in response to the user's trigger operation for the recording control in the video recording function, etc. The application embodiment does not limit the applicable scene of the image processing method.

[0153] S602, the camera algorithm library acquires the original image from the camera device driver.

[0154] Specifically, the camera hardware abstraction layer can acquire the original image from the camera device driver, and the camera algorithm library can acquire the original image from the camera hardware abstraction layer.

[0155] S603, the camera algorithm library processes the original image into a clear image.

[0156] The process of the camera algorithm library processing the raw image into a clear image can be: the camera algorithm library can obtain at least two images with different exposure times at the same time in the raw image, and perform image pre-processing on any frame image in the at least two images. Further, the camera algorithm library can perform image fusion on at least two images obtained at the same time after image pre-processing, to obtain a clear image. The clear image can also be referred to as a first image.

[0157] It can be understood that, since the clear image is obtained by image fusion based on at least two images with different exposure times, the clear image has an HDR effect.

[0158] The image pre-processing can include one or more of the following: bad pixel correction processing, RAW domain noise reduction processing, black level correction processing, optical shading correction processing / automatic white balance processing, color interpolation processing, color correction processing, or gamma correction processing, etc., which are not limited in the embodiments of the present application.

[0159] In S604, the camera algorithm library performs luminance calculation on the clear image to obtain a luminance image.

[0160] The luminance image is used to represent the luminance information of the image. The luminance image includes the luminance value of each pixel point, which is used to indicate the luminance or brightness of the pixel point. In some embodiments, the luminance image can be understood as a grayscale image, and the luminance value of the pixel point can be understood as the grayscale value of the pixel point.

[0161] In a possible implementation, when the clear image is represented by RGB space, the clear image includes information of three color channels of green (G), red (R) and blue (B). The camera algorithm library can obtain the luminance value of each pixel point in the clear image by one or more of the information of the three color channels, to obtain the luminance image.

[0162] In some embodiments, the luminance of the clear image is represented by the information of the green channel corresponding to each pixel in the clear image, to obtain the luminance image. It can be understood that, since the human eye is more sensitive to the color of the green band, the information of the green channel in the clear image is usually more, and can more accurately reflect the luminance of the clear image.

[0163] In other embodiments, each of the three color channels is provided with a weight, and the luminance value of each pixel point in the clear image is obtained by weighted calculation of the information of the three color channels corresponding to each pixel in the clear image, to obtain the luminance image.

[0164] In a possible implementation, when the clear image is represented by HSV space, the clear image includes information of three attribute channels of hue, saturation and value. The camera algorithm library can obtain the luminance value of each pixel in the clear image by one or more of the information of the three attribute channels, and obtain the luminance image.

[0165] In some embodiments, the luminance of each pixel in the clear image is represented by the numerical value of the luminance attribute corresponding to the pixel, and the luminance image is obtained.

[0166] In a possible implementation, when the clear image is represented by YUV space, the clear image includes three components: Y value, U value and V value. The Y value represents luminance (or luma), which can also be referred to as a gray value; and the U value and the V value are used to represent chrominance (or chroma) and are used to describe the color and saturation of the image and specify the color of the pixel. The camera algorithm library can represent the luminance of each pixel in the clear image by the Y value corresponding to the pixel, and obtain the luminance image.

[0167] It can be understood that the RGB space, the HSV space and the YUV space can be converted into each other by a series of mathematical calculations. When the clear image is represented by RGB space, the clear image can be first converted into HSV space, and the luminance image is obtained according to one or more of the information of the three attributes. When the clear image is in HSV space, the clear image can be first converted into RGB space, and the luminance image is obtained according to one or more of the information of the three color channels. The specific calculation manner of the luminance image is not limited in the embodiments of the present application.

[0168] S605, the camera algorithm library identifies the light spot region in the luminance image, and obtains a light spot binary image.

[0169] The camera algorithm can identify the light spot region in the luminance image according to the light spot threshold, and obtain the light spot binary image A. Specifically, taking 0 and 1 included in the light spot binary image as an example, when the luminance of a pixel is greater than or equal to the light spot threshold, the pixel is located in the light spot region, and the corresponding value is 1; when the luminance of a pixel is less than the light spot threshold, the pixel is located in the non-light spot region, and the corresponding value is 0.

[0170] In the embodiments of the present application, the light spot threshold value can be 230, 235 or any value, which is not limited here. In some embodiments, the light spot threshold value is positively correlated with the brightness average value of the brightness image. When the brightness average value is larger, the light spot threshold value is larger. When the brightness average value is smaller, the light spot threshold value is smaller. The brightness average value of the brightness image can be the average value of the brightness values of all pixel points in the brightness image. In this way, the light spot threshold value is adjusted according to the overall brightness of the image, reducing the situation that the light spot threshold value is too high or too low, and improving the accuracy of light spot area recognition.

[0171] On the basis of the above-mentioned embodiments, the electronic device can also expand the connected domain in the light spot binary image A according to the brightness image and a fixed value A to obtain a light spot binary image B. The connected domain can be understood as a region composed of pixel points with the same pixel value and adjacent to each other in the image.

[0172] In some embodiments, when the difference between the brightness value of the pixel point at the boundary of the connected domain in the light spot binary image A and the brightness value of the adjacent pixel point is less than the fixed value A, the adjacent pixel point is located in the light spot region, and the value corresponding to the adjacent pixel point in the light spot binary image B is modified from 0 to 1. When the difference between the brightness value of the pixel point at the boundary of the connected domain in the light spot binary image A and the brightness value of the adjacent pixel point is greater than or equal to the fixed value A, the adjacent pixel point is not located in the light spot region, and the value corresponding to the adjacent pixel point in the light spot binary image B is 0. The fixed value A is 15, 20 or any value greater than zero, which is not limited here. In this way, the omission of the pixel points corresponding to the light spot region can be reduced, the situation that the light spot is not accurately recognized can be reduced, and the situation that the area of the light spot region is small can be reduced.

[0173] In other embodiments, when the difference between the brightness value of the pixel point adjacent to the boundary of the connected domain in the light spot binary image A and the light spot threshold value is less than the fixed value A, the adjacent pixel point is located in the light spot region, and the value corresponding to the adjacent pixel point in the light spot binary image B is modified from 0 to 1. When the difference between the brightness value of the pixel point adjacent to the boundary of the connected domain in the light spot binary image A and the light spot threshold value is greater than or equal to the fixed value A, the adjacent pixel point is not located in the light spot region, and the value corresponding to the adjacent pixel point in the light spot binary image B is 0. The fixed value A is 15, 20 or any value greater than zero, which is not limited here. In this way, the omission of the light spot can be reduced, the situation that the light spot is not accurately recognized can be reduced, and the situation that the area of the light spot region is small can be reduced.

[0174] For example, the brightness values of the pixel points in the brightness image are shown in Table 1, the light spot threshold value is 230, and the fixed value A is 20. The light spot binary image A and the light spot binary image B can be shown in Table 2 and Table 3, respectively.

[0175] Table 1: Brightness values of pixel points in the brightness image

[0176] 21 22 22 25 30 32 32 39 49 65 71 20 22 23 27 38 69 76 91 66 45 39 21 22 24 31 59 180 239 255 192 83 43 22 23 26 37 91 232 255 255 255 168 75 22 24 28 45 133 255 255 255 255 226 142 23 24 27 36 92 227 255 255 244 216 148 24 25 27 32 54 100 170 194 178 117 79 24 28 27 30 34 42 50 51 44 38 34 24 43 54 60 61 42 34 30 30 28 25 32 71 178 227 205 98 46 32 29 26 25

[0177] Table 2, the pixel point corresponding to the value of the 8th column of the 3rd row in the spot binary graph A, the pixel point corresponding to the value of the 7th to 9th columns of the 4th row, and the pixel point corresponding to the value of the 6th to 9th columns of the 5th row are all 1, and the rest are 0. The connected domain includes the pixel point of the 8th column of the 3rd row, the pixel point of the 7th to 9th columns of the 4th row, and the pixel point of the 6th to 9th columns of the 5th row.

[0178] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0179] As can be seen from Table 1, the luminance values corresponding to the pixel points of the 7th and 8th columns of the 3rd row, the luminance values corresponding to the pixel points of the 6th to 9th columns of the 4th row, and the luminance values corresponding to the pixel points of the 6th to 9th columns of the 5th row are all greater than the spot threshold value. The pixel points are all located in the spot region in the image.

[0180] As can be seen from Table 2, the values corresponding to the pixel point of the 8th column of the 3rd row, the values corresponding to the pixel point of the 7th to 9th columns of the 4th row, and the values corresponding to the pixel point of the 6th to 9th columns of the 5th row in the spot binary graph A in Table 2 are all 1, and the rest are 0. The connected domain includes the pixel point of the 8th column of the 3rd row, the pixel point of the 7th to 9th columns of the 4th row, and the pixel point of the 6th to 9th columns of the 5th row.

[0181] Table 3, the values corresponding to the pixel point of the 9th column of the 5th row, the values corresponding to the pixel point of the 6th and 10th columns of the 4th row are all 1.

[0182]

[0183]

[0184] As can be seen from Table 2, the boundary of the connected domain includes the pixel point of the 7th column of the 3rd row, the pixel point of the 7th and 9th columns of the 4th row, and the pixel point of the 6th and 9th columns of the 5th row; taking the difference between the luminance value of the pixel point adjacent to the boundary of the connected domain and the spot threshold value as an example, which is less than 20, compared with Table 2, the values corresponding to the pixel point of the 9th column of the 5th row, the values corresponding to the pixel point of the 6th and 10th columns of the 4th row in Table 3 are all 1.

[0185] S606, the camera algorithm library determines the spot region in the clear image according to the spot binary graph, and performs color optimization on the spot region in the clear image to obtain an optimized clear image.

[0186] In the embodiment of the present application, by improving the saturation degree corresponding to the pixel point, the color vividness of the spot region is improved, and the subsequent color dimness of the spot is reduced.

[0187] For example, taking the YUV space color coordinate system as an example, the saturation (Saturation, S) of the pixel point satisfies: Wherein, U N is the horizontal coordinate of the pixel point in the YUV space color coordinate system, and V N is the vertical coordinate of the pixel point in the YUV space color coordinate system.

[0188] For example, FIG. 7A schematic diagram of a YUV space color coordinate system is provided in some embodiments of the present application. As shown in FIG. 1, the horizontal axis is Cb, corresponding to the U value of a pixel point, and the vertical axis is Cr, corresponding to the V value of the pixel point. FIG. 7

[0189] The coordinate point of the pixel point in the YUV space color coordinate system is (U N , V N ). The saturation S can be understood as the distance between the coordinate point of the pixel point and the origin.

[0190] Taking the range of the color coordinate system as -1 to 1, and the value range of the U value of the pixel point in the YUV space and the V value of the pixel point as 0 to 255 as an example, the horizontal coordinate U N in the color coordinate system satisfies The vertical coordinate V N in the color coordinate system satisfies

[0191] In some embodiments, the saturation can be increased by ensuring that the straight line between the coordinate point and the origin is unchanged, and increasing the distance between the coordinate point (U N , V N ) and the origin. In this way, the color deviation caused by saturation adjustment can be reduced.

[0192] Specifically, the ratio A and the ratio B are the same. The ratio A is the ratio of the horizontal coordinate to the vertical coordinate of the pixel point in the YUV space color coordinate system in the optimized clear image, and the ratio B is the ratio of the horizontal coordinate to the vertical coordinate of the pixel point in the YUV space color coordinate system in the clear image. The distance between the coordinate point of the pixel point after color optimization and the origin is greater than the distance between the coordinate point of the pixel point before color optimization and the origin.

[0193] In this way, the straight line between the coordinate point of the pixel point in the YUV space color coordinate system and the origin is unchanged before and after color optimization, which can reduce the color deviation caused by color optimization.

[0194] In some embodiments, the saturation is increased by multiplying the coordinate point (U N , V N ) by a saturation enhancement factor C. The saturation enhancement factor C can be any value greater than 1. Herein, no specific limitation is made.

[0195] In some embodiments, the saturation enhancement factor C is positively correlated with the minimum value A. The minimum value A is the minimum value between the absolute value of the horizontal coordinate and the absolute value of the vertical coordinate of the pixel point in the YUV space color coordinate system.

[0196] In this way, selecting a smaller value can reduce the color deviation during color adjustment of the pixel point and improve the color optimization effect.

[0197] ​In some embodiments, the saturation enhancement factor C is determined by a pre-set Gamma coefficient γ. Specifically, the saturation enhancement factor C is the minimum value between the absolute value of the horizontal coordinate U of the pixel point in the color coordinate system and the absolute value of the Gamma coefficient mapped value of the vertical coordinate V of the pixel point. N the absolute value of the Gamma coefficient mapped value of the vertical coordinate V of the pixel point. The saturation enhancement factor C satisfies: C = MIN(|U N , |V N |). γ N . γ

[0198] In some embodiments, the value range of the U value of the pixel point in the YUV space and the value range of the V value of the pixel point are both between 0 and 255. The U value before adjustment is U old ; the V value before adjustment is V old ; the horizontal coordinate U Nold of the pixel point in the color coordinate system satisfies: the vertical coordinate V Nold of the pixel point in the color coordinate system satisfies: the U value after adjustment U New satisfies: U New = C*U Nold + 128. The V value after adjustment V New satisfies: V New = C*V Nold + 128.

[0199] In the embodiments of the present application, the Gamma coefficient γ can be obtained by experiment or any method, which is not limited here. For example, the same batch of image samples can be color optimized by using different Gamma coefficients, and the optimized images corresponding to the Gamma coefficients are obtained. The Gamma coefficient corresponding to the image with the best optimization effect is selected as the pre-set Gamma coefficient γ. Alternatively, the Gamma coefficients corresponding to the images with the top N optimization effects are counted, and the Gamma coefficient with the second highest frequency is selected as the pre-set Gamma coefficient γ. The method for determining the pre-set Gamma coefficient γ is not limited in the embodiments of the present application.

[0200] In some embodiments, the color optimization is performed when the saturation of the pixel point is less than a threshold A, and the color optimization is not performed when the saturation of the pixel point is greater than or equal to the threshold A. The threshold A can be 0.3 or any value, which is not limited here.

[0201] ​​In this way, the pixel points with high saturation can not be subjected to color optimization, the calculation amount of color optimization is reduced, and the color optimization speed is improved. The pixel points with low saturation are subjected to color optimization, the phenomenon of color dimming and low saturation caused by overexposure is reduced, and the conditions of flare whitening and low saturation in the subsequent blurred image are reduced.

[0202] In S607, the camera algorithm library performs brightness optimization on each flare region in the image according to the area of each flare region in the image, to obtain an optimized brightness image.

[0203] It should be noted that the principle of flare generation is that the high-brightness region in the clear image has a higher blur weight corresponding to the brightness value, and the blur degree of this region is higher; during subsequent filtering processing, the pixel value of the high-brightness region in the clear image has a greater influence on the brightness of the pixel point in the blurred image. Therefore, in a possible design, when the area of the flare region in the image is large, the overall brightness of the flare in the blurred image can be high, and the transparency of the flare is poor; when the brightness of the flare region in the image is low, the brightness of the flare in the blurred image can be low, and the flare blur is poor.

[0204] In the embodiments of the present application, the brightness optimization is achieved by reducing the brightness value of the pixel point at the edge of the flare region and / or increasing the brightness value of the pixel point at the center of the flare region. In this way, the brightness ratio of the edge and the center of the flare region is adjusted, the contrast between the edge and the center of the flare region is higher, and the transparency of the flare in the blurred image is better.

[0205] It can be understood that reducing the brightness value of the pixel point at the edge of the flare region can reduce the corresponding blur weight of the region, reduce the condition that the overall brightness of the flare is high, and improve the transparency of the flare. For example, in the case where adjacent flares overlap, the brightness of the overlapping region in the flare is higher than the brightness of the non-overlapping region in the flare, the overlapping region is more obvious, and the transparency of the flare in the blurred image is better.

[0206] For example, FIG. 8 The schematic diagram of the blurred image corresponding to the unadjusted brightness value of the flare region and the blurred image corresponding to the adjusted brightness value of the flare region provided in the embodiments of the present application is shown.

[0207] From FIG. 8 It can be seen from FIG. 8 In the flare region 801 shown by a in FIG. 8B, the brightness of the region where the adjacent flares overlap is the same as the brightness of one of the flares, and the transparency of the flare in the flare region 801 is poor. FIG. 8 In the flare region 802 shown by b in FIG. 8B, the brightness of the region where the adjacent flares overlap is higher than the brightness of the adjacent flare, and the transparency of the flare in the flare region 802 is good.

[0208] In some embodiments, in the case that the area of the light spot region in the image is greater than threshold B, the luminance value of the pixel points at the edge of the light spot region is reduced to reduce the area of the light spot region. Threshold B can be 16, 15 or any value, which is not specifically limited here. In this way, the luminance value of the pixel points at the edge of the light spot region is not adjusted for the light spot region with a smaller area, reducing the case of light spot blur caused by the smaller luminance of the light spot region. The luminance value of the pixel points at the edge of the light spot region is adjusted for the light spot region with a larger area, improving the contrast between the edge and the center of the light spot region, and the permeability of the light spot.

[0209] In some embodiments, in the case that the area of the light spot region in the image is greater than threshold B, the luminance value of each pixel in region A is reduced. Region A is a region in the light spot region other than region B, and region B is a region corresponding to the reduced light spot region. Region B includes: the pixel points at the edge of the light spot region.

[0210] In some embodiments, the center point of region B is the same as the center point of the light spot region. It can be understood that the center points can be absolutely the same, or the center points can have a certain deviation, for example, within 1 pixel point. In this way, the center of the light spot region remains unchanged before and after reduction, reducing the change of the position of the light spot in the subsequent blurred image.

[0211] In the embodiments of the present application, the center of the light spot region can be understood as the average value of the positions of the pixel points in the light spot region.

[0212] In some embodiments, the shape of region B is the same as the shape of the center point of the light spot region. In this way, the shape of the light spot region is maintained, reducing the shape change of the light spot in the blurred image.

[0213] In the embodiments of the present application, region B can be determined in various ways. For example, reducing to a preset area, reducing the area of the light spot region by a certain ratio, reducing the distance from the center point of the light spot region to the boundary of the light spot region by M pixel points, etc. The implementation of the reduction of the light spot region is not specifically limited in the embodiments of the present application. The reduced light spot region is not specifically limited.

[0214] In a possible implementation, the preset area can be 16 pixels*16 pixels, 15 pixels*15 pixels or any value, which is not specifically limited here. In this way, the reduction to the preset area makes the area of the reduced light spot region fixed, has a smaller calculation amount and is easy to implement.

[0215] It can be understood that, generally, the light spot area corresponding to the point light source is a circular area. In some embodiments, the electronic device can determine the area of the light spot area through the center point of the light spot area and the radius of the light spot area. Adaptively, the manner of reducing to the preset area can also be understood as reducing to the preset radius. For example, assuming that the radius of the light spot area before reduction is 7 pixels and the preset radius is 3 pixels, the light spot area after reduction can be a circular area with the center of the light spot area as the center and a radius of 3 pixels.

[0216] Alternatively, it can also be understood that the distance of each pixel point in the region A to the center of the light spot area is greater than the preset radius; and the distance of each pixel point in the region B to the center of the light spot area is less than or equal to the preset radius.

[0217] In the possible implementation manner two, the certain ratio can be 0.4, 0.6 or any value, which is not limited here. In this way, the manner of reducing the area of the light spot area by the certain ratio can make the size of the light spot area after reduction positively correlated with the size of the light spot area before reduction, and the size of the light spot area is more consistent with the brightness rule in the actual scene, so that the light spot in the blurred image is consistent with the high-brightness object in the actual scene.

[0218] For example, assuming that the light spot area is a circular area, the manner of reducing the area of the light spot area by the certain ratio can also be understood as reducing the radius of the light spot area by the certain ratio. For example, assuming that the radius of the light spot area before reduction is 7 and the certain ratio is 0.5, the light spot area after reduction can be a circular area with the center of the light spot area as the center and a radius of 3.5.

[0219] Alternatively, it can also be understood that the ratio C is greater than the certain ratio and the ratio D is less than or equal to the certain ratio; the ratio C is the ratio of the distance of each pixel point in the region A to the center of the light spot area to the distance of the boundary of the light spot area to the center of the light spot area; and the ratio D is the ratio of the distance of each pixel point in the region B to the center of the light spot area to the distance of the boundary of the light spot area to the center of the light spot area.

[0220] In the possible implementation manner three, M can be an integer greater than 1. In this way, the manner of reducing M pixel points from the distance of the center point of the light spot area to the boundary of the light spot area can make the size of the light spot area after reduction positively correlated with the size of the light spot area before reduction, and the size of the light spot area is more consistent with the brightness rule in the actual scene, so that the light spot in the blurred image is consistent with the high-brightness object in the actual scene.

[0221] The above manner of determining the region A and the region B is described, and the brightness adjustment of each pixel in the region A is described below.

[0222] In some embodiments, the brightness value of each pixel in the region A is adjusted to a brightness value D. The brightness value D is less than the light spot threshold.

[0223] The luminance value D can be a preset value, for example, 50 or 80, etc., which is not limited herein. The luminance value D can also be the minimum value among the luminance values of the pixel points in the region C. The region C can be a circular region with the center of the light spot region as the center and the threshold value D as the radius. The threshold value D can be 10, 15, or any numerical value, which is not limited herein.

[0224] In this way, the luminance value selected near the pixel point can reduce the mismatch between the luminance value of the region A and the luminance near the light spot region, reduce the over-dark or over-bright situation caused by the mismatch, and improve the display effect of the light spot in the subsequent blurred image. The specific value and specific confirmation method of the luminance value D are not limited herein.

[0225] The above embodiment describes the adjustment of the pixel points on the edge of the light spot region. The adjustment of the pixel points in the center of the light spot region and around the center is described below.

[0226] On the basis of the above embodiment, in the case that the luminance value of the center of the light spot region in the image is less than the threshold value C, the luminance of the center point of the light spot region and the N pixel points around the center point is improved to the luminance value E. N is an integer greater than 0. N can be 1, 4, 9, or any numerical value, which is not limited herein. In this way, improving the luminance value of the pixel points of the light spot region can improve the corresponding blurring weight of the high-light region, reduce the blur and poor visual effect caused by the low luminance of the light spot, and improve the clarity of the light spot.

[0227] In some embodiments, the luminance value of the pixel points in the region D is improved, taking the light spot region as a circular region as an example. The region D is a circular region with the center point of the light spot region as the center and the threshold value D as the radius.

[0228] In the embodiment of the present application, the luminance value E can be a fixed value, for example, 255, 240, etc. The luminance value E can also be inversely proportional to the distance A, which is the distance between the pixel point and the center point of the light spot region. The greater the distance A, the smaller the luminance value E; the smaller the distance A, the greater the luminance value E. In this way, the luminance of the light spot region is more consistent with the luminance variation law of the high-light object, improving the display effect of the light spot.

[0229] For example, the luminance value E and the distance A satisfy a preset quadratic formula. In this way, the luminance of the light spot region satisfies the luminance variation law of the high-light object, improving the display effect of the light spot.

[0230] S608, the camera algorithm library performs depth calculation on the clear image to obtain a depth image.

[0231] The depth image can include depth information of any pixel point, and the depth information can represent the distance from each point in the scene to the camera plane, and can reflect the geometric shape of the visible surface in the scene. The depth information of any pixel point in the depth image can be determined based on monocular depth estimation, binocular depth estimation, or depth learning-based depth estimation, which is not limited in the embodiments of the present application.

[0232] The depth image is used to calculate the blurring weight in the clear image. It can also be understood that the blurring weight is related to the depth information of each pixel point in the depth image. For example, the greater the depth corresponding to the pixel point, the greater the blurring weight, and the more blurred the pixel point; the smaller the depth corresponding to the pixel point, the smaller the blurring weight, and the clearer the pixel point.

[0233] In some embodiments, the greater the difference between the depth corresponding to the pixel point and the depth corresponding to the focus, the greater the blurring weight, and the more blurred the pixel point; the smaller the difference between the depth corresponding to the pixel point and the depth corresponding to the focus, the smaller the blurring weight, and the clearer the pixel point.

[0234] The focus can be determined by the electronic device in response to an operation for indicating the focus, or can be determined by the electronic device based on the portrait or object in the clear image. For example, in the portrait shooting function of the camera, the electronic device can realize the detection of the shooting object, such as when the portrait is detected to include a portrait, the position where the portrait is located is set as the focus, the focusing of the portrait is completed, and the portrait is clearly visible in the picture. The focus is not limited in the embodiments of the present application.

[0235] In some embodiments, the electronic device can distinguish the foreground and the background based on the focus and the depth image. For example, the depth at the focus can be a first depth, the depth of field at the focus can be set to 0, and the depth of field at other positions in the depth of field image can be the difference between the depth at the other positions and the first depth, to obtain the foreground and the background. The foreground can include the clear range before the focus, and the background can include the clear range after the focus.

[0236] S609, the camera algorithm library looks up the weight table based on the optimized brightness image and the depth image, to obtain the blurring weight corresponding to each pixel in the clear image.

[0237] In some embodiments, the weight table is used to indicate the corresponding relationship between the brightness value, the depth value and the blurring weight. The brightness value and the depth value are positively correlated with the blurring weight.

[0238] In another embodiment, the weight table is used to indicate the corresponding relationship between the brightness value and the brightness coefficient, and the corresponding relationship between the depth value and the depth coefficient.

[0239] In the embodiments of the present application, the brightness value is positively correlated with the blurring weight. The greater the brightness value, the greater the blurring weight; the smaller the brightness value, the smaller the blurring weight.

[0240] In some embodiments, the blurring weight is also affected by the depth value corresponding to the pixel. The depth value corresponding to the pixel is positively correlated with the blurring weight. The greater the depth value, the greater the weight; the smaller the depth value, the smaller the weight. In this way, the weight of the pixel is also related to the depth value of the pixel, so that the farther the distance of the region in the blurred image, the higher the degree of blurring, and the spatial sense of the blurred image is improved.

[0241] For example, the blurring weight can be the product of weight A and weight B. Weight A is the weight obtained by looking up the weight table according to the brightness value; and weight B is the weight obtained by looking up the weight table according to the depth value.

[0242] For example, the blurring weight can be the sum of weight A and weight B. Weight A is the weight obtained by looking up the weight table according to the brightness value; and weight B is the weight obtained by looking up the weight table according to the depth value.

[0243] The weight table is only an example, and the corresponding relationship between the brightness value, the depth value and the blurring weight can also be embodied in other arbitrary forms, such as a formula, etc.

[0244] In some embodiments, the electronic device can also not perform S608. Adaptively, the camera algorithm library looks up the weight table according to the optimized brightness image in S609 to obtain the blurring weight corresponding to each pixel in the clear image. In some embodiments, the brightness coefficient of each pixel can also be used as the blurring weight corresponding to each pixel.

[0245] S610, the camera algorithm library performs filtering processing on the optimized clear image according to the blurring weight to obtain a blurred image.

[0246] S610 can include: the camera algorithm library adjusts the pixel value of the optimized clear image according to the blurring weight to obtain a weighted image. The camera algorithm library performs filtering processing on the weighted image to obtain a blurred image.

[0247] The pixel value of each pixel in the weighted image is the product of the pixel value of each pixel in the optimized clear image and the blurring weight corresponding thereto.

[0248] In the embodiments of the present application, the pixel value can be understood as the value of each channel in the image. For example, if the image is represented by RGB space, the pixel value includes the values corresponding to the green, red and blue color channels. For example, if the image is represented by HSV space, the pixel value includes the values corresponding to the hue, saturation and brightness attribute channels. For example, if the image is represented by YUV space, the pixel value includes the Y value, the U value and the V value.

[0249] The filtering processing can be a Gaussian blur filtering, a mean blur filtering, etc., which is not limited here.

[0250] For example, in the Gaussian blur filtering, for any pixel point, the pixel value of the pixel point and the pixel values of the pixel points within the blurring radius of the pixel point are weighted and averaged to obtain the pixel value of the pixel point in the blurred image. It can also be understood that the pixel value of the pixel point and the pixel values of the pixel points within the blurring radius of the pixel point are added to obtain a sum of the pixel values, and then the sum is divided by the sum of the filtering weights of the pixel point and the pixel points within the blurring radius of the pixel point to obtain the pixel value of the pixel point in the blurred image. The filtering weights of the pixel point and the pixel points within the blurring radius of the pixel point satisfy the density function of the Gaussian distribution (normal distribution).

[0251] It can be understood that the Gaussian blur considers the influence of the distance between each pixel point within the blurring radius and the center pixel point, and can retain some image details, and the processing effect is good.

[0252] In the embodiments of the present application, the blurring radius can be 5 pixels, or 10 pixels or any value, which is not limited here. The filtering weight is used to indicate the weight corresponding to the pixel point in the filtering processing.

[0253] For example, in the mean blur filtering, for any pixel point, the average value of the pixel value of the pixel point and the pixel values of the pixel points within the blurring radius of the pixel point is obtained to obtain the pixel value of the pixel point in the blurred image. It can be understood that the calculation of the mean blur is relatively simple and fast.

[0254] The blurred image can include a clear foreground and a blurred background. When the clear image is displayed as shown in FIG. 1A , the blurred image obtained after the blurring processing of the clear image can be a portrait foreground clear and background blurred image, and the blurred image can be the image displayed in the interface shown in FIG. 1C .

[0255] After the camera algorithm library determines the blurred image, the camera algorithm library can store the blurred image to the gallery through the steps shown in S612-S613, and process the blurred image into a thumbnail and display the thumbnail through S614-S616. The present application does not limit the sequence between the above two processes.

[0256] S612, the gallery obtains the blurred image from the camera algorithm library.

[0257] Specifically, the camera access interface (or first interface) can obtain the thumbnail from the camera algorithm library, and then the camera can obtain the thumbnail from the camera access interface (or first interface). Wherein, the first interface (in the embodiment of the present application, the first interface is the camera access interface) can be a camera access interface of the camera, or a camera access interface of the gallery.FIG. 5 (Not shown in the image) can be used to establish a data path between the image library and the camera algorithm library. The type of the first interface is not limited in the embodiments of this application.

[0258] S613, Gallery stores blurred images.

[0259] After S613, electronic devices can display blurred images in response to a user opening the gallery application.

[0260] S614, the camera algorithm library processes blurred images into thumbnails.

[0261] Camera algorithm libraries can process blurred images into thumbnails using thumbnail processing methods such as sampling or neural networks, and this application does not limit this.

[0262] S615: The camera retrieves thumbnails from the camera algorithm library.

[0263] Specifically, the camera access interface can obtain thumbnails from the camera algorithm library, and then the camera can obtain the thumbnails from the camera access interface.

[0264] S616: The camera accesses the display screen to show thumbnails.

[0265] For example, the camera can FIG. 1B The thumbnail is displayed in the lower left corner of the interface shown.

[0266] Understandable FIG. 6 The sequential relationship between the steps described herein is merely an example and should not be construed as limiting the embodiments of this application. FIG. 6 In the illustrated embodiment, the optimization of the light spot in the image is achieved by adjusting both the saturation of the pixels in the light spot region and the brightness of the pixels in the light spot region.

[0267] In some embodiments, the electronic device may also optimize the blurring of the light spot in the image simply by adjusting the saturation of the pixels in the light spot region of the image. For example, the electronic device may omit S607. Adaptably, the optimized brightness image in S609 can be replaced with the brightness image in S604.

[0268] In some embodiments, the electronic device may also optimize the blurring of the light spots in the image simply by adjusting the brightness values ​​of the pixels in the light spot region of the image. For example, the electronic device may omit S606. Adaptably, the optimized sharp image in S609 can be replaced with a sharp image.

[0269] FIG. 6In the illustrated embodiment, by increasing the saturation of the pixel points in the bright spot area of the clear image, the dark situation of the bright spot after filtering is reduced, and the color effect of the bright spot in the bright spot area is improved. By increasing the brightness of the center of the bright spot area and reducing the brightness of the edge of the bright spot area, the weight corresponding to the bright spot area is adjusted to improve the shape and transparency of the bright spot. In addition, FIG. 6 In the illustrated embodiment, when the clear image is blurred by S604-S610, the brightness of the clear image as a whole does not need to be stretched, which can reduce the overexposure of the clear image and reduce the poor display effect of the bright spot caused by overexposure. FIG. 6 In the illustrated embodiment, when the filtering calculation is performed, the natural bright spot can be generated, the high-light object does not need to be segmented from the clear image for separate filtering calculation, the subsequent fusion processing is reduced, and the computing resources are saved.

[0270] The above FIG. 6 to FIG. 8 The image processing method in the camera shooting scene is described. Next, the image processing method in the gallery editing scene is described. FIG. 9 The image processing method in the gallery editing scene is described.

[0271] Exemplarily, FIG. 9 A flowchart of an image processing method provided by an embodiment of the present application is shown. Taking the editing scene of a gallery application as an example, in the FIG. 9 In the corresponding embodiment, the electronic device can include a gallery, a gallery access interface, a camera algorithm library, a gallery hardware abstraction layer, and a TP driver. The functions of any module can be referred to FIG. 5 In the corresponding embodiment, details are not repeated here. As shown in FIG. 9 As shown, the image processing method can include the following steps: the gallery application displays an interface A, and the interface A includes a clear image and a blurring control.

[0272] S901, in response to a trigger operation on the blurring control, the gallery application calls the camera algorithm library through the gallery access interface to blur the clear image.

[0273] S902, the camera algorithm library performs brightness calculation on the clear image to obtain a brightness image.

[0274] S903, the camera algorithm library identifies the bright spot area in the brightness image to obtain a bright spot binary image.

[0275] S904, the bright spot area in the clear image is determined according to the bright spot binary image, and the color of the bright spot area in the clear image is optimized to obtain an optimized clear image.

[0276] S905, the brightness of the pixels in each bright spot area in the image is optimized according to the area of each bright spot area in the image to obtain an optimized brightness image.

[0277] S906, the camera algorithm library performs depth calculation on the clear image to obtain a depth image.

[0278] S907, the optimized brightness image and the depth image are used to search a weight table to obtain a blurring weight corresponding to the clear image.

[0279] S908, the camera algorithm library performs filtering processing on the optimized clear image according to the blurring weight to obtain a blurred image.

[0280] S909, the gallery obtains the blurred image from the camera algorithm library.

[0281] S910, the gallery displays the blurred image.

[0282] The above S903 to S909 can refer to the corresponding step description in FIG. 6 , which will not be repeated here.

[0283] In this way, the blurring processing of the image in the gallery can be implemented, and the display effect of the light spot in the blurred image can be improved. Specifically, by increasing the saturation of the pixel points in the light spot area in the clear image, the light spot darkening condition after filtering is reduced, and the color effect of the light spot in the light spot area is improved. By increasing the center brightness of the light spot area and reducing the brightness of the edge of the light spot area, the light spot shape and light spot transparency are improved.

[0284] In addition, FIG. 9 In the embodiment shown, when the clear image is processed, the brightness of the clear image as a whole does not need to be stretched, which can reduce the overexposure of the clear image and reduce the poor display effect of the light spot caused by overexposure. FIG. 9 In the embodiment shown, the natural light spot can be generated when filtering calculation is performed, and the high-light object in the clear image does not need to be segmented and processed separately, which reduces the subsequent fusion processing and saves the computing resources.

[0285] Exemplarily, FIG. 10 A flowchart of an image processing method provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the method comprises the following steps. FIG. 10

[0286] S1001, a clear image is obtained.

[0287] Taking a photographing scene as an example, in response to a photographing operation, the clear image is obtained, and S1002 to S1008 are executed. Taking a gallery editing scene as an example, in response to an operation for indicating that an image is selected, the clear image is obtained. In response to a triggering operation on a blurring control, S1002 to S1008 are executed. The triggering condition for obtaining the clear image is not limited in the embodiment of the present application.

[0288] ​S1002, acquire the brightness image corresponding to the clear image.

[0289] S1002 can refer to the description of the corresponding step in S604.

[0290] S1003, identify the light spot area in the brightness image, and obtain a light spot binary image.

[0291] S1004, color optimization of the light spot area in the clear image according to the light spot binary image, and obtain the optimized clear image.

[0292] In the embodiments of the present application, the saturation of each pixel point in the light spot area in the clear image can be adjusted according to the light spot binary image. The specific adjustment method can refer to the corresponding description in the above FIG. 6 .

[0293] S1005, brightness optimization of the light spot area in the clear image according to the light spot binary image, and obtain the optimized brightness image.

[0294] In the embodiments of the present application, the brightness value of each pixel point in the light spot area in the brightness image can be adjusted according to the light spot binary image. The specific adjustment method can refer to the corresponding description in the above FIG. 6 .

[0295] S1006, according to the brightness value of each pixel in the optimized brightness image, the weight table is looked up to obtain the weight corresponding to each pixel.

[0296] S1007, multiply the weight corresponding to each pixel with the background area in the optimized clear image to obtain a weighted clear image.

[0297] S1008, filter calculation is performed on the weighted clear image to obtain a blurred image.

[0298] S1006 to S1008 can refer to the description of the corresponding steps in the above FIG. 6 .

[0299] In this way, the electronic device can identify the light spot area in the image, and perform color optimization, transparency optimization processing, etc. on the light spot area in the image; perform blurring processing on the image after optimization processing, to improve the display effect of the light spot in the blurred image. In addition, the light spot area does not need to be segmented from the image for processing, reducing the steps of subsequent fusion processing, simple steps, easy to implement.

[0300] It should be noted that the module names involved in the embodiments of the present application can be defined as other names, as long as the functions of the modules can be realized, and the names of the modules are not limited.

[0301] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0302] The image processing method of the present application embodiments has been described above. The apparatus for performing the above method provided in the present application embodiments is described below. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced by each other, and the related apparatus provided in the present application embodiments can perform the steps in the above method.

[0303] like FIG. 11 As shown, FIG. 11 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device may be an electronic device in the embodiment of this application, or it may be a chip or chip system within an electronic device.

[0304] like FIG. 11 As shown, the image processing apparatus can be used in communication equipment, circuits, hardware components, or chips. The image processing apparatus includes a display unit 1101 and a processing unit 1102. The display unit 1101 supports the display steps performed by the display method; the processing unit 1102 supports the information processing steps performed by the image processing apparatus.

[0305] In one possible implementation, the image processing apparatus may further include a communication unit 1103, which supports the image processing apparatus in performing steps such as receiving or sending messages.

[0306] The image processing apparatus described in the embodiments of this application may include all of the following: FIG. 11 The units described in the corresponding embodiments.

[0307] Specifically, the processing unit 1102 can be integrated with the display unit 1101, and the processing unit 1102 and the display unit 1101 may communicate with each other.

[0308] In one possible implementation, the image processing apparatus may further include a storage unit 1104. The storage unit 1104 may include one or more memories, which may be devices in one or more devices or circuits used for storing programs or data.

[0309] The storage unit 1104 can exist independently, and is connected to the processing unit 1102 through a communication bus. The storage unit 1104 can also be integrated with the processing unit 1102.

[0310] Taking the image processing apparatus as an example of a chip or a chip system of an electronic device in the embodiments of the present application, the storage unit 1104 can store computer execution instructions of a method of the electronic device, so that the processing unit 1102 executes the method of the electronic device in the above embodiments. The storage unit 1104 can be a register, a cache or a random access memory (RAM), etc. The storage unit 1104 can be integrated with the processing unit 1102. The storage unit 1104 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 1104 can be independent of the processing unit 1102.

[0311] In a possible implementation, the image processing apparatus can further include a communication unit 1103. The communication unit 1103 is configured to support the image processing apparatus to interact with other devices. For example, when the image processing apparatus is an electronic device, the communication unit 1103 can be a communication interface or an interface circuit. When the image processing apparatus is a chip or a chip system in the electronic device, the communication unit 1103 can be a communication interface. For example, the communication interface can be an input / output interface, a pin or a circuit, etc.

[0312] The apparatus of the present embodiment can be used to perform the steps performed in the above method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0313] The image processing method provided by the embodiments of the present application can be applied to an electronic device with image processing function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be described here.

[0314] The embodiments of the present application provide an electronic device, which includes one or more processors and a memory. The memory is coupled to the one or more processors, and the memory is configured to store computer program codes including computer instructions. The one or more processors invoke the computer instructions to enable the electronic device to perform the above method.

[0315] The embodiments of the present application provide a chip or a chip system. The chip or the chip system includes one or more processors configured to invoke computer instructions to enable the electronic device to perform the technical solutions in the above embodiments. The implementation principles and technical effects are similar to those of the above related embodiments, which will not be described here.

[0316] The embodiments of the present application further provide a computer readable storage medium. The computer readable storage medium includes computer instructions, when the computer instructions are executed on an electronic device, cause the electronic device to perform the above method. The method described in the above embodiments can be realized by software, hardware, firmware or any combination thereof, in whole or in part. If realized in software, the functions can be stored as one or more instructions or codes on a computer readable medium or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can carry computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0317] In a possible implementation, the computer readable medium can include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is targeted to carrying the desired program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is properly referred to as a computer readable medium. For example, if software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of medium. As used herein, magnetic disks and optical disks include compact disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually reproduce data magnetically and optical disks reproduce data optically with laser. The above combinations should also be included in the scope of computer readable medium.

[0318] The embodiments of the present application provide a computer program product, which includes computer program code, when the computer program code is executed, causes a computer to perform the above method.

[0319] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0320] The above detailed description has further explained the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a first image; adjusting pixel values of the first image to obtain a second image; wherein a first ratio is greater than a second ratio; the first ratio is a ratio of a pixel value of a first pixel point in a light spot region in the first image to a pixel value of a second pixel point in the light spot region in the first image, and the second ratio is a ratio of the pixel value of the first pixel point in the second image to the pixel value of the second pixel point in the second image; the first pixel point comprises a pixel point at an edge of the light spot region in the image, and the second pixel point comprises a pixel point corresponding to a center point of the light spot region and N pixel points around the pixel point, wherein N is an integer greater than 0; performing filtering processing on the second image to obtain a blurred image.

2. The method of claim 1, wherein, The adjusting of the pixel values of the first image to obtain the second image comprises: adjusting luminance values of the light spot region in the first image to obtain a third image; a luminance value of the first pixel point in the third image is less than a luminance value of the first pixel point in the first image, and / or a luminance value of the second pixel point of the light spot region in the third image is greater than a luminance value of the second pixel point of the light spot region in the first image; obtaining weights of the pixel points in the third image by using the luminance values of the pixel points in the third image and a corresponding relationship; the corresponding relationship is a relationship between the luminance values and the weights; multiplying pixel values of the pixel points in the first image by corresponding weights to obtain the second image.

3. The method of claim 2, wherein, The corresponding relationship comprises a relationship among the luminance values, depth values and the weights; The obtaining of the weights of the pixel points in the third image by using the luminance values of the pixel points in the third image and the corresponding relationship comprises: obtaining the weights of the pixel points in the third image by using the luminance values of the pixel points in the third image, depth values of the pixel points in the first image and the corresponding relationship.

4. The method according to claim 2 or 3, characterized in that, The adjusting of the luminance values of the light spot region in the first image to obtain the third image comprises: in a case where an area of a first light spot region in the first image is greater than a first threshold value, adjusting a luminance value of a first pixel point in the first light spot region to a first value to reduce the area of the first light spot region; the first value is less than a light spot threshold value used to determine the light spot region in the first image.

5. The method of claim 4, wherein, The adjusting of the luminance value of the first pixel point in the first light spot region to the first value comprises: performing expansion on the first light spot region to obtain a first region, the first region being greater than the first light spot region; counting luminance values of the pixel points in the first region; adjusting the luminance value of the first pixel point in the first light spot region to a minimum value among the luminance values of the pixel points in the first region.

6. The method according to claim 5, wherein the first pixel point is a pixel point in the first light spot region except for a first region, and distances of the pixel points in the first region to a center point of the first light spot region are less than or equal to a second threshold value.

7. The method according to any one of claims 2-6, wherein In a case where a luminance value of a center point of a second light spot region in the first image is less than a third threshold value, adjusting a luminance value of a second pixel point in the second light spot region to a second value, the second value being greater than or equal to the third threshold value.

8. The method of claim 7, wherein, the second value is positively correlated with a first distance, the first distance being negatively correlated with a distance between the second pixel point and a center point in the second light spot region.

9. The method according to any one of claims 1 to 8, characterized in that, the adjusting the pixel value of the first image to obtain a second image comprises: adjusting a saturation of a pixel point in a light spot region in the first image to obtain a fourth image, wherein the saturation of the pixel point in the light spot region in the fourth image is greater than the saturation of the pixel point in the light spot region in the first image; adjusting a pixel value of the fourth image to obtain the second image.

10. The method of claim 9, wherein, a coordinate point of a pixel point in a YUV space color coordinate system comprises: an abscissa corresponding to a U value of the pixel point and an ordinate corresponding to a V value of the pixel point; the adjusting the saturation of the pixel point in the light spot region in the first image comprises: adjusting a coordinate point of a pixel point in a light spot region in the first image in a YUV space color coordinate system to adjust the saturation of the pixel point; wherein a ratio of an ordinate to an abscissa of the pixel point in the light spot region in the YUV space color coordinate system in the fourth image is the same as a ratio of an ordinate to an abscissa of the pixel point in the light spot region in the YUV space color coordinate system in the first image; the saturation of the pixel point is positively correlated with a second distance in the YUV space color coordinate system, the second distance being a distance between the coordinate point of the pixel point in the YUV space color coordinate system and an origin.

11. The method of claim 10, wherein, a third ratio value is positively correlated with a third value; the third ratio value is a ratio of an ordinate of the pixel point in the light spot region in the fourth image to an ordinate of the pixel point in the light spot region in the first image; the third value is a minimum value between an absolute value of the abscissa and an absolute value of the ordinate of the pixel point in the YUV space color coordinate system.

12. The method of claim 11, wherein, The third ratio satisfies: C=MIN(|U N | γ ,|V N | γ ), wherein the C is the third value, the γ is a gamma coefficient, the U N is an abscissa of a pixel point in a flare region in the first image in a YUV space color coordinate system, and the V N is an ordinate of the pixel point in the flare region in the first image in the YUV space color coordinate system.

13. The method of any one of claims 10-12, wherein, in a case where a saturation corresponding to a third pixel point of the light spot region in the first image is less than or equal to a fourth threshold value, the saturation of the third pixel point is increased.

14. The method according to any one of claims 1 to 13, characterized in that, Before the adjusting the first image, the method further comprises: obtaining a first light spot map of the first image according to luminance values of each pixel point in the first image and a light spot threshold value, a connected domain in the first light spot map is used to indicate a light spot region in the first image, and a pixel point in the connected domain of the first light spot map corresponds to a pixel point in the first image with a luminance value greater than or equal to the light spot threshold value.

15. The method according to any one of claims 1 to 14, characterized in that, after obtaining the first light spot map of the first image and before adjusting the first image, the method further comprises: The first light spot map is adjusted according to the brightness values of the pixels in the first image to obtain a second light spot map, and the connected domains of the second light spot map correspond to the light spot region in the first image. In a case where a difference between the light spot threshold value and the brightness value corresponding to the fourth pixel point is less than a fifth threshold value, the fourth pixel point is located in the connected domain of the second light spot map, and the fourth pixel point is a pixel point adjacent to a boundary of the connected domain of the first light spot map.

16. The method according to any one of claims 1 to 15, characterized in that, The first image is obtained, including: In response to a photographing operation, the first image is acquired.

17. The method according to any one of claims 1 to 15, characterized in that, The method further includes: A first interface is displayed, and the first interface displays the first image and a first control; The first image is obtained, including: In response to a triggering operation on the first control, the first image is obtained.

18. An electronic device, comprising: The electronic device includes one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is configured to store computer program code including computer instructions, and the one or more processors are configured to invoke the computer instructions to cause the electronic device to perform the method in any one of claims 1 to 17.

19. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to perform the method in any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions, and when the computer instructions run on an electronic device, the electronic device performs the method in any one of claims 1 to 17.

21. A computer program product, characterised in that, The computer program product includes computer program code, and when the computer program code runs on an electronic device, the electronic device performs the method in any one of claims 1 to 17.

Images