Image processing method and apparatus

Abstract

This application provides an image processing method and apparatus, relating to the field of terminal technology. The method includes: acquiring a first image; adjusting the pixel values ​​of the first image to obtain a second image; wherein a first ratio is greater than a second ratio; the first ratio is the ratio of the pixel value of a first pixel in a spot region of the first image to the pixel value of a second pixel in the same spot region, and the second ratio is the ratio of the pixel value of a first pixel in the second image to the pixel value of a second pixel in the second image; the first pixel includes pixels at the edge of the spot region in the image, and the second pixel includes the pixel corresponding to the center point of the spot region and N surrounding pixels, where N is an integer greater than 0; filtering the second image to obtain a blurred image. In this way, the pixel values ​​of the pixels at the edge of the spot region in the second image are more clearly contrasted with the pixel values ​​of the pixels at the center of the spot region, improving the transparency of the spot in the blurred image.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method and apparatus. Background Technology

[0002] Mobile phones and other electronic devices not only support shooting functions, but also support blurring the image to create a bokeh effect. In a bokeh image, the foreground is sharp while the background is blurred, creating a good sense of space.

[0003] When there are bright objects in the background of an image, the electronic device can create light spots in the area corresponding to the bright objects after blurring the image. Bright objects can include: night scene lights, candlelight, or gaps between leaves under sunlight, etc.

[0004] However, the display effect of light spots in blurred images is poor. Summary of the Invention

[0005] This application provides an image processing method and apparatus, applicable to the field of terminal technology. The method adjusts the pixel values ​​of an image to make the pixel values ​​of pixels at the edges of the light spot region more distinct from those at the center of the light spot region, thereby improving the contrast between pixels at the edges and the center of the light spot region, enhancing the transparency of the light spot in the blurred image, and improving the display effect of the light spot.

[0006] In a first aspect, embodiments of this application propose an image processing method. The method includes: acquiring a first image; adjusting the pixel values ​​of the first image to obtain a second image; wherein a first ratio is greater than a second ratio; the first ratio is the ratio of the pixel value of a first pixel in a spot region of the first image to the pixel value of a second pixel in the spot region of the first image, and the second ratio is the ratio of the pixel value of a first pixel in the second image to the pixel value of a second pixel in the second image; the first pixel includes pixels at the edge of the spot region in the image, and the second pixel includes pixels corresponding to the center point of the spot region and N surrounding pixels, where N is an integer greater than 0; and filtering the second image to obtain a blurred image.

[0007] The first image can be an image captured by a camera application, an image saved by a gallery application, or an image from other scenarios, such as an image captured by a non-camera application. For example, the first image can correspond to the sharp image described below. The second image can correspond to the weighted sharp image described below. The first pixel can correspond to the pixel in region A described below; the second pixel can correspond to the pixel in region D described below.

[0008] Thus, compared to the first image, the second image has higher contrast between the edges and center of the light spot area, resulting in better transparency of the light spot in the blurred image and improving the display effect of the light spot.

[0009] In one possible implementation, adjusting the pixel values ​​of the first image to obtain the second image includes: adjusting the brightness values ​​of the spot regions in the first image to obtain the third image; the brightness value of the first pixel in the third image is less than the brightness value of the first pixel in the first image, and / or the brightness value of the second pixel in the spot regions in the third image is greater than the brightness value of the second pixel in the spot regions in the first image; using the brightness values ​​and corresponding relationships of each pixel in the third image to obtain the weight of each pixel in the third image, where the corresponding relationship is the relationship between the brightness value and the weight; and multiplying the pixel values ​​of each pixel in the first image by their corresponding weights to obtain the second image.

[0010] The third image can be associated with the optimized brightness image described below. The weights of each pixel can be associated with the blur weights described below. The blur weights are used to indicate the degree of blurring of pixels in the image; the higher the blur weight, the higher the degree of blurring; the lower the blur weight, the lower the degree of blurring.

[0011] In this way, adjusting the brightness value at the edge of the spot area in the first image reduces the weight of that pixel, lowers the corresponding pixel value in the second image, and reduces the brightness of the spot in the blurred image, thus improving clarity. Furthermore, reducing the brightness of the spot in the blurred image reduces instances of excessively bright spots, making the overlapping areas of adjacent spots more distinct. Adjusting the brightness value at the center of the spot area in the first image increases the weight of that pixel, raises the corresponding pixel value in the second image, and increases the brightness of the spot in the blurred image, reducing blurring caused by low spot brightness and improving the display effect of the spot.

[0012] In one possible implementation, the correspondence includes the relationship between brightness value, depth value and weight; the weight of each pixel in the third image is obtained by using the brightness value and correspondence of each pixel in the second image, including: obtaining the weight of each pixel in the third image by using the brightness value of each pixel in the third image, the depth value of each pixel in the first image and the correspondence.

[0013] The weights in the correspondence can be compared to the blur weights mentioned below. In this way, the weight of a pixel is also related to the depth value of that pixel, so that the blurriness of areas farther away from the foreground in the blurred image is greater, thus enhancing the spatial sense of the blurred image.

[0014] In one possible implementation, adjusting the brightness value of the spot region in the first image to obtain the third image includes: if the area of ​​the first spot region in the first image is greater than a first threshold, adjusting the brightness value of the first pixel in the first spot region to a first value to reduce the area of ​​the first spot region; the first value is less than the spot threshold, and the spot threshold is used to determine the spot region in the first image.

[0015] The first threshold can correspond to threshold B in the following text, and the first value can correspond to the brightness value D in the following text. In this way, the brightness value of the pixels at the edge of the spot area is adjusted to a fixed value, which is simple and requires little computation.

[0016] In one possible implementation, adjusting the brightness value of the first pixel in the first light spot region to a first value includes: expanding the first light spot region to obtain a first region, the first region being larger than the first light spot region; counting the brightness values ​​of each pixel in the first region; and adjusting the brightness value of the first pixel in the first light spot region to a first value, the first value being the minimum value among the brightness values ​​of each pixel in the first region.

[0017] The first region can correspond to region C in the following text. In this way, selecting a brightness value near the spot area can reduce the mismatch between the brightness value of the first region and the brightness of the spot area, reduce the situation of being too dark or too bright caused by the mismatch, and improve the display effect of the spot in the subsequent blurred image.

[0018] In one possible implementation, the first pixel is a pixel in the first spot region excluding the first region, and the distance between each pixel in the first region and the center point of the first spot region is less than or equal to a second threshold.

[0019] The second threshold can correspond to threshold D as described below.

[0020] In this way, the first region can be identified by the second threshold and the center point of the spot region, which requires less computation and is easy to implement.

[0021] In one possible implementation, if the brightness value of the center point of the second spot region in the first image is less than a third threshold, the brightness value of the second pixel in the second spot region is adjusted to a second value, which is greater than or equal to the third threshold.

[0022] The third threshold can correspond to threshold C as described below. The second value can correspond to the brightness value E as described below.

[0023] In this way, when the brightness at the center of the spot area is low, the brightness at the center of the spot area is increased, reducing blurring caused by low spot brightness in the blurred image and improving the display effect of the spot.

[0024] In one possible implementation, the second value is positively correlated with the first distance, which is negatively correlated with the distance between the second pixel and the center point in the second spot region.

[0025] The first distance can correspond to distance A in the following text. In this way, the brightness of the light spot area is more in line with the brightness variation law of bright objects (point light sources such as lamps and candlelight), thus improving the display effect of the light spot.

[0026] In one possible implementation, adjusting the pixel values ​​of the first image to obtain the second image includes: adjusting the saturation of pixels in the spot region of the first image to obtain a fourth image; wherein the saturation of pixels in the spot region of the fourth image is greater than the saturation of pixels in the spot region of the first image; and adjusting the pixel values ​​of the fourth image to obtain the second image.

[0027] The fourth image corresponds to the optimized, clearer image described below. This increases the saturation of individual pixels, enhancing the vibrancy of the colors in the light spot area and reducing the likelihood of subsequent light spots appearing dull.

[0028] In one possible implementation, the coordinates of a pixel in the YUV color space coordinate system include: an abscissa and a ordinate, where the abscissa corresponds to the U value of the pixel and the ordinate corresponds to the V value of the pixel; adjusting the saturation of pixels in the spot region of the first image includes: adjusting the coordinates of pixels in the spot region of the first image in the YUV color space coordinate system to adjust the saturation of the pixels; wherein, the ratio of the ordinate to the abscissa of pixels in the spot region of the fourth image in the YUV color space coordinate system is the same as the ratio of the ordinate to the abscissa of pixels in the spot region of the first image in the YUV color space coordinate system; the saturation of a pixel is positively correlated with a second distance in the YUV color space coordinate system, where the second distance is the distance between the coordinates of the pixel in the YUV color space coordinate system and the origin.

[0029] In this way, the slope of the pixel's coordinates in the YUV color coordinate system is the same before and after saturation adjustment; it can also be understood as the coordinates being adjusted proportionally, reducing color deviation caused by saturation adjustment.

[0030] In one possible implementation, the third ratio is positively correlated with the third value; the third ratio is the ratio of the ordinate of the pixel in the spot region of the fourth image to the ordinate of the pixel in the spot region of the first image; the third value is the minimum value between the absolute values ​​of the x-coordinate and y-coordinate of the pixel in the YUV color coordinate system.

[0031] By selecting a smaller value, color cast during pixel color adjustment can be reduced, thus improving color optimization results.

[0032] In one possible implementation, the third ratio satisfies: C = MIN(|U N | γ ,|V N | γ ), where C is the third value, γ is the gamma coefficient, and U N V represents the x-coordinate of a pixel in the light spot region of the first image in the YUV color coordinate system. N The value is the ordinate of the pixel in the spot region of the first image in the YUV color coordinate system.

[0033] In this way, gamma mapping is performed on the coordinates of the pixel.

[0034] In one possible implementation, if the saturation of the third pixel in the spot region of the first image is less than or equal to a fourth threshold, the saturation of the third pixel is increased.

[0035] In this way, pixels with high saturation can avoid color optimization, reducing the computational load and speed of color optimization. Pixels with low saturation can be color optimized to reduce the dullness and low saturation caused by overexposure, and to reduce the appearance of washed-out and low-saturation spots in subsequent blurred images.

[0036] In one possible implementation, before adjusting the first image, the method further includes: obtaining a first spot map of the first image based on the brightness value of each pixel in the first image and the spot threshold, wherein the connected components in the first spot map are used to indicate the spot regions in the first image, and the pixels in the connected components of the first spot map correspond to the pixels in the first image whose brightness value is greater than or equal to the spot threshold.

[0037] The first spot image can be associated with the binary spot image A described below. The spot threshold can be associated with the spot threshold described below.

[0038] In this way, determining the spot area through the spot threshold is a simple method with low computational cost.

[0039] In one possible implementation, after obtaining the first spot map of the first image and before adjusting the first image, the method further includes: adjusting the connected regions in the first spot map according to the brightness values ​​of each pixel in the first image to obtain a second spot map, wherein the connected regions of the second spot map correspond to the spot regions in the first image; wherein, if the difference between the spot threshold and the brightness value corresponding to the fourth pixel is less than a fifth threshold, the fourth pixel is located in the connected region of the second spot map, and the fourth pixel is a pixel adjacent to the boundary of the connected region of the first spot map.

[0040] The fifth threshold can correspond to the fixed value A in the following text, and the second spot pattern can correspond to the spot binary pattern B in the following text.

[0041] This reduces the omission of pixels corresponding to the spot area, reduces inaccurate spot recognition, and reduces the occurrence of small spot areas.

[0042] In one possible implementation, acquiring the first image includes: in response to a photo-taking operation, capturing the first image.

[0043] This allows for image processing of images captured in camera or non-camera applications, improving the display effect of light spots in blurred images within those scenarios.

[0044] In one possible implementation, the method further includes: displaying a first interface, the first interface displaying a first image and a first control; and acquiring the first image, including: acquiring the first image in response to a triggering operation on the first control.

[0045] The first control is used to instruct the first image to be blurred. The first control may correspond to the blur control mentioned below, and is not specifically limited here. In this way, image processing can be performed on the image displayed on the electronic device (e.g., an image displayed in a gallery application) to improve the display effect of light spots in the blurred image in that scene.

[0046] Secondly, embodiments of this application provide an image processing apparatus, which may be an electronic device, a chip, or a chip system within an electronic device. The image processing apparatus may include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit may be a display screen. The display unit is used to perform display steps to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is an electronic device, the processing unit may be a processor. The image processing apparatus may further include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or a chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0047] For example, a display unit is used to display a blurred image. A processing unit is used to handle the data processing steps in the image processing apparatus.

[0048] In one possible implementation, the image processing apparatus may further include a storage unit, which may include one or more memories, which may be devices in one or more devices or circuits used to store programs or data.

[0049] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory for storing code instructions, and the processor for running the code instructions to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0050] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0051] Fifthly, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0052] Sixthly, this application provides a chip or chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0053] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0054] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0055] Figure 1A A scenario diagram provided for an embodiment of this application;

[0056] Figure 1B This application provides an embodiment of an electronic device's interface in a photography scenario.

[0057] Figure 1C This application provides an embodiment of an electronic device's interface in a photography scenario.

[0058] Figure 2 This is a flowchart illustrating one possible image processing method in a design.

[0059] Figure 3A This is a schematic diagram of the interface of an electronic device in a gallery editing scenario, provided by an embodiment of this application.

[0060] Figure 3B This is a schematic diagram of the interface of an electronic device in a gallery editing scenario, provided by an embodiment of this application.

[0061] Figure 3C This is a schematic diagram of the interface of an electronic device in a gallery editing scenario, provided by an embodiment of this application.

[0062] Figure 3D This is a schematic diagram of the interface of an electronic device in a gallery editing scenario, provided by an embodiment of this application.

[0063] Figure 3E This is a schematic diagram of the interface of an electronic device in a gallery editing scenario, provided by an embodiment of this application.

[0064] Figure 4A This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0065] Figure 4B This is a schematic diagram showing the position of a camera in an electronic device according to an embodiment of this application;

[0066] Figure 5 A schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0067] Figure 6 A flowchart illustrating an image processing method in a photography scenario provided in this application embodiment;

[0068] Figure 7 A schematic diagram of a YUV color coordinate system provided in an embodiment of this application;

[0069] Figure 8 A comparative schematic diagram of a blurred image provided for an embodiment of this application;

[0070] Figure 9 A flowchart illustrating an image processing method in an editing scene, provided as an embodiment of this application;

[0071] Figure 10 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0072] Figure 11 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0073] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0074] 1. Previewing images and taking photos

[0075] Preview images can be data captured in real-time by the camera of an electronic device and displayed in the preview screen. For example, when an electronic device receives a user's command to open the camera application, it can capture a preview image from the camera and display it in real-time in the camera application's preview screen.

[0076] The captured image can be data obtained based on the camera control in the electronic device, such as the target image described in the embodiments of this application. For example, when the electronic device receives a user's trigger operation on the camera control, the electronic device can acquire the captured image based on the camera at the moment of taking the picture.

[0077] 2. High Dynamic Range (HDR)

[0078] HDR is a processing technique that enhances the brightness and contrast of images. Compared to ordinary images, HDR can provide more dynamic range and image detail. It uses images with the best detail corresponding to each exposure time to synthesize the final HDR image, which can better reflect the visual effects of the real environment.

[0079] One possible implementation for an electronic device to determine whether the current scene is an HDR scene is as follows: The electronic device downsamples the preview image by 4 times to obtain a preview thumbnail, and determines whether the proportion of the number of bright pixels in the preview thumbnail to the total number of pixels in the preview thumbnail is greater than a preset pixel threshold. The preview thumbnail can be obtained by storing every row of pixels in the frame corresponding to the preview image, keeping one row of pixels for every two rows. The bright pixels can be determined based on a grayscale threshold, which can be used to determine whether the current scene is a high dynamic range scene. The electronic device can determine whether the current scene is a high dynamic range scene based on one frame of data or multiple frames of data; this embodiment does not limit this.

[0080] 3. Exposure time (or exposure duration)

[0081] Exposure time is the time the shutter needs to be open to project light onto the photosensitive surface of a photographic material, or it can be understood as the time interval between the shutter opening and closing.

[0082] Exposure time refers to the exposure time of the film. The longer the exposure time, the brighter the resulting photograph, and vice versa. In low-light conditions, it is generally necessary to extend the exposure time to obtain a brighter image.

[0083] 4. Saturation

[0084] Saturation is one of the fundamental characteristics of color, used to describe the purity or intensity of a color as perceived by the human eye. The purity of a color can also be referred to as its vividness.

[0085] 5. Other terms

[0086] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0087] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0088] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0089] 6. Electronic equipment

[0090] The electronic devices in this application embodiment may include handheld devices with image processing functions, vehicle-mounted devices, etc. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices (e.g., smartwatches, smart glasses, smart bracelets, or smart jewelry), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices, and the Internet of Things (IoT). Terminal devices in IoT systems, 5G networks, or future public land mobile networks (PLMNs) are not limited to this category in this application.

[0091] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0092] In this embodiment, the electronic device or various network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0093] Electronic devices can blur the background of images captured by the device. Specifically, they can blur bright objects in the background of an image into light spots. Bright objects can include: nighttime lights, candlelight, or gaps between leaves in sunlight, etc.

[0094] However, the display effect of light spots in blurred images is poor.

[0095] The following example uses a mobile phone as an example for illustration. This example does not constitute a limitation on the embodiments of this application.

[0096] For example, Figure 1A This is a schematic diagram of a scenario provided for an embodiment of this application. Figure 1B and Figure 1C for Figure 1A The diagram shows the interface of the electronic device corresponding to the scenario depicted. Figure 1A The scene shown may include a person 101 in the foreground and a light 102 in the background.

[0097] In response to the camera being turned on, the electronic device can display something like... Figure 1B The interface shown can be a preview interface for the portrait photography function. This interface may include one or more of the following: a camera control 103, a preview screen, controls for switching cameras, or controls for indicating the use of the photography function. The camera application may also include other modes besides portrait photography mode, such as aperture shooting mode, night scene mode, video recording mode, short video mode, and HDR mode.

[0098] The preview image 104 displayed in the preview screen can be obtained by the electronic device recognizing the foreground and background of the captured image and blurring the recognized background. In a possible implementation, the blurred preview image may not be displayed in the preview screen, and this embodiment of the application does not limit this.

[0099] In response to the user's trigger operation on the camera control 103, the electronic device... Figure 1A Take photos of the scene shown and obtain the following: Figure 1C The captured image 105 in the interface shown. The captured image 105 can be obtained by the electronic device recognizing the foreground and background of the captured scene and blurring the recognized background.

[0100] The captured image 105 may include: a clear portrait area 106, and a bokeh area 107, etc. See also... Figure 1A The scenes and Figure 1C In the photographed image shown, the color of the light spot in the light spot area 107 is relatively dim, and the shape and transparency of the light spot are poor.

[0101] Understandably, the areas corresponding to lights in an image are high-brightness areas. When electronic devices perform filtering to blur the background area of ​​an image, larger light spots have higher brightness, lower detail clarity, and lower transparency. For example, the brightness of the overlapping area of ​​adjacent light spots in a blurred image is similar to the brightness of the non-overlapping area, making it impossible for users to distinguish the overlapping area, resulting in poor transparency.

[0102] The following is combined Figure 2 The image blurring process in a possible design is explained. For example, Figure 2 This is a flowchart illustrating one possible image processing method in a design. For example... Figure 2 As shown, the process includes:

[0103] S201. In response to the photo-taking operation, a clear image is obtained.

[0104] The photo-taking action can be any action triggered by the user in the portrait photo-taking function, such as clicking the photo-taking control within the photo-taking function. No specific limitations are specified here.

[0105] S202. Obtain the brightness image corresponding to the clear image.

[0106] A luminance image is used to represent the luminance information of an image. A luminance image includes: the luminance value of each pixel, which indicates the brightness or darkness of the pixel.

[0107] S203. Based on the brightness value of each pixel in the brightness image, look up the weight in the weight table to obtain the weight corresponding to each pixel.

[0108] S204. Multiply the weight corresponding to each pixel by the background area in the clear image to obtain a weighted clear image.

[0109] S205. Perform filtering calculations on the weighted clear image to obtain a blurred image.

[0110] from Figure 2 As can be seen from the process shown, the design may not have adjusted and optimized the point light source area in the image. As a result, the light spot corresponding to the area in the blurred image may appear dull in color, and the shape and transparency of the light spot may be poor, resulting in a poor effect of blurring the image.

[0111] In a possible design, the electronic device could stretch the brightness of a sharp image and then perform a weighted calculation on the stretched image using a spot template to obtain a blurred image. However, the center of the spot in the blurred image tends to be too bright, resulting in an unnatural effect. Furthermore, stretching the brightness of a sharp image may cause overexposure of bright objects, leading to poor spot display in the blurred image.

[0112] In view of this, embodiments of this application provide an image processing method and related apparatus. The electronic device can identify light spot regions in an image and perform color optimization, transparency optimization, and other processing on the light spot regions; it then performs blurring processing on the optimized image to improve the display effect of the light spots in the blurred image.

[0113] For example, Figures 3A to 3E This is a schematic diagram of the interface of an electronic device in another scenario provided by an embodiment of this application. Taking an image saved by a gallery application as an example, Figure 3A The interface shown is a gallery interface. This gallery interface may include at least one thumbnail. This at least one thumbnail may be a thumbnail of an image or a thumbnail of a video; there is no specific limitation here.

[0114] When the electronic device receives a trigger operation such as a click or touch on the thumbnail 301 of the image, the electronic device enters a state such as Figure 3B The image interface shown includes: image 302 corresponding to thumbnail 301 and settings items. Settings items include, but are not limited to: share, favorite, edit, delete, more 303, or other types of editing options.

[0115] When the electronic device receives a trigger action (e.g., a click action) for more 303 errors, the electronic device displays more editable items and enters... Figure 3C The interface shown includes editable options such as: Blur 304, Details, etc.

[0116] In some embodiments, "virtualization 304" can also be set. Figure 3B The interface shown is designed to make image editing easier, more convenient, and enhance the user experience.

[0117] When the electronic device receives a click, touch, or other trigger operation targeting the blurring 304, the electronic device enters... Figure 3D The interface shown includes an adjustment bar 305, a slider 306, and an image 302. When the electronic device receives a sliding operation on the slider 306, it determines the background blur ratio in the image 302 based on the position of the slider 306 on the adjustment bar 305. The electronic device blurs the background in the image 302 using the image processing method provided in this embodiment and then enters... Figure 3E The interface shown. This interface displays a blurred image 307.

[0118] In some embodiments, the electronic device may also not display anything. Figure 3D As shown in the interface, when the electronic device receives a click, touch or other trigger operation for the blurred image 304, the electronic device blurs the background in the image 302 using the image processing method provided in this application embodiment, and obtains and displays the blurred image.

[0119] It is understood that the camera's shooting scene and the gallery's editing scene in the above embodiments are merely examples. The image processing method provided in this application embodiment can also be applied to other scenarios, such as background blurring in video recording mode, and background blurring in scenarios involving taking photos, video calls, and live streaming in third-party camera applications. Third-party camera applications may include: Applications are not specifically limited here. The scenarios in which the image processing method provided in this application is applied are not specifically limited.

[0120] To better understand the embodiments of this application, the structure of the electronic device according to the embodiments of this application is described below. For example, Figure 4A This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0121] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0122] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0123] The processor 110 may include one or more processing units. These processing units may be independent devices or integrated within one or more processors. The processor 110 may also include a memory for storing instructions and data. For example, the processor 110 may store instructions and data related to the image processing method provided in the embodiments of this application.

[0124] The camera 193 consists of a PCB board, lens, fixture, color filter, DSP, and sensor. The imaging principle of the camera 193 is that the scene passes through the lens, projecting the generated optical image onto the sensor. The optical image is then converted into an electrical signal. The electrical signal is converted from analog to digital, and the digital signal is processed by the DSP before being sent to the processor of the terminal device for further processing, ultimately converting it into an image displayed on the electronic device's screen.

[0125] Camera lenses (193) can be categorized by lens type: wide-angle lenses, standard lenses, telephoto lenses, periscope zoom lenses, depth-of-field lenses, etc. Two important parameters of a camera lens (193) are aperture and focal length. The aperture, mounted on the lens, controls the amount of light reaching the sensor. Besides controlling the amount of light, the aperture also controls the depth of field; a larger aperture results in a shallower depth of field. A large aperture effect is a manifestation of a shallow depth of field. Focal length refers to the distance from the center point of the lens to the sensor plane where the image is sharp. The focal length of the lens determines the size of the image of the scene captured on the sensor; that is, in the case of a telephoto lens, the farther away the object, the larger the image. A camera (193) is an imaging sensor for taking pictures; generally speaking, the more lenses used, the sharper the image.

[0126] It is understood that electronic devices may include one or more cameras. This application does not specifically limit the number of cameras.

[0127] In some embodiments, the electronic device includes at least two cameras. This allows for shooting using two cameras, improving image capture quality. For example, such as... Figure 4B As shown, the electronic device includes: camera 401 and camera 402. Camera 401 and camera 402 are used to capture images from the same side.

[0128] It is understandable that cameras 401 and 402 do not completely overlap. Therefore, the exposure and color information of the images captured by cameras 401 and 402 may be different. Consequently, the image after fusing the images captured by cameras 401 and 402 has richer details and better image quality.

[0129] It is understandable that when the distance between cameras 401 and 402 is small, the field of view of the two cameras differs very little, and the impact of misalignment during image fusion is minimal. When the distance between cameras 401 and 402 is large, depth information can be calculated more effectively, facilitating foreground and background segmentation and subsequent background blurring. This application embodiment does not specifically limit the distance or position between cameras 401 and 402.

[0130] Understandably, apart from cameras 401 and 402, Figure 4B The electronic device shown may also include: the camera shown in the dashed box. No specific limitation is made here.

[0131] For example, in this embodiment of the application, after the electronic device detects a trigger operation through the touch sensor 180K, the display screen of the electronic device can realize... Figures 1A to 1C Switching the display of the corresponding interface, and / or, Figures 3A to 3EThe corresponding interface is switched and displayed. When the electronic device can detect a trigger operation for "blurring" through the touch sensor 180K, the processor 110 can perform blurring processing on the clear image using the image processing method provided in this application embodiment to obtain a blurred image.

[0132] The software systems of electronic devices can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc., which will not be elaborated here.

[0133] For example, Figure 5 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0134] The application layer may include a series of application packages. In this embodiment, the application package may include: a camera, a photo library, etc. The camera can capture images and display them. The photo library, also known as a photo album, can store and access captured images.

[0135] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. In this embodiment, the application framework layer may include a camera access interface and a gallery access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide APIs and programming frameworks for camera applications. The gallery access interface is used to provide APIs and programming frameworks for gallery applications.

[0136] The hardware abstraction layer is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. In this embodiment, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0137] The camera hardware abstraction layer can provide virtual hardware for camera device 1, camera device 2, or more camera devices. The camera algorithm library may include runtime code and data implementing the image processing methods provided in the embodiments of this application. For example, the camera algorithm library may include a bokeh algorithm module, etc. The bokeh algorithm module is used to implement bokeh processing of images.

[0138] The driver layer is the layer between hardware and software. It includes drivers for various hardware components, such as camera drivers, digital signal processor drivers, and image processor drivers.

[0139] The camera device driver is used to drive the camera sensor to acquire images and to drive the image signal processor to preprocess the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0140] The image processing method in this application embodiment will be described in detail below with reference to the above system structure:

[0141] Taking the photo-taking process as an example, in response to a user's action of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface in the application framework layer to launch the camera application. This then sends a command to start the camera by calling the camera device (Camera Device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends this command to the camera device driver in the kernel layer. This camera device driver can then start the corresponding camera sensor and acquire image light signals through the sensor. One camera device in the camera hardware abstraction layer corresponds to one camera sensor in the hardware layer.

[0142] Then, the camera sensor can transmit the acquired image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the raw image to the camera hardware abstraction layer through the camera device driver.

[0143] The camera hardware abstraction layer can send the raw image to the camera algorithm library. The camera algorithm library stores program code that implements the image processing method provided in the embodiments of this application. Based on a digital signal processor and an image processor, the camera algorithm library executes the above code to implement the process of generating a blurred image in the image processing method described in the embodiments of this application.

[0144] The camera algorithm library can detect and send the raw images captured by the camera to the camera hardware abstraction layer. The camera hardware abstraction layer can then display these images.

[0145] Taking image editing in a gallery application as an example, in response to the user's operation on the "blur" control, the gallery application calls the camera algorithm library through the gallery access interface, and implements the process of generating a blurred image in the image processing method described in the embodiments of this application through the camera algorithm library.

[0146] It is understood that the software architecture provided in this application is only an example and does not constitute a limitation on the embodiments of this application.

[0147] The following is combined Figures 6 to 10 This section explains the image processing methods used in camera shooting scenarios and stock photo editing scenarios.

[0148] For example, Figure 6 This is a flowchart illustrating an image processing method provided in an embodiment of this application. Taking a camera application's photo-taking scenario as an example, in... Figure 6 In a corresponding embodiment, the electronic device may include: a camera, a gallery, a camera access interface, a camera algorithm library, a camera hardware abstraction layer, and a camera device driver. The function of any of these modules can be found in [reference needed]. Figure 5 The corresponding implementation examples will not be described in detail here. Figure 6 As shown, the image processing method may include the following steps:

[0149] S601, In response to the photo-taking operation, the camera device driver acquires the raw image.

[0150] The original image may include one or more of the following: long frames, short frames, or normal frames, wherein the exposure time of any frame in a long frame is greater than the exposure time of any frame in a normal frame, and the exposure time of any frame in a normal frame is greater than the exposure time of any frame in a short frame.

[0151] It is understood that the original image may include at least two image frames captured at the same time with different exposure times, so that the electronic device can generate a photographic image with HDR effect based on the image fusion of at least two image frames with different exposure times in an HDR photography scene.

[0152] The response to the photo-taking operation includes: responding to a user's trigger operation on the photo-taking control in the portrait photo-taking function, or responding to a user's trigger operation on the photo-taking control in the HDR photo-taking function, or responding to a user's trigger operation on the photo-taking control in the large aperture photo-taking function, or responding to a user's trigger operation on the recording control in the video recording function, etc. The applicable scenarios for the image processing method are not limited in the embodiments of this application.

[0153] S602, the camera algorithm library, obtains raw images from the camera device driver.

[0154] Specifically, the camera hardware abstraction layer can obtain raw images from the camera device driver, and the camera algorithm library can obtain raw images from the camera hardware abstraction layer.

[0155] The S603 camera algorithm library processes raw images into sharp images.

[0156] The process by which a camera algorithm library processes a raw image into a sharp image can be as follows: The camera algorithm library can acquire at least two images with different exposure times at the same moment from the raw image, and perform image preprocessing on any frame of these at least two images. Then, the camera algorithm library can perform image fusion on at least two frames acquired at the same moment from the at least two preprocessed images to obtain a sharp image. The sharp image can also be referred to as the first image.

[0157] It is understandable that since the sharp image is obtained by fusing at least two images with different exposure times, the sharp image has an HDR effect.

[0158] Image preprocessing may include one or more of the following: bad pixel correction processing, RAW domain noise reduction processing, black level correction processing, optical shadow correction processing / automatic white balance processing, color interpolation processing, color correction processing, or gamma correction processing, etc., which are not limited in this embodiment.

[0159] The S604 camera algorithm library performs brightness calculations on clear images to obtain a brightness image.

[0160] A luminance image is used to represent the luminance information of an image. A luminance image includes: a luminance value for each pixel, which indicates the brightness or darkness of the pixel. In some embodiments, a luminance image can be understood as a grayscale image, and the luminance value of a pixel can be understood as the grayscale value of that pixel.

[0161] In one possible implementation, when the sharp image is represented in RGB space, it contains information from three color channels: green (G), red (R), and blue (B). The camera algorithm library can obtain the brightness value of each pixel in the sharp image using information from one or more of these three color channels, thus obtaining a brightness image.

[0162] In some embodiments, the brightness of a sharp image is characterized by the information of the green channel corresponding to each pixel in the sharp image, thus obtaining a brightness image. It is understood that, because the human eye is more sensitive to the green band, a sharp image typically contains more information in the green channel, which can more accurately reflect the brightness of the sharp image.

[0163] In other embodiments, each of the three color channels is assigned a weight, and the brightness value of each pixel in the clear image is obtained by weighted calculation using the information of the three color channels corresponding to each pixel in the clear image, thus obtaining the brightness image.

[0164] In the second possible implementation, when the sharp image is represented in HSV space, it contains information from three attribute channels: hue, saturation, and value. The camera algorithm library can obtain the brightness value of each pixel in the sharp image using one or more of these attribute channels, thus obtaining a brightness image.

[0165] In some embodiments, the brightness of each pixel in the clear image is characterized by the numerical value of the brightness attribute corresponding to each pixel in the clear image, thus obtaining a brightness image.

[0166] In the third possible implementation, when a sharp image is represented in the YUV space, it contains three components: Y value, U value, and V value. The Y value represents luminance (or luma), also known as grayscale; the U and V values ​​represent chrominance (or chroma), describing the image's color and saturation, specifying the color of a pixel. The camera algorithm library can characterize the brightness of each pixel in the sharp image using the Y value corresponding to each pixel, thus obtaining a luminance image.

[0167] It is understandable that RGB, HSV, and YUV color spaces can be converted to each other through a series of mathematical calculations. When a sharp image is represented in RGB space, it can first be converted to HSV space, and the luminance image is obtained based on one or more of the three attribute information. When a sharp image is in HSV space, it can first be converted to RGB space, and the luminance image is obtained based on one or more of the three color channel information. This application does not limit the specific calculation method for the luminance image in its embodiments.

[0168] S605 and the camera algorithm library identify the light spot region in the brightness image and obtain the light spot binary map.

[0169] The camera algorithm can identify the spot regions in the brightness image based on the spot threshold, obtaining a binary spot image A. Specifically, taking the binary spot image containing 0 and 1 as an example, when the brightness of a pixel is greater than or equal to the spot threshold, the pixel is located in the spot region, and the corresponding value is 1; when the brightness of a pixel is less than the spot threshold, the pixel is located in the non-spot region, and the corresponding value is 0.

[0170] In this embodiment, the spot threshold can be 230, 235, or any other value; no specific limitation is made here. In some embodiments, the spot threshold is positively correlated with the average brightness of the brightness image. The larger the average brightness, the larger the spot threshold. The smaller the average brightness, the smaller the spot threshold. The average brightness of the brightness image can be the average brightness value of all pixels in the brightness image. Thus, by adjusting the spot threshold according to the overall brightness of the image, the situation of an excessively high or low spot threshold is reduced, improving the accuracy of spot region recognition.

[0171] Based on the above embodiments, the electronic device can also expand the connected components in the binary image of the light spot A according to the brightness image and the fixed value A to obtain the binary image of the light spot B. A connected component can be understood as a region in an image composed of pixels with the same pixel value that are adjacent to each other.

[0172] In some embodiments, when the difference between the brightness value of a pixel at the boundary of a connected region in the binary image A and the brightness value of its neighboring pixel is less than a fixed value A, the neighboring pixel is located in the spot region, and the value of the neighboring pixel in the binary image B is changed from 0 to 1. When the difference between the brightness value of a pixel at the boundary of a connected region in the binary image A and the brightness value of its neighboring pixel is greater than or equal to the fixed value A, the neighboring pixel is not located in the spot region, and the value of the neighboring pixel in the binary image B is 0. The fixed value A is 15, 20, or any value greater than zero, and is not specifically limited here. This can reduce the omission of pixels corresponding to the spot region, reduce the inaccuracy of spot identification, and reduce the occurrence of small spot regions.

[0173] In other embodiments, when the difference between the brightness value of a pixel adjacent to the boundary of a connected region in the binary image A and the spot threshold is less than a fixed value A, the adjacent pixel is located in the spot region, and the value of the adjacent pixel in the binary image B is changed from 0 to 1. When the difference between the brightness value of a pixel adjacent to the boundary of a connected region in the binary image A and the spot threshold is greater than or equal to the fixed value A, the adjacent pixel is not located in the spot region, and the value of the adjacent pixel in the binary image B is 0. The fixed value A is 15, 20, or any value greater than zero, and is not specifically limited here. This can reduce the omission of spots, reduce the inaccuracy of spot identification, and reduce the occurrence of small spot regions.

[0174] For example, taking the brightness values ​​of each pixel in the brightness image as shown in Table 1, with a spot threshold of 230 and a fixed value A of 20 as an example, the spot binary image A and spot binary image B can be shown in Table 2 and Table 3, respectively.

[0175] Table 1. Brightness values ​​of each pixel in the brightness image.

[0176] 21 22 22 25 30 32 32 39 49 65 71 20 22 23 27 38 69 76 91 66 45 39 21 22 24 31 59 180 239 255 192 83 43 22 23 26 37 91 232 255 255 255 168 75 22 24 28 45 133 255 255 255 255 226 142 23 24 27 36 92 227 255 255 244 216 148 24 25 27 32 54 100 170 194 178 117 79 24 28 27 30 34 42 50 51 44 38 34 24 43 54 60 61 42 34 30 30 28 25 32 71 178 227 205 98 46 32 29 26 25

[0177] Table 2 Binary Image of Light Spot A

[0178] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0179] As shown in Table 1, the brightness values ​​of the pixels in rows 3, columns 7 and 8, rows 4, columns 6 to 9, and rows 5, columns 6 to 9 are all greater than the spot threshold. These pixels are all located in the spot area of ​​the image.

[0180] Adaptively, in the binary image A of the light spot in Table 2, the values ​​corresponding to the pixels in the 3rd row and 8th column, the pixels in the 4th row and 7th to 9th columns, and the pixels in the 5th row and 6th to 9th columns are all 1, and the rest are 0. The connected components include: the pixels in the 3rd row and 8th column, the pixels in the 4th row and 7th to 9th columns, and the pixels in the 5th row and 6th to 9th columns.

[0181] Table 3 Binary Image of Light Spot (B)

[0182]

[0183]

[0184] As can be seen from Table 2, the boundaries of the connected components include: the pixel in the 7th column of the 3rd row, the pixels in the 7th and 9th columns of the 4th row, and the pixels in the 6th and 9th columns of the 5th row. Taking the example that the difference between the brightness value of the pixel adjacent to the boundary of the connected component and the spot threshold is less than 20, compared with Table 2, the values ​​corresponding to the pixel in the 9th column of the 5th row and the pixels in the 6th and 10th columns of the 4th row in Table 3 are all 1.

[0185] S606: The camera algorithm library determines the spot regions in the clear image based on the spot binary map, and performs color optimization on the spot regions in the clear image to obtain the optimized clear image.

[0186] In this embodiment, by increasing the saturation of the corresponding pixel, the color vibrancy of the light spot area is improved, and the dullness of the subsequent light spot color is reduced.

[0187] For example, taking the YUV color coordinate system as an example, the saturation (S) of a pixel satisfies: Among them, U N V represents the x-coordinate of the pixel in the YUV color coordinate system. N The vertical coordinate of the pixel in the YUV color coordinate system.

[0188] For example, Figure 7This is a schematic diagram of a YUV color coordinate system provided in an embodiment of this application. Figure 7 As shown, the horizontal axis is Cb, which corresponds to the U value of the pixel, and the vertical axis is Cr, which corresponds to the V value of the pixel.

[0189] The coordinates of the pixel in the YUV color coordinate system are (U N V N Saturation S can be understood as the distance between the coordinates of a pixel and the origin.

[0190] Taking a color coordinate system ranging from -1 to 1, and the U and V values ​​of a pixel in the YUV space both ranging from 0 to 255 as an example, the horizontal coordinate U in the color coordinate system... N satisfy The vertical coordinate V in the color coordinate system N satisfy

[0191] In some embodiments, saturation can be achieved by increasing the coordinate point (U) while keeping the straight line between the coordinate point and the origin unchanged. N V N This is done by adjusting the distance between the origin and the saturation point. This reduces color deviation caused by saturation adjustments.

[0192] Specifically, ratios A and B are the same. Ratio A is the ratio of the x-coordinate to the y-coordinate of a pixel in the optimized, clear image within the YUV color coordinate system, and ratio B is the same as the ratio of the x-coordinate to the y-coordinate of a pixel in the YUV color coordinate system. The distance between the coordinates of a pixel after color optimization and the origin is greater than the distance between the coordinates of a pixel before color optimization and the origin.

[0193] In this way, before and after color optimization, the coordinates of the pixel in the YUV color coordinate system remain unchanged, which can reduce the color deviation caused by color optimization.

[0194] In some embodiments, by coordinate point (U) N V N All values ​​are multiplied by the saturation enhancement factor C to achieve saturation enhancement. The saturation enhancement factor C can be any value greater than 1. No specific restrictions are imposed here.

[0195] In some embodiments, the saturation enhancement factor C is positively correlated with the minimum value A. The minimum value A is the minimum between the absolute values ​​of the x-coordinate and y-coordinate of a pixel in the YUV color coordinate system.

[0196] By selecting a smaller value, color cast during pixel color adjustment can be reduced, thus improving color optimization results.

[0197] In some embodiments, the saturation enhancement factor C is determined by a pre-set Gamma coefficient γ. Specifically, the saturation enhancement factor C is the abscissa U... N The value obtained by mapping the absolute value to the Gamma coefficient, and the ordinate V. N Find the minimum value between the absolute value and the value obtained after mapping with the Gamma coefficient. The saturation enhancement factor C satisfies: C = MIN(|U N | γ ,|V N | γ ).

[0198] Adaptively, taking the example where the U value and V value of a pixel in YUV space are both between 0 and 255, the U value before adjustment is U old The value of V before adjustment is V old The x-coordinate U of this pixel in the color coordinate system Nold satisfy The vertical coordinate V of this pixel in the color coordinate system Nold satisfy Adjusted U value U New Satisfy: U New =C*U Nold +128. Adjusted V value V New Satisfy: V New =C*V Nold +128.

[0199] In this embodiment, the Gamma coefficient γ can be obtained experimentally or by any other means, without specific limitations. For example, taking experimental methods, different Gamma coefficient values ​​can be used to perform color optimization processing on the spot regions in the same batch of image samples, resulting in optimized images corresponding to each Gamma coefficient value. The Gamma coefficient corresponding to the image with the best optimization effect among the optimized images is used as the pre-set Gamma coefficient γ. Alternatively, the Gamma coefficients corresponding to the top N images in terms of optimization effect among the optimized images are statistically analyzed, and the Gamma coefficient that appears most frequently among these N Gamma coefficients is selected as the pre-set Gamma coefficient γ. This embodiment does not specifically limit the method for confirming the pre-set Gamma coefficient γ.

[0200] In some embodiments, color optimization is performed when the saturation of a pixel is less than a threshold A; color optimization is not performed when the saturation of a pixel is greater than or equal to the threshold A. The threshold A can be 0.3 or any value, and is not specifically limited here.

[0201] In this way, pixels with high saturation can avoid color optimization, reducing the computational load and speed of color optimization. Pixels with low saturation can be color optimized to reduce the dullness and low saturation caused by overexposure, and to reduce the appearance of washed-out and low-saturation spots in subsequent blurred images.

[0202] S607, the camera algorithm library optimizes the brightness of each spot region in the image based on the area of ​​each spot region, and obtains the optimized brightness image.

[0203] It's important to note that the principle behind the generation of light spots is that the brightness values ​​of the bright areas in the sharp image correspond to a higher blur weight, resulting in a higher degree of blurring in those areas. During subsequent filtering, the pixel values ​​of the bright areas in the sharp image significantly influence the brightness of the pixels in the blurred image. Therefore, in a possible design, when the area of ​​the light spot region in the image is large, the overall brightness of the light spot in the blurred image may be high, resulting in poor light spot transparency; conversely, when the brightness of the light spot region in the image is low, the brightness of the light spot in the blurred image may be low, leading to a blurred light spot.

[0204] In this embodiment, brightness optimization is achieved by reducing the brightness values ​​of pixels at the edges of the light spot region and / or increasing the brightness values ​​of pixels at the center of the light spot region. This improves the brightness ratio between the edges and center of the light spot region, resulting in higher contrast between the edges and center of the subsequent light spot region and better transparency of the light spot in the blurred image.

[0205] Understandably, reducing the brightness of pixels at the edges of the light spot area can decrease the corresponding blur weight of that area, reducing the overall brightness of the light spot and improving its clarity. For example, when adjacent light spots overlap, the brightness of the overlapping areas is higher than that of the non-overlapping areas, making the overlap more noticeable and resulting in better clarity of the light spot in the blurred image.

[0206] For example, Figure 8 This is a schematic diagram of a blurred image corresponding to the brightness value of the unadjusted spot area and a blurred image corresponding to the brightness value of the adjusted spot area, provided in an embodiment of this application.

[0207] from Figure 8 It can be seen from this that Figure 8 In the light spot region 801 shown in Figure a, the brightness of the area where adjacent light spots overlap is the same as the brightness of one of the light spots, and the light spot in the light spot region 801 has poor transparency. Figure 8 In the light spot region 802 shown in b, the brightness of the overlapping area of ​​adjacent light spots is greater than that of the adjacent light spot, and the light spot in the light spot region 802 has better transparency.

[0208] In some embodiments, when the area of ​​the spot region in the image is larger than a threshold B, the brightness value of the pixels at the edge of the spot region is reduced to shrink the area of ​​the spot region. The threshold B can be 16, 15, or any value, and is not specifically limited here. This way, for smaller spot regions, the brightness value of the pixels at the edge of the spot region is not adjusted, reducing the blurring caused by low brightness in the spot region. For larger spot regions, the brightness value of the pixels at the edge of the spot region is adjusted to improve the contrast between the edge and center of the spot region, thus enhancing the transparency of the spot.

[0209] In some embodiments, if the area of ​​the spot region in the image is greater than a threshold B, the brightness value of each pixel in region A is reduced. Region A is the area in the spot region excluding region B, and region B is the region corresponding to the reduced spot region. Region B includes the pixels at the edge of the spot region.

[0210] In some embodiments, the center point of region B is the same as the center point of the spot region. It is understood that the center points being the same can be absolutely identical, or they can have a certain deviation, for example, a deviation of less than one pixel. This ensures that the center of the spot region remains unchanged before and after shrinking, reducing changes in the spot's position in the subsequent blurred image.

[0211] In this embodiment of the application, the center of the light spot region can be understood as the average value of the position of each pixel in the light spot region.

[0212] In some embodiments, the shape of region B is the same as the shape of the center point of the spot region. This preserves the shape of the spot region and reduces morphological changes of the spot in the blurred image.

[0213] In this embodiment, region B can be determined in various ways. For example, it can be reduced to a preset area, the area of ​​the spot region can be reduced by a certain ratio, or the distance from the center point of the spot region to its boundary can be reduced by M pixels. This embodiment does not specifically limit the method of reducing the spot region. The reduced spot region is not specifically limited.

[0214] In one possible implementation, the preset area can be 16 pixels * pixel, 15 pixels * pixel, or any other value; no specific limitation is made here. This method of shrinking to the preset area ensures that the area of ​​the shrunken spot is fixed, resulting in less computation and ease of implementation.

[0215] It is understood that, typically, the light spot area corresponding to a point light source is a circular area. In some embodiments, the electronic device can determine the area of ​​the light spot area by the center point and the radius of the light spot area. Adaptively, reducing the area to a preset size can also be understood as reducing it to a preset radius. For example, if the radius of the light spot area before reduction is 7 pixels and the preset radius is 3 pixels, the reduced light spot area can be a circular area with the center of the light spot area as the center and a radius of 3 pixels.

[0216] Alternatively, it can be understood that the distance from each pixel in region A to the center of the spot area is greater than the preset radius; and the distance from each pixel in region B to the center of the spot area is less than or equal to the preset radius.

[0217] In the second possible implementation, the ratio can be 0.4, 0.6, or any other value; no specific limitation is made here. This method of reducing the area of ​​the light spot by a certain ratio ensures that the size of the reduced light spot area is positively correlated with the size of the original light spot area. The size of the light spot area better matches the brightness patterns in the actual scene, making the light spot in the blurred image match the bright objects in the actual scene.

[0218] For example, taking a circular area as an example, reducing the area of ​​the light spot by a certain proportion can also be understood as reducing the radius of the light spot by a certain proportion. For example, if the radius of the light spot before reduction is 7 and the certain proportion is 0.5, the reduced light spot area can be a circular area with a radius of 3.5 centered at the center of the light spot area.

[0219] Alternatively, it can be understood that the ratio C is greater than a certain proportion, and the ratio D is less than or equal to a certain proportion; the ratio C is the ratio of the distance from each pixel in region A to the center of the spot region to the distance from the boundary of the spot region to the center of the spot region; the ratio D is the ratio of the distance from each pixel in region B to the center of the spot region to the distance from the boundary of the spot region to the center of the spot region.

[0220] In the third possible implementation, M can be an integer greater than 1. In this way, by reducing the size of the spot area by M pixels based on the distance from the center point of the spot area to the boundary of the spot area, the size of the reduced spot area can be positively correlated with the size of the original spot area. The size of the spot area is more in line with the brightness rules in the actual scene, so that the spot in the blurred image matches the bright objects in the actual scene.

[0221] The above explains how regions A and B are determined. The following explains the brightness adjustment of each pixel in region A.

[0222] In some embodiments, the brightness value of each pixel in region A is adjusted to a brightness value D. The brightness value D is less than the spot threshold.

[0223] The brightness value D can be a pre-set value, such as 50 or 80, etc., without specific limitations here. The brightness value D can also be the minimum brightness value among all pixels in region C. Region C can be a circular region with the center of the spot area as the center and the radius as the threshold D. The threshold D can be 10, 15, or any value, without specific limitations here.

[0224] In this way, selecting a brightness value near the pixel can reduce the mismatch between the brightness value of region A and the brightness of the vicinity of the spot area, thereby reducing the situation of being too dark or too bright due to the mismatch and improving the display effect of the spot in the subsequent blurred image. This application embodiment does not specifically limit the specific value or the specific method of determining the brightness value D.

[0225] The above embodiments describe the adjustment of pixels at the edge of the light spot area. The following describes the adjustment of pixels at the center and around the light spot area.

[0226] Based on the above embodiment, when the brightness value at the center of the spot area in the image is less than a threshold C, the brightness of the center point of the spot area and the surrounding N pixels is increased to a brightness value E. N is an integer greater than 0. N can be 1, 4, 9, or any value, without specific limitation here. In this way, increasing the brightness value of the pixels in the spot area can increase the corresponding blur weight of the bright area, reduce the blurring and poor visual effect caused by low spot brightness, and improve the clarity of the spot.

[0227] In some embodiments, taking a circular area as an example, the brightness value of the pixels in region D is increased. Region D is a circular area with the center point of the light spot area as the center and a threshold D as the radius.

[0228] In this embodiment, the brightness value E can be a fixed value, such as 255, 240, etc. The brightness value E can also be negatively correlated with distance A, where distance A is the distance between the pixel and the center point of the light spot area. The larger the distance A, the smaller the brightness value E; the smaller the distance A, the larger the brightness value E. In this way, the brightness of the light spot area better conforms to the brightness variation law of bright objects, improving the display effect of the light spot.

[0229] For example, the brightness value E and the distance A satisfy a pre-set quadratic formula. This ensures that the brightness of the light spot area conforms to the brightness variation law of the bright object, thus improving the display effect of the light spot.

[0230] The S608 camera algorithm library performs depth calculations on clear images to obtain depth images.

[0231] A depth image can include depth information of any pixel. The depth information can represent the distance from each point in the scene to the camera plane and can reflect the geometry of visible surfaces in the scene. The depth information of any pixel in the depth image can be determined based on monocular depth estimation, binocular depth estimation, or depth estimation based on deep learning. This application does not limit this.

[0232] Depth images are used to calculate the blur weights in a sharp image. This can also be understood as the blur weights being related to the depth information of each pixel in the depth image. For example, the greater the depth of a pixel, the greater the blur weight, and the more blurred the pixel; conversely, the smaller the depth of a pixel, the smaller the blur weight, and the sharper the pixel.

[0233] In some embodiments, the greater the difference between the depth corresponding to a pixel and the depth corresponding to the focal point, the greater the blur weight, and the more blurred the pixel; the smaller the difference between the depth corresponding to a pixel and the depth corresponding to the focal point, the smaller the blur weight, and the clearer the pixel.

[0234] The focus point can be determined by the electronic device in response to an operation indicating the focus, or it can be determined by the electronic device based on a portrait or object captured in a clear image. For example, in the portrait function of a camera, the electronic device can detect the subject being photographed. For instance, when a portrait is detected among the subjects, the location of the portrait is set as the focus point, completing the focusing of the portrait and making it clearly visible in the image. This application does not specifically limit the focus point in its embodiments.

[0235] In some embodiments, the electronic device can distinguish between foreground and background based on focus and depth images. For example, the depth at the focus point can be a first depth, which can be set to 0. The depth at other locations in the depth image can be the difference between the depth at those other locations and the first depth, thus obtaining the foreground and background. The foreground can include a sharp range before the focus point, and the background can include a sharp range after the focus point.

[0236] S609, the camera algorithm library searches the weight table for the optimized brightness and depth images to obtain the blur weights corresponding to each pixel in the clear image.

[0237] In some embodiments, a weight table is used to indicate the correspondence between luminance values, depth values, and bokeh weights. Both luminance and depth values ​​are positively correlated with the bokeh weights.

[0238] In other embodiments, a weight table is used to indicate the correspondence between luminance values ​​and luminance coefficients, and the correspondence between depth values ​​and depth coefficients.

[0239] In this embodiment, the brightness value is positively correlated with the blur weight. The larger the brightness value, the larger the blur weight; the smaller the brightness value, the smaller the blur weight.

[0240] In some embodiments, the blurring weight is also affected by the depth value corresponding to the pixel. The depth value corresponding to the pixel is positively correlated with the blurring weight. The larger the depth value, the larger the weight; the smaller the depth value, the smaller the weight. In this way, the weight of a pixel is also related to the depth value of that pixel, which makes the blurring degree of areas farther away from the foreground in the blurred image higher, thus improving the spatial sense of the blurred image.

[0241] For example, the blur weight can be the product of weight A and weight B. Weight A is the weight obtained by looking up the weight table based on the luminance value; weight B is the weight obtained by looking up the weight table based on the depth value.

[0242] For example, the blur weight can be the sum of weight A and weight B. Weight A is the weight obtained by looking up the weight table based on the luminance value; weight B is the weight obtained by looking up the weight table based on the depth value.

[0243] The weight table is just an example. The correspondence between brightness value, depth value and blur weight can also be represented in any other form, such as formulas.

[0244] In some embodiments, the electronic device may not execute S608. Adaptively, in S609, the camera algorithm library looks up the weight table in the optimized brightness image to obtain the bokeh weight corresponding to each pixel in the clear image. In some embodiments, the brightness coefficient of each pixel may also be used as the bokeh weight corresponding to each pixel.

[0245] The S610 camera algorithm library filters the optimized sharp image based on the blur weights to obtain a blurred image.

[0246] S610 may include: a camera algorithm library adjusting the pixel values ​​of the optimized sharp image according to the blur weights to obtain a weighted image; and the camera algorithm library filtering the weighted image to obtain a blurred image.

[0247] The pixel value of each pixel in the weighted image is the product of the pixel value of each pixel in the optimized clear image and its corresponding blur weight.

[0248] In this embodiment, pixel values ​​can be understood as the values ​​of various channels in an image. Taking an image represented in RGB space as an example, pixel values ​​include the values ​​corresponding to the green, red, and blue color channels. Taking an image represented in HSV space as an example, pixel values ​​include the values ​​corresponding to the hue, saturation, and brightness channels. Taking an image represented in YUV space as an example, pixel values ​​include the Y, U, and V values.

[0249] Filtering can be Gaussian blur filtering, mean blur filtering, etc., and no specific limitation is made here.

[0250] For example, taking Gaussian blur filtering as an example, for any pixel, the pixel value of that pixel is obtained by taking a weighted average of the pixel values ​​of all pixels within its blur radius. Alternatively, this can be understood as adding the pixel value of that pixel to the pixel values ​​of all pixels within its blur radius, then dividing by the sum of the filtering weights of that pixel and the filtering weights of all pixels within its blur radius. The filtering weights of the pixel and the filtering weights of the pixels within its blur radius satisfy a Gaussian (normal) distribution density function.

[0251] Understandably, Gaussian blur takes into account the distance between each pixel within the blur radius and the central pixel, which may preserve some image details and result in good processing effects.

[0252] In this embodiment, the blurring radius can be 5 pixels, 10 pixels, or any other value; no specific limitation is made here. The filter weight is used to indicate the weight corresponding to each pixel during the filtering process.

[0253] For example, taking mean blur filtering as an example, for any pixel, the average pixel value of that pixel and the pixel values ​​of all pixels within its blur radius are calculated to obtain the pixel value of that pixel in the blurred image. It is understandable that mean blur is relatively simple to calculate and has a fast processing speed.

[0254] A blurred image can include a sharp foreground and a blurred background. When a sharp image is displayed as... Figure 1A As shown, after blurring a sharp image, a blurred image with a sharp foreground and a blurred background can be obtained. The blurred image can be... Figure 1C The image displayed in the interface shown.

[0255] After the camera algorithm library determines the blurred image, it can store the blurred image in the image library through the steps shown in S612-S613, and process the blurred image into a thumbnail and display the thumbnail through S614-S616. The embodiments of this application do not limit the order of the above two processes.

[0256] S612: The image library retrieves the blurred image from the camera algorithm library.

[0257] Specifically, the camera access interface (or the first interface) can obtain a thumbnail from the camera algorithm library, and the camera can then obtain that thumbnail from the camera access interface (or the first interface). The first interface (in...) Figure 5 (Not shown in the image) can be used to establish a data path between the image library and the camera algorithm library. The type of the first interface is not limited in the embodiments of this application.

[0258] S613, Gallery stores blurred images.

[0259] After S613, electronic devices can display blurred images in response to a user opening the gallery application.

[0260] S614, the camera algorithm library processes blurred images into thumbnails.

[0261] Camera algorithm libraries can process blurred images into thumbnails using thumbnail processing methods such as sampling or neural networks, and this application does not limit this.

[0262] S615: The camera retrieves thumbnails from the camera algorithm library.

[0263] Specifically, the camera access interface can obtain thumbnails from the camera algorithm library, and then the camera can obtain the thumbnails from the camera access interface.

[0264] S616: The camera accesses the display screen to show thumbnails.

[0265] For example, the camera can Figure 1B The thumbnail is displayed in the lower left corner of the interface shown.

[0266] Understandable Figure 6 The sequential relationship between the steps described herein is merely an example and should not be construed as limiting the embodiments of this application. Figure 6 In the illustrated embodiment, the optimization of the light spot in the image is achieved by adjusting both the saturation of the pixels in the light spot region and the brightness of the pixels in the light spot region.

[0267] In some embodiments, the electronic device may also optimize the blurring of the light spot in the image simply by adjusting the saturation of the pixels in the light spot region of the image. For example, the electronic device may omit S607. Adaptably, the optimized brightness image in S609 can be replaced with the brightness image in S604.

[0268] In some embodiments, the electronic device may also optimize the blurring of the light spots in the image simply by adjusting the brightness values ​​of the pixels in the light spot region of the image. For example, the electronic device may omit S606. Adaptably, the optimized sharp image in S609 can be replaced with a sharp image.

[0269] Figure 6In the illustrated embodiment, by increasing the saturation of pixels in the bright spot region of the clear image, the dimness of the bright spot after filtering is reduced, thereby improving the color effect of the bright spot region. This is achieved by increasing the brightness of the center of the bright spot region and reducing the brightness of the edges; adjusting the weights corresponding to the bright spot region, thus improving the shape and transparency of the bright spot. Furthermore, Figure 6 In the illustrated embodiment, when blurring a clear image through S604-S610, there is no need to stretch the brightness of the entire clear image, which can reduce overexposure of the clear image and reduce poor display effect of light spots caused by overexposure. Figure 6 In the illustrated embodiment, naturally generated light spots can be used during filtering calculations, eliminating the need to segment bright objects from a clear image for separate filtering calculations, thus reducing subsequent fusion processing and saving computational resources.

[0270] The above Figures 6 to 8 The image processing methods used in camera photography scenarios are explained below. Figure 9 This section explains the image processing methods used in the gallery editing scenario.

[0271] For example, Figure 9 This is a flowchart illustrating an image processing method provided in an embodiment of this application. Taking an editing scenario in a gallery application as an example, in... Figure 9 In a corresponding embodiment, the electronic device may include: a gallery, a gallery access interface, a camera algorithm library, a gallery hardware abstraction layer, and a TP driver. The function of any of these modules can be found in [reference needed]. Figure 5 The corresponding implementation examples will not be described in detail here. Figure 9 As shown, the image processing method may include the following steps: The gallery application displays interface A, which includes: a sharp image and a blur control.

[0272] S901. In response to a trigger operation on the blur control, the gallery application calls the camera algorithm library via the gallery access interface to blur the sharp image.

[0273] S902, the camera algorithm library performs brightness calculations on clear images to obtain brightness images.

[0274] S903 and camera algorithm library identify the light spot region in the brightness image and obtain the light spot binary map.

[0275] S904. Determine the spot region in the clear image based on the spot binary image, and perform color optimization on the spot region in the clear image to obtain the optimized clear image.

[0276] S905. Based on the area of ​​each spot region in the image, optimize the brightness of the pixels in each spot region to obtain an optimized brightness image.

[0277] The S906 camera algorithm library performs depth calculations on clear images to obtain depth images.

[0278] S907. Look up the weight table for the optimized brightness and depth images to obtain the blur weights corresponding to the sharp images.

[0279] The S908 camera algorithm library filters the optimized sharp image based on the blur weight to obtain a blurred image.

[0280] S909: The image library obtains the blurred image from the camera algorithm library.

[0281] S910, Gallery displays blurred images.

[0282] The above S903 to S909 can be referred to Figure 6 The corresponding steps are explained in the instructions, and will not be repeated here.

[0283] This allows for the blurring of images in the gallery, enhancing the display effect of light spots in the blurred image. Specifically, by increasing the saturation of pixels in the light spot area of ​​the clear image, the dimness of the light spots after filtering is reduced, thus improving the color effect of the light spots. By increasing the brightness of the center of the light spot area and reducing the brightness of the edge of the light spot area, and by adjusting the weights corresponding to the light spot area, the shape and transparency of the light spots are improved.

[0284] also, Figure 9 In the illustrated embodiment, when blurring a clear image, there is no need to stretch the brightness of the entire clear image, which can reduce overexposure of the clear image and reduce poor display effect of light spots caused by overexposure. Figure 9 In the illustrated embodiment, naturally generated light spots can be used during filtering calculations, eliminating the need to segment bright objects from a clear image for separate filtering calculations, thus reducing subsequent fusion processing and saving computational resources.

[0285] For example, Figure 10 This is a schematic flowchart illustrating the image processing method provided in an embodiment of this application. Figure 10 As shown, the method includes:

[0286] S1001, Obtain a clear image.

[0287] Taking a photo-taking scenario as an example, in response to the photo-taking operation, a clear image is acquired, and steps S1002 to S1008 are executed. Taking a gallery editing scenario as an example, in response to an operation used to indicate the selected image, a clear image is acquired. In response to a trigger operation on the blur control, steps S1002 to S1008 are executed. This application embodiment does not specifically limit the triggering conditions for acquiring a clear image.

[0288] S1002. Obtain the brightness image corresponding to the clear image.

[0289] S1002 can be referred to the description of the corresponding steps in S604 above, and will not be repeated here.

[0290] S1003. Identify the spot regions in the brightness image and obtain a binary image of the spot.

[0291] S1004. Based on the binary image of the light spot, perform color optimization on the light spot region in the clear image to obtain the optimized clear image.

[0292] In this embodiment, the saturation of each pixel in the spot region of a clear image can be adjusted based on the spot binary image. The specific adjustment method can be referred to the above. Figure 6 The corresponding explanations in the document will not be repeated here.

[0293] S1005. Optimize the brightness of the spot area in the clear image based on the spot binary image to obtain the optimized brightness image.

[0294] In this embodiment, the brightness value of each pixel in the spot region of the brightness image can be adjusted based on the spot binary image. The specific adjustment method can be referred to the above. Figure 6 The corresponding explanations in the document will not be repeated here.

[0295] S1006. Based on the brightness value of each pixel in the optimized brightness image, look up the weight in the weight table to obtain the weight corresponding to each pixel.

[0296] S1007. Multiply the weight corresponding to each pixel by the background area in the optimized clear image to obtain a weighted clear image.

[0297] S1008. Perform filtering calculations on the weighted sharp image to obtain a blurred image.

[0298] S1006 to S1008 can be referred to the above. Figure 6 The corresponding steps are explained in detail here.

[0299] In this way, electronic devices can identify the glare areas in an image and perform color optimization and transparency optimization on these areas. The optimized image is then blurred to enhance the display effect of the glare areas. Furthermore, there is no need to segment the glare areas from the image for processing, reducing subsequent fusion steps; the process is simple and easy to implement.

[0300] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0301] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0302] The image processing method of the present application embodiments has been described above. The apparatus for performing the above method provided in the present application embodiments is described below. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced by each other, and the related apparatus provided in the present application embodiments can perform the steps in the above method.

[0303] like Figure 11 As shown, Figure 11 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device may be an electronic device in the embodiment of this application, or it may be a chip or chip system within an electronic device.

[0304] like Figure 11 As shown, the image processing apparatus can be used in communication equipment, circuits, hardware components, or chips. The image processing apparatus includes a display unit 1101 and a processing unit 1102. The display unit 1101 supports the display steps performed by the display method; the processing unit 1102 supports the information processing steps performed by the image processing apparatus.

[0305] In one possible implementation, the image processing apparatus may further include a communication unit 1103, which supports the image processing apparatus in performing steps such as receiving or sending messages.

[0306] The image processing apparatus described in the embodiments of this application may include all of the following: Figure 11 The units described in the corresponding embodiments.

[0307] Specifically, the processing unit 1102 can be integrated with the display unit 1101, and the processing unit 1102 and the display unit 1101 may communicate with each other.

[0308] In one possible implementation, the image processing apparatus may further include a storage unit 1104. The storage unit 1104 may include one or more memories, which may be devices in one or more devices or circuits used for storing programs or data.

[0309] The storage unit 1104 can exist independently or be connected to the processing unit 1102 via a communication bus. Alternatively, the storage unit 1104 can be integrated with the processing unit 1102.

[0310] Taking the image processing device as an example, which may be a chip or chip system of the electronic device in the embodiments of this application, the storage unit 1104 may store computer-executable instructions for the method of the electronic device, so that the processing unit 1102 executes the method of the electronic device in the above embodiments. The storage unit 1104 may be a register, cache, or random access memory (RAM), etc., and the storage unit 1104 may be integrated with the processing unit 1102. The storage unit 1104 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 1104 may be independent of the processing unit 1102.

[0311] In one possible implementation, the image processing apparatus may further include a communication unit 1103. The communication unit 1103 supports interaction between the image processing apparatus and other devices. For example, when the image processing apparatus is an electronic device, the communication unit 1103 may be a communication interface or interface circuit. When the image processing apparatus is a chip or chip system within an electronic device, the communication unit 1103 may be a communication interface. For example, the communication interface may be an input / output interface, pins, or circuits.

[0312] The apparatus in this embodiment can be used to execute the steps performed in the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0313] The image processing method provided in this application can be applied to electronic devices with image processing capabilities. Electronic devices include terminal devices, and the specific device form of the terminal device can be referred to the above-described related information, which will not be repeated here.

[0314] This application provides an electronic device, which includes one or more processors and a memory; the memory is coupled to one or more processors and is used to store computer program code, which includes computer instructions, and one or more processors call the computer instructions to cause the electronic device to perform the above-described method.

[0315] This application provides a chip or chip system. The chip or chip system includes one or more processors, which invoke computer instructions to cause an electronic device to execute the technical solutions described above. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0316] This application also provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted on the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0317] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0318] This application provides a computer program product, which includes computer program code. When the computer program code is run, it causes the computer to perform the above-described method.

[0319] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0320] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, include: Get the first image; Adjust the pixel values ​​of the first image to obtain the second image; Wherein, the first ratio is greater than the second ratio; the first ratio is the ratio of the pixel value of the first pixel in the spot region of the first image to the pixel value of the second pixel in the spot region of the first image, and the second ratio is the ratio of the pixel value of the first pixel in the second image to the pixel value of the second pixel in the second image; the first pixel includes: the pixel at the edge of the spot region in the image, and the second pixel includes: the pixel corresponding to the center point of the spot region and N pixels surrounding it, where N is an integer greater than 0; the spot region is determined based on the brightness value of each pixel in the first image and the spot threshold; The second image is filtered to obtain a blurred image; The step of adjusting the pixel values ​​of the first image to obtain the second image includes: Adjust the brightness value of the spot area in the first image to obtain a third image; the brightness value of the first pixel in the third image is less than the brightness value of the first pixel in the first image, and / or the brightness value of the second pixel in the spot area in the third image is greater than the brightness value of the second pixel in the spot area in the first image; By using the brightness values ​​and corresponding relationships of each pixel in the third image, the weight of each pixel in the third image is obtained, where the corresponding relationship is the relationship between the brightness value and the weight. The second image is obtained by multiplying the pixel value of each pixel in the first image with its corresponding weight; The step of adjusting the brightness value of the spot area in the first image to obtain the third image includes: If the area of ​​the first spot region in the first image is greater than the first threshold, the first spot region is expanded to obtain a first region, and the first region is larger than the first spot region. Calculate the brightness value of each pixel in the first region; The brightness value of the first pixel in the first spot region is adjusted to a first value to reduce the area of ​​the first spot region. The first value is the minimum brightness value of each pixel in the first region. The first value is less than the spot threshold. The spot threshold is used to determine the spot region in the first image.

2. The method according to claim 1, characterized in that, The correspondence includes the relationship between brightness value, depth value and weight; The step of obtaining the weights of each pixel in the third image using the brightness values ​​and corresponding relationships of each pixel in the second image includes: The weight of each pixel in the third image is obtained by using the brightness value of each pixel in the third image, the depth value of each pixel in the first image, and their corresponding relationship.

3. The method according to claim 1 or 2, characterized in that, The first pixel is a pixel in the first spot region other than the first region, and the distance between each pixel in the first region and the center point of the first spot region is less than or equal to the second threshold.

4. The method according to claim 1 or 2, characterized in that, If the brightness value of the center point of the second spot region in the first image is less than the third threshold, the brightness value of the second pixel in the second spot region is adjusted to a second value, which is greater than or equal to the third threshold.

5. The method according to claim 4, characterized in that, The second value is positively correlated with the first distance, which is negatively correlated with the distance between the second pixel and the center point of the second spot region.

6. The method according to any one of claims 1-2 and 5, characterized in that, Before adjusting the brightness value of the spot area in the first image to obtain the third image, the method further includes: Adjust the saturation of pixels in the spot region of the first image to obtain a fourth image; wherein the saturation of pixels in the spot region of the fourth image is greater than the saturation of pixels in the spot region of the first image; The pixel values ​​of the fourth image are adjusted to obtain the second image.

7. The method according to claim 6, characterized in that, The coordinates of a pixel in the YUV color coordinate system include: an abscissa and a ordinate, wherein the abscissa corresponds to the U value of the pixel and the ordinate corresponds to the V value of the pixel; Adjusting the saturation of pixels in the spot region of the first image includes: Adjust the coordinates of the pixels in the YUV color coordinate system of the spot area in the first image to adjust the saturation of the pixels; Wherein, the ratio of the vertical coordinate to the horizontal coordinate of the pixels in the spot region of the fourth image in the YUV color coordinate system is the same as the ratio of the vertical coordinate to the horizontal coordinate of the pixels in the spot region of the first image in the YUV color coordinate system. The saturation of a pixel is positively correlated with the second distance in the YUV color coordinate system, which is the distance between the pixel's coordinate point and the origin in the YUV color coordinate system.

8. The method according to claim 7, characterized in that, The third ratio is positively correlated with the third value; The third ratio is the ratio of the ordinate of the pixel in the spot region of the fourth image to the ordinate of the pixel in the spot region of the first image; The third value is the minimum value between the absolute values ​​of the x-coordinate and y-coordinate of a pixel in the YUV color coordinate system.

9. The method according to claim 8, characterized in that, The third ratio satisfies: Wherein, C is the third value, and the The gamma coefficient, the The x-coordinate of the pixel in the spot region of the first image in the YUV color coordinate system is given by the given coordinate. The vertical coordinate of the pixel in the spot region of the first image in the YUV color coordinate system.

10. The method according to any one of claims 7-9, characterized in that, If the saturation of the third pixel in the spot region of the first image is less than or equal to the fourth threshold, the saturation of the third pixel is increased.

11. The method according to any one of claims 1-10, characterized in that, Before adjusting the first image, the method further includes: Based on the brightness value of each pixel and the spot threshold in the first image, a first spot map of the first image is obtained. The connected components in the first spot map are used to indicate the spot regions in the first image, and the pixels in the connected components of the first spot map correspond to the pixels in the first image whose brightness values ​​are greater than or equal to the spot threshold.

12. The method according to any one of claims 1-10, characterized in that, After obtaining the first spot pattern of the first image, and before adjusting the first image, the method further includes: The connected components in the first spot image are adjusted based on the brightness values ​​of each pixel in the first image to obtain a second spot image, wherein the connected components of the second spot image correspond to the spot region in the first image. Wherein, if the difference between the light spot threshold and the brightness value corresponding to the fourth pixel is less than the fifth threshold, the fourth pixel is located in the connected domain of the second light spot image, and the fourth pixel is a pixel adjacent to the boundary of the connected domain of the first light spot image.

13. The method according to any one of claims 1-10, characterized in that, The acquisition of the first image includes: In response to the photo-taking operation, the first image is captured.

14. The method according to any one of claims 1-10, characterized in that, The method further includes: The first interface is displayed, and the first interface displays the first image and the first control; Acquire the first image, including: In response to a trigger operation on the first control, the first image is acquired.

15. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 14.

16. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 14.

18. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 14.

Images