Image processing method and device

Abstract

The present application discloses an image processing method and device, belonging to the field of image processing technology. The method includes: obtaining a first image and setting a spot template; processing the first image to obtain a first blur radius map, a transformation brightness map, a highlight point map and a first blur radius map corresponding to the highlight point map corresponding to the first image; rotating the set spot template according to the second blur radius map, and obtaining a spot effect map corresponding to the first image according to the rotated set spot template and the highlight point map; obtaining a background blur map corresponding to the first image according to the first image, the first blur radius map, the set spot template and the transformation brightness map; obtaining a second image according to the spot effect map and the background blur map.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method and device. Background Art

[0002] In recent years, using mobile phones to take photos with blurred background has become a basic function on many mobile phones. As a result, the light spots on the blurred background have also become a focus of users when taking photos.

[0003] At present, the blurred effect with light spot synthesized by algorithm generally simulates the ideal SLR effect without aperture eclipse. However, for a real SLR, there will be aperture eclipse and phase field distortion, which causes the direction of the light spot to rotate only around the center of the image, that is, the direction of the light spot is consistent. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method that can solve the problem of consistent direction of existing image light spots.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Acquire a first image and set a spot template;

[0007] Processing the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map;

[0008] Rotating the set light spot template according to the second blur radius map, and obtaining a light spot effect map corresponding to the first image according to the rotated set light spot template and the highlight point map;

[0009] Obtaining a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map;

[0010] A second image is obtained according to the light spot effect image and the background blur image.

[0011] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0012] A first acquisition module, used for acquiring a first image and setting a light spot template;

[0013] A first processing module is used to process the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map;

[0014] a second processing module, configured to rotate the set light spot template according to the second blur radius map, and obtain a light spot effect map corresponding to the first image according to the rotated set light spot template and the highlight point map;

[0015] A second acquisition module, configured to obtain a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map;

[0016] The third processing module is used to obtain a second image according to the light spot effect image and the background blur image.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0020] In the embodiment of the present application, after acquiring the first image and setting the spot template, the first image is processed to obtain the highlight point map of the first image, the first blur radius map corresponding to the highlight point map, the second blur radius map corresponding to the first image, and the transformation brightness map, and the set spot template is rotated according to the second blur radius map, and the spot effect map is obtained according to the rotated set spot template and the highlight point map, and the background blur map is obtained according to the first image, the first blur radius map, the set spot template, and the transformation brightness map, and then the spot effect map and the background blur map are merged to obtain the second image. That is, it can generate different blur effects in different spot directions by setting the spot template, thereby improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flowchart of an image processing method provided in an embodiment of the present application;

[0022] Figure 2 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0023] Figure 3is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0024] Figure 4 It is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0028] Please see Figure 1 , which is a flow chart of an image processing method provided in an embodiment of the present application. The method can be applied to an electronic device, which can be a mobile phone, a tablet computer, a laptop computer, etc. Figure 1 As shown, the method may include steps 1100 to 1500, which are described in detail below.

[0029] Step 1100, acquiring a first image and setting a spot template.

[0030] The first image may be a preview image displayed on a shooting preview interface of the electronic device, or may be any image displayed in an album application of the electronic device.

[0031] When the first image is a preview image displayed on a shooting preview interface of an electronic device, it indicates that a user is calling a camera application of the electronic device to take a photo, and the image generated after taking the photo is the second image.

[0032] In the case where the first image is any image displayed in the album application of the electronic device, it indicates that the user needs to process any image that has been taken, and the image generated after the processing is the second image.

[0033] The set spot template may be a circular spot template, a five-pointed star spot template, or a butterfly spot template. Of course, the set spot template may also be a spot template of other shapes, which will not be described in detail in this embodiment.

[0034] After acquiring the first image and setting the spot template, enter:

[0035] Step 1200 , processing the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map.

[0036] In this embodiment, the first image is processed in step 1200 to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map, which may further include the following steps 1210 to 1250:

[0037] Step 1210: Obtain a depth map corresponding to the first image based on the first image.

[0038] The pixel value corresponding to each pixel point in the depth map may be referred to as a depth value, and the depth value is used to indicate the actual distance between the camera device in the electronic device and each point in the captured scene.

[0039] In this step 1210, the first image can be input into the depth estimation network module in the deep learning network, so that a depth map corresponding to the first image can be obtained.

[0040] Step 1220: Obtain a first blur radius map corresponding to the depth map according to the depth map.

[0041] The pixel value corresponding to each pixel point in the first blur radius map may be referred to as a blur radius value, and the blur radius value is used to indicate the blur degree of each point in the depth map.

[0042] In this step 1220, the depth of field range can be determined according to the depth map and the focus point information clicked by the user, and the blur radius value corresponding to the depth value of each pixel point in different depth maps is calculated to obtain a depth of field mask map and a first blur radius map.

[0043] Step 1230: Obtain the transformed brightness map according to the first image and the camera response function.

[0044] The camera response function can reflect the mapping relationship between the real brightness of each point in the shooting scene and the pixel value corresponding to each pixel in the first image, wherein the pixel value corresponding to the pixel in the first image is the brightness value corresponding to the pixel in the first image.

[0045] The pixel value corresponding to each pixel point in the transformed brightness map is called a transformed brightness value, and the transformed brightness value is used to represent the real brightness of each point in the shooting scene.

[0046] In this step 1230, the first image may be transformed by a camera response function to obtain a corresponding transformed brightness map.

[0047] Step 1240: Perform highlight detection on the transformed brightness map to obtain the highlight point map.

[0048] The highlight point map can reflect the position of the light source point in the shooting scene, that is, the highlight point map is an image that marks the position of the light source point in the shooting scene, and the position of the light source point can be called a highlight position.

[0049] In this step 1240, highlight detection can be performed on the transformed brightness map row by row, and the pixel values ​​corresponding to the pixel points that are greater than or equal to the average pixel value of the transformed brightness map are set to 255, and the pixel values ​​corresponding to the pixel points that are less than the average pixel value of the transformed brightness map are set to 0. In other words, the pixel points with pixel values ​​set to 255 are highlight points.

[0050] Step 1250: Obtain a second blur radius map corresponding to the highlight point map according to the highlight point map and the first blur radius map.

[0051] The second blur radius map may reflect blur radius values ​​corresponding to the highlight positions in the highlight point map, and the blur radius values ​​corresponding to the highlight positions in the highlight point map are used to indicate the blur degree of the highlight positions in the highlight point map.

[0052] In this step 1250, the first blur radius map may be processed according to the highlight point map to obtain a processed second blur radius map as the second blur radius map.

[0053] Specifically, the highlight point map can be expanded according to the blur radius value of the corresponding position. It can be understood that the highlight point map can be expanded according to the blur radius value of the corresponding position so that the highlight area in the highlight point map can be enlarged. Here, only the blur radius value corresponding to the expanded area is retained, and the blur radius values ​​corresponding to other areas are set to 0, so as to obtain a second blur radius map.

[0054] After the first image is processed to obtain a first blur radius map, a transformed brightness map, a highlight point map, and a second blur radius map corresponding to the highlight point map of the first image, proceed to:

[0055] Step 1300: Rotate the set light spot template according to the second blur radius map, and obtain a light spot effect map corresponding to the first image according to the rotated set light spot template and the highlight point map.

[0056] The light spot directions in the light spot effect diagram corresponding to the first image are different, that is, the light spot effect diagram corresponding to the first image is a light spot effect with different rotation directions.

[0057] In this embodiment, in step 1300, the set spot template is rotated according to the second blur radius map, and the spot effect map corresponding to the first image is obtained according to the rotated set spot template and the highlight point map, which may further include the following steps 1310 to 1330:

[0058] Step 1310: adjust the set light spot template according to the pixel value of each first pixel point in the second blur radius map to obtain a light spot template corresponding to each first pixel point.

[0059] The first pixel point may be a pixel point whose pixel value corresponding to the pixel point in the first virtual radius map is not zero.

[0060] Taking the setting of the spot template as a circular spot template as an example, the first pixel point in the second blur radius map can be traversed, and the circular spot template can be adjusted according to the pixel value of the first pixel point to obtain a spot template corresponding to each first pixel point. In other words, the different spot templates corresponding to different first pixel points obtained are circular spot templates of different sizes. For example, when the pixel value of the first pixel point is large, the circular spot template is adjusted to be larger to obtain the spot template corresponding to the first pixel point, and when the pixel value of the first pixel point is small, the circular spot template is adjusted to be smaller to obtain the spot template corresponding to the first pixel point, that is, the adjustment value of the circular spot template is proportional to the pixel value of the first pixel point.

[0061] Taking the five-pointed star light spot template as an example, the first pixel point in the second blur radius map can be traversed, and the five-pointed star light spot template can be adjusted according to the pixel value of the first pixel point to obtain a light spot template corresponding to each first pixel point. In other words, the different light spot templates corresponding to different first pixel points are five-pointed star light spot templates of different sizes.

[0062] Step 1320: Rotate the light spot template corresponding to each of the first pixel points based on a first set rotation direction to obtain a target light spot template corresponding to each of the first pixel points.

[0063] The first set rotation direction may be a tangent direction of a vector between the rotation center point and the first pixel point in the second blur radius map, wherein the rotation center point may be the image center of the first image or the focus point clicked by the user.

[0064] In one example, in step 1320, rotating the light spot template corresponding to each of the first pixel points based on the first set rotation direction to obtain the target light spot template corresponding to each of the first pixel points may further include:

[0065] The light spot template corresponding to each of the first pixel points is rotated based on a first set rotation direction to obtain a rotated light spot template corresponding to each of the first pixel points as a target light spot template corresponding to each of the first pixel points.

[0066] Continuing with the example of setting the spot template to a circular spot template, after obtaining the spot template corresponding to each first pixel point in the second blurred radius map according to the above step 1310, the spot template corresponding to each first pixel point can be rotated and transformed based on the first set rotation direction to obtain the rotated spot template, and the first set rotation direction is the vector tangent direction of the rotation center point and the corresponding first pixel point.

[0067] Continuing with the example of setting the spot model to a five-pointed star spot template, after obtaining the spot template corresponding to each first pixel point in the second blurred radius map according to step 1310 above, the spot template corresponding to each first pixel point can be rotated based on the first set rotation direction to obtain the rotated spot template.

[0068] According to this example, it is possible to generate blurring effects with different light spot directions for the first image based on a set light spot template, thereby improving the existing problem of a single light spot direction.

[0069] In one example, in step 1320, rotating the light spot template corresponding to each of the first pixel points based on the first set rotation direction to obtain the target light spot template corresponding to each of the first pixel points may further include the following steps 1321 to 1322:

[0070] Step 1321: Rotate the light spot template corresponding to each of the first pixel points based on the first set rotation direction to obtain a rotated light spot template corresponding to each of the first pixel points.

[0071] Continuing with the example of setting the spot template to a circular spot template, after obtaining the spot template corresponding to each first pixel point in the first blurred radius map according to the above step 1310, the spot template corresponding to each first pixel point can be rotated and transformed based on the first set rotation direction to obtain the rotated spot template, and the first set rotation direction is the vector tangent direction of the rotation center point and the corresponding first pixel point.

[0072] Step 1322: stretch the rotated light spot template corresponding to each of the first pixel points based on the first set stretching parameter to obtain a target light spot template corresponding to each of the first pixel points.

[0073] The first set stretching parameter includes a stretching direction and a stretching ratio value, and the larger the stretching ratio value, the more stretching. The stretching direction includes the x direction and the y direction of the first pixel point.

[0074] Continuing to take the setting of the spot template as a circular spot template as an example, after obtaining the rotated spot template corresponding to each first pixel point in the second blur radius map according to the above step 1321, the rotated spot template corresponding to each first pixel point can be stretched and deformed based on the first set stretching parameter to obtain a stretched spot template as the target spot template corresponding to each first pixel point. It can be understood that the target spot templates corresponding to different first pixel points are different olive-shaped spot templates.

[0075] According to this example, it is possible to generate olive-shaped spot templates with different rotation directions and stretching deformations for the first image based on the circular spot template, thereby simulating blurring effects with different spot directions.

[0076] Step 1330: Obtain a light spot effect map corresponding to the first image according to the target light spot template corresponding to each of the first pixel points and the highlight point map.

[0077] In this step 1330, the target light spot template corresponding to each first pixel point may be used to filter the highlight point image, thereby obtaining a light spot effect image corresponding to the first image.

[0078] Continuing with the example of setting the spot template to a circular spot template, after obtaining the olive-shaped spot template corresponding to each first pixel point according to step 1320, the highlight point image can be filtered based on the olive-shaped spot template corresponding to each first pixel point to obtain a spot effect image.

[0079] Continuing with the example of setting the spot template to a five-pointed star spot template, after obtaining the five-pointed star spot template corresponding to each first pixel point according to step 1320, the highlight point image can be filtered based on the five-pointed star spot template corresponding to the first pixel point to obtain a spot effect image.

[0080] After rotating the set light spot template according to the second blur radius map, and obtaining the light spot effect map corresponding to the first image according to the rotated set light spot template and the highlight point map, proceed to:

[0081] Step 1400: Obtain a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map.

[0082] The background blur image includes a first background blur image and a second background blur image, wherein the first background blur image is a background blur effect image with a sense of rotation in a brightness channel, and the second background blur image is a background blur effect image with a sense of rotation in a color channel.

[0083] In this embodiment, in step 1400, obtaining the background blur map corresponding to the first image according to the first image, the first blur radius map, the set spot template and the transformed brightness map may further include the following steps 1410 to 1420:

[0084] Step 1410: adjust the set light spot template according to the pixel value of each second pixel point in the first blur radius map to obtain a light spot template corresponding to the pixel value of each second pixel point.

[0085] The second pixel point may be a pixel point whose pixel value corresponding to the pixel point in the first virtual radius map is not zero.

[0086] Taking the setting of the spot template as a circular spot template as an example, the second pixel in the first background blur image can be traversed, and the circular spot template can be adjusted according to the pixel value of the second pixel to obtain a spot template corresponding to each second pixel. In other words, the different spot templates corresponding to different second pixels are circular spot templates of different sizes. For example, when the pixel value of the second pixel is large, the circular spot template can be adjusted to be larger to obtain the spot template corresponding to the second pixel. When the pixel value of the second pixel is small, the circular spot template can be adjusted to be smaller to obtain the spot template corresponding to the second pixel, that is, the adjustment value of the circular spot template is proportional to the pixel value of the second pixel.

[0087] Taking the five-pointed star light spot template as an example, the second pixel point in the first background blur image can be traversed, and the five-pointed star light spot template can be adjusted according to the pixel value of the second pixel point to obtain a light spot template corresponding to each second pixel point. In other words, the different light spot templates corresponding to different second pixel points are five-pointed star light spot templates of different sizes.

[0088] Step 1420: Obtain a first background blur image and a second background blur image according to the light spot template corresponding to each of the second pixel points, the brightness transformation map and the first image.

[0089] In this embodiment, in step 1420, obtaining the first background blur image and the second background blur image according to the spot template corresponding to each second pixel point, the brightness transformation map and the first image may further include the following steps 1421 to 1423:

[0090] Step 1421: Rotate the light spot template corresponding to each of the second pixel points based on a second set rotation direction to obtain a target light spot template corresponding to each of the second pixel points.

[0091] The second set rotation direction may be a tangent direction of a vector between the rotation center point and the second pixel point in the first blur radius map, wherein the rotation center point may be the image center of the first image or the focus point clicked by the user.

[0092] In one example, in step 1421, rotating the light spot template corresponding to each second pixel point based on the second set rotation direction to obtain the target light spot template corresponding to each second pixel point may further include:

[0093] The light spot template corresponding to each of the second pixel points is rotated based on the second set rotation direction to obtain the rotated light spot template corresponding to each of the second pixel points as the target light spot template corresponding to each of the second pixel points.

[0094] Continuing with the example of setting the spot template to a circular spot template, after obtaining the spot template corresponding to each second pixel point in the first blurred radius map according to the above step 1410, the spot template corresponding to each second pixel point can be rotated and transformed based on the second set rotation direction to obtain the rotated spot template, and the second set rotation direction is the vector tangent direction of the rotation center point and the corresponding second pixel point.

[0095] Continuing with the example of setting the spot model to a five-pointed star spot template, after obtaining the spot template corresponding to each second pixel point in the first blurred radius map according to the above step 1410, the spot template corresponding to each second pixel point can be rotated based on the second set rotation direction to obtain the rotated spot template.

[0096] According to this example, it is possible to generate a background blur effect with a sense of rotation for the brightness channel of the first image based on a set light spot template.

[0097] In one example, in step 1421, the spot template corresponding to each second pixel point is rotated based on the second set rotation direction to obtain the target spot template corresponding to each second pixel point, and it can also further include: rotating the spot template corresponding to each second pixel point based on the second set rotation direction to obtain the rotated spot template corresponding to each second pixel point; stretching the rotated spot template corresponding to each second pixel point based on the second set stretching parameter to obtain the target spot template corresponding to each second pixel point.

[0098] The second set stretching parameter includes a stretching direction and a stretching ratio value, and the larger the stretching ratio value, the more stretching. The stretching direction includes the x direction and the y direction of the second pixel point.

[0099] Continuing to take the setting of the spot template as a circular spot template as an example, after obtaining the rotated spot template corresponding to each second pixel point in the first blur radius map according to the above step 1410, the rotated spot template corresponding to each second pixel point can be stretched and deformed based on the second set stretching parameter to obtain a stretched spot template as the target spot template corresponding to each second pixel point. It can be understood that the target spot templates corresponding to different second pixel points are different olive-shaped spot templates.

[0100] According to this example, it is possible to generate olive-shaped spot templates with different rotation directions and stretching deformations for the first image based on the circular spot template, thereby simulating blurring effects with different spot directions.

[0101] Step 1422: Obtain the first background blur image according to the target light spot template corresponding to each of the second pixel points and the brightness transformation map.

[0102] In this step 1422, the target spot template corresponding to each second pixel point may be used to filter the brightness transformation image, thereby obtaining a first background blur image corresponding to the first image.

[0103] Continuing with the example of setting the spot template to a circular spot template, after obtaining the olive-shaped spot template corresponding to each second pixel point according to step 1421, the brightness transformation map can be filtered based on the olive-shaped spot template corresponding to each second pixel point to obtain the first background blur map.

[0104] Continuing with the example of setting the spot template to a five-pointed star spot template, after obtaining the five-pointed star spot template corresponding to each second pixel point according to step 1421, the brightness transformation map can be filtered based on the five-pointed star spot template corresponding to each second pixel point to obtain the first background blur map.

[0105] Step 1423, obtaining the second background blur image according to the target light spot template corresponding to each of the second pixel points and the first image.

[0106] In step 1423, the first image may be filtered using a target light spot template corresponding to each second pixel point, thereby obtaining a second background blur image corresponding to the first image.

[0107] Continuing with the example of setting the spot template to a circular spot template, after obtaining the olive-shaped spot template corresponding to each second pixel point according to step 1421, the first image can be filtered based on the olive-shaped spot template corresponding to each second pixel point to obtain a second background blur image.

[0108] Continuing with the example of setting the spot template to a five-pointed star spot template, after obtaining the five-pointed star spot template corresponding to each second pixel point according to step 1421, the first image can be filtered based on the five-pointed star spot template corresponding to each second pixel point to obtain a second background blur image.

[0109] After obtaining the background blur map corresponding to the first image according to the first image, the first blur radius map, the set spot template and the transformed brightness map, proceed to:

[0110] Step 1500: Obtain a second image according to the light spot effect image and the background blur image.

[0111] The second image is an out-of-focus rotation blur result image with a different light spot direction and a rotating blurred background.

[0112] In this embodiment, obtaining the second image according to the light spot effect image and the background blur image in step 1500 may further include the following steps 1510 to 1530:

[0113] Step 1510: Obtain a third image according to the light spot effect image and the first background blur effect image.

[0114] The third image is a blurred background rotation and a brightness channel blurred effect diagram with different light spot directions.

[0115] In this step 1510, the spot effect image and the first background blur image can be superimposed to enhance the brightness value of the spot, and the inverse transformation of the camera response function can be performed to obtain a brightness channel blur effect image with different blurred background rotation and spot directions.

[0116] Step 1520: Obtain a fourth image according to the light spot effect image and the second background blur effect image.

[0117] The fourth image is a blurred background rotation and a blurred color channel with different light spot directions

[0118] In this step 1520, the spot effect image and the second background blur image may be superimposed to enhance the color saturation information of the spot area, thereby obtaining a color channel blur effect image with different background rotation and spot directions.

[0119] Step 1530: Obtain the second image according to the third image and the fourth image.

[0120] In this step 1530, the third image and the fourth image are fused to obtain a second image.

[0121] According to this embodiment, after acquiring the first image and setting the spot template, the first image is processed to obtain the highlight point map of the first image, the first blur radius map corresponding to the highlight point map, the second blur radius map corresponding to the first image, and the transformation brightness map, and the set spot template is rotated according to the second blur radius map, and the spot effect map is obtained according to the rotated set spot template and the highlight point map, and the background blur map is obtained according to the first image, the first blur radius map, the set spot template, and the transformation brightness map, and then the spot effect map and the background blur map are merged to obtain the second image. That is, it can generate different blur effects in different spot directions by setting the spot template, thereby improving the user's viewing experience.

[0122] Corresponding to the above embodiment, see Figure 2 The embodiment of the present application further provides an image processing device 200, comprising:

[0123] The acquisition module 210 is used to acquire the first image and set the spot template.

[0124] The first processing module 220 is used to process the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map.

[0125] The second processing module 230 is used to rotate the set spot template according to the second blur radius map, and obtain a spot effect map corresponding to the first image according to the rotated set spot template and the highlight point map.

[0126] The third acquisition module 240 is used to obtain a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map.

[0127] The fourth processing module 250 is used to obtain a second image according to the light spot effect image and the background blur image.

[0128] In one embodiment, the first processing module 220 is specifically used to: obtain a depth map corresponding to the first image based on the first image; obtain a first blur radius map corresponding to the depth map based on the depth map; obtain the transformed brightness map based on the first image and the camera response function; perform highlight detection on the transformed brightness map to obtain the highlight point map; obtain a second blur radius map corresponding to the highlight point map based on the highlight point map and the first blur radius map.

[0129] In one embodiment, the second processing module 230 is specifically used to: adjust the set spot template according to the pixel value of each first pixel point in the second blurred radius map to obtain a spot template corresponding to each first pixel point; rotate the spot template corresponding to each first pixel point based on the first set rotation direction to obtain a target spot template corresponding to each first pixel point; obtain a spot effect map corresponding to the first image according to the target spot template corresponding to each first pixel point and the highlight point map.

[0130] In one embodiment, the background blur image includes a first background blur image and a second background blur image.

[0131] The third processing module 240 is specifically used to: adjust the set spot template according to the pixel value of each second pixel point in the first blur radius map to obtain the spot template corresponding to each second pixel point; obtain the first background blur map and the second background blur map according to the spot template corresponding to each second pixel point, the transformed brightness map and the first image.

[0132] In one embodiment, the third processing module 240 is specifically used to: rotate the light spot template corresponding to each of the second pixel points based on a second set rotation direction to obtain a target light spot template corresponding to each of the second pixel points; obtain the first background blur image according to the target light spot template corresponding to each of the second pixel points and the brightness transformation map; and obtain the second background blur image according to the target light spot template corresponding to each of the second pixel points and the first image.

[0133] The image processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0134] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0135] The image processing device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0136] Corresponding to the above embodiment, optionally, Figure 3 As shown, an embodiment of the present application also provides an electronic device 300, including a processor 301, a memory 302, and a program or instruction stored in the memory 302 and executable on the processor 301. When the program or instruction is executed by the processor 301, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0137] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0138] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0139] The electronic device 400 includes but is not limited to components such as a radio frequency unit 401 , a network module 402 , an audio output unit 403 , an input unit 404 , a sensor 405 , a display unit 406 , a user input unit 407 , an interface unit 408 , a memory 409 , and a processor 410 .

[0140] Those skilled in the art will appreciate that the electronic device 400 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 410 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0141] Among them, the processor 410 is used to obtain a first image and a set spot template; process the first image to obtain a first blur radius map, a transformation brightness map, a highlight point map and a second blur radius map corresponding to the highlight point map corresponding to the first image; rotate the set spot template according to the second blur radius map, and obtain the spot effect map corresponding to the first image according to the rotated set spot template and the highlight point map; obtain the background blur map corresponding to the first image according to the first image, the first blur radius map, the set spot template and the transformation brightness map; obtain the second image according to the spot effect map and the background blur map.

[0142] In one embodiment, the processor 410 is used to obtain a depth map corresponding to the first image based on the first image; obtain a first blur radius map corresponding to the depth map based on the depth map; obtain the transformed brightness map based on the first image and the camera response function; perform highlight detection on the transformed brightness map to obtain the highlight point map; obtain a second blur radius map corresponding to the highlight point map based on the highlight point map and the first blur radius map.

[0143] In one embodiment, the processor 410 is also used to adjust the set spot template according to the pixel value of each first pixel point in the second blurred radius map to obtain a spot template corresponding to each first pixel point; rotate the spot template corresponding to each first pixel point based on the first set rotation direction to obtain a target spot template corresponding to each first pixel point; and obtain a spot effect map corresponding to the first image according to the target spot template corresponding to each first pixel point and the highlight point map.

[0144] In one embodiment, the processor 410 is also used to adjust the set spot template according to the pixel value of each second pixel point in the first blurred radius map to obtain a spot template corresponding to each second pixel point; and obtain a first background blurred image and a second background blurred image according to the spot template corresponding to each second pixel point, the transformed brightness map and the first image.

[0145] In one embodiment, the processor 410 is also used to rotate the light spot template corresponding to each of the second pixel points based on a second set rotation direction to obtain a target light spot template corresponding to each of the second pixel points; obtain the first background blur image according to the target light spot template corresponding to each of the second pixel points and the brightness transformation map; and obtain the second background blur image according to the target light spot template corresponding to each of the second pixel points and the first image.

[0146] It should be understood that in the embodiment of the present application, the input unit 404 may include a graphics processor (GPU) 4041 and a microphone 4042, and the graphics processor 4041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 409 may be used to store software programs and various data, including but not limited to applications and operating systems. The processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 410.

[0147] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0148] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0149] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0150] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0151] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0153] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image and set a spot template; Processing the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map; Rotating the set light spot template according to the second blur radius map, and obtaining a light spot effect map corresponding to the first image according to the rotated set light spot template and the highlight point map; Obtaining a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map; Obtaining a second image according to the light spot effect image and the background blur image; The step of rotating the set spot template according to the second blur radius map, and obtaining the spot effect map corresponding to the first image according to the rotated set spot template and the highlight point map comprises: adjusting the set spot template according to the pixel value of each first pixel point in the second blur radius map to obtain a spot template corresponding to each first pixel point; rotating the spot template corresponding to each first pixel point based on the first set rotation direction to obtain a target spot template corresponding to each first pixel point; The first pixel point is a pixel point whose corresponding pixel value in the first virtual radius map is not zero; the first set rotation direction is the vector tangent direction between the rotation center point and the first pixel point in the second virtual radius map.

2. The method according to claim 1, characterized in that The processing of the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map includes: Obtaining a depth map corresponding to the first image according to the first image; According to the depth map, obtaining a first blur radius map corresponding to the depth map; Obtaining a transformed brightness map according to the first image and the camera response function; Performing highlight detection on the transformed brightness map to obtain the highlight point map; A second blur radius map corresponding to the highlight point map is obtained according to the highlight point map and the first blur radius map.

3. The method according to claim 1, characterized in that The background blur image includes a first background blur image and a second background blur image. The step of obtaining a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map comprises: According to the pixel value of each second pixel point in the first blur radius map, the set light spot template is adjusted to obtain a light spot template corresponding to each second pixel point; A first background blur image and a second background blur image are obtained according to the light spot template corresponding to each second pixel point, the transformed brightness map and the first image.

4. The method according to claim 3, characterized in that The step of obtaining a first background blur image and a second background blur image according to the light spot template corresponding to each second pixel point, the transformed brightness map and the first image comprises: Rotate the light spot template corresponding to each of the second pixel points based on a second set rotation direction to obtain a target light spot template corresponding to each of the second pixel points; Obtaining the first background blur image according to the target light spot template corresponding to each of the second pixel points and the transformed brightness image; and The second background blur image is obtained according to the target light spot template corresponding to each second pixel point and the first image.

5. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first image and setting a light spot template; A first processing module is used to process the first image to obtain a first blur radius map, a transformed brightness map, a highlight point map corresponding to the first image, and a second blur radius map corresponding to the highlight point map; a second processing module, configured to rotate the set light spot template according to the second blur radius map, and obtain a light spot effect map corresponding to the first image according to the rotated set light spot template and the highlight point map; A third processing module, configured to obtain a background blur map corresponding to the first image according to the first image, the first blur radius map, the set light spot template and the transformed brightness map; A fourth processing module, configured to obtain a second image according to the light spot effect image and the background blur image; The second processing module is specifically configured to adjust the set spot template according to the pixel value of each first pixel in the second blur radius map to obtain a spot template corresponding to each first pixel; rotate the spot template corresponding to each first pixel based on the first set rotation direction to obtain a target spot template corresponding to each first pixel; and obtain a spot effect map corresponding to the first image according to the target spot template corresponding to each first pixel and the highlight point map; The first pixel point is a pixel point whose corresponding pixel value in the first virtual radius map is not zero; the first set rotation direction is the vector tangent direction between the rotation center point and the first pixel point in the second virtual radius map.

6. The device according to claim 5, characterized in that The first processing module is specifically used for: Obtaining a depth map corresponding to the first image according to the first image; According to the depth map, obtaining a first blur radius map corresponding to the depth map; Obtaining the transformed brightness map according to the first image and the camera response function; Performing highlight detection on the transformed brightness map to obtain the highlight point map; A second blur radius map corresponding to the highlight point map is obtained according to the highlight point map and the first blur radius map.

7. The device according to claim 5, characterized in that The background blur image includes a first background blur image and a second background blur image. The third processing module is specifically used for: According to the pixel value of each second pixel point in the first blur radius map, the set light spot template is adjusted to obtain a light spot template corresponding to each second pixel point; A first background blur image and a second background blur image are obtained according to the light spot template corresponding to each second pixel point, the transformed brightness map and the first image.

8. The device according to claim 7, characterized in that The third processing module is specifically used for: Rotate the light spot template corresponding to each of the second pixel points based on a second set rotation direction to obtain a target light spot template corresponding to each of the second pixel points; Obtaining the first background blur image according to the target light spot template corresponding to each of the second pixel points and the transformed brightness image; The second background blur image is obtained according to the target light spot template corresponding to each second pixel point and the first image.

Images