Image processing method and device, storage medium and chip

Abstract

The invention relates to an image processing method and device, a storage medium and a chip. The image processing method comprises the following steps: acquiring pixel information of pixel points in a target image; the pixel information comprises a depth value of the pixel point and a relative position of the pixel point and the center of the target image; determining a blurring parameter of the pixel point according to the depth value and the relative position; the blurring parameters comprise strength parameters and direction parameters; and performing blurring processing on the target image according to the intensity parameter and the direction parameter. According to the image processing method, the blurring processing of simulating the rotating focus effect and the radiation effect can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to image processing methods, devices, storage media and chips. Background Art

[0002] In the related art, some image processing devices are limited in their image shooting functions and cannot shoot images with blur effects. Therefore, blur processing is required based on the images shot by the image processing devices. When blurring, a corresponding blur template is used to simulate a light spot effect for the image area that needs to be blurred. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, device, storage medium and chip, which can realize blurring processing of simulating rotation focus effect and radiation effect.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising: acquiring pixel information of a pixel point in a target image, the pixel information comprising a depth value of the pixel point and a relative position of the pixel point with respect to a center of the target image; determining a blurring parameter of the pixel point according to the depth value and the relative position, the blurring parameter comprising an intensity parameter and a direction parameter; and blurring the target image according to the intensity parameter and the direction parameter.

[0005] In an embodiment of the present disclosure, the relative position includes the distance between the pixel point and the center of the target image, and the angle between the pixel point and the center of the target image; determining the blurring parameter of the pixel point according to the depth value and the relative position includes: determining a first blurring intensity of the pixel point according to the depth value; determining a second blurring intensity according to the first blurring intensity, the distance between the pixel point and the center of the target image, and a preset distance; the intensity parameters include the first blurring intensity and the second blurring intensity; and determining the direction parameter according to the angle between the pixel point and the center of the target image.

[0006] In the disclosed embodiment, blurring the target image according to the intensity parameter and the direction parameter includes: determining a blur template corresponding to the pixel point according to the intensity parameter and the direction parameter; and blurring the target image according to the blur template.

[0007] In an embodiment of the present disclosure, the blur template includes an elliptical convolution kernel, the intensity parameters include a first blur intensity and a second blur intensity, and determining the blur template corresponding to the pixel point according to the intensity parameter and the direction parameter includes: performing image transformation processing on a preset circular convolution kernel template based on the intensity parameter and the direction parameter to obtain an elliptical convolution kernel; wherein the first blur intensity is the major axis of the elliptical convolution kernel, the second blur intensity is the minor axis of the elliptical convolution kernel, and the direction parameter characterizes the direction of the elliptical convolution kernel compared to the center of the target image.

[0008] In the disclosed embodiment, determining the blur template corresponding to the pixel point according to the intensity parameter and the direction parameter includes: determining the blur template corresponding to the pixel point from a plurality of preset blur templates according to the intensity parameter and the direction parameter.

[0009] In the embodiment of the present disclosure, the image processing method further includes: determining a preset number of blur templates according to a preset accuracy condition; presetting a plurality of blur templates according to the preset number, and determining intensity parameters and direction parameters respectively corresponding to the plurality of blur templates.

[0010] In an embodiment of the present disclosure, determining the first blurring intensity of the pixel point according to the depth value includes: determining the first blurring intensity according to the depth value, a preset blurring intensity and a preset depth.

[0011] In the embodiment of the present disclosure, the first blurring intensity is expressed as: K(i, j)=D(i, j) / Dmax*BH; wherein, (i, j) represents the pixel point, BH represents the preset blurring intensity, Dmax represents the preset depth, and D(i, j) represents the depth value.

[0012] In the embodiment of the present disclosure, the second blurring strength is expressed as: in, (i, j) represents the pixel point, represents the center of the target image, w represents the width of the target image, h represents the height of the target image, d max represents the preset distance, l r represents the blur intensity, d (i,j) represents the distance between the pixel point and the center of the target image; the direction parameter is expressed as:

[0013] In the disclosed embodiment, the target image is an image determined based on a target subject image, the target subject image includes a subject area and a non-subject area, and the target image corresponds to an area in the non-subject area that needs to be blurred.

[0014] According to a second aspect of an embodiment of the present disclosure, an image processing device is provided, comprising: an acquisition module, configured to acquire pixel information of a pixel in a target image; the pixel information comprises a depth value of the pixel and a relative position of the pixel to a center of the target image; a determination module, configured to determine a blurring parameter of the pixel according to the depth value and the relative position; the blurring parameter comprises an intensity parameter and a direction parameter; and a blurring module, configured to blur the target image according to the intensity parameter and the direction parameter.

[0015] According to a third aspect of an embodiment of the present disclosure, there is provided an image processing device, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to: execute the steps of the image processing method provided in the first aspect of the present disclosure.

[0016] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the image processing method provided in the first aspect of the present disclosure are implemented.

[0017] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, including a processor and an interface; the processor is used to read instructions to execute the steps of the image processing method provided in the first aspect of the present disclosure.

[0018] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: determining the blur parameters of the pixel points through the pixel information of the pixel points, and then realizing the blurring processing of the target image based on the blur parameters of the pixel points; since the blur parameters of the pixel points include not only intensity parameters but also direction parameters, the target image blurred based on the blur parameters can simulate the blurring effects based on the intensity parameters and direction parameters, including the rotation focus effect and the radiation effect; thereby, realizing the blurring processing simulating the rotation focus effect and the radiation effect.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] Figure 1The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0022] Figure 2A-2B is a schematic diagram of a virtual template according to an exemplary embodiment.

[0023] Figure 3 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0024] Figure 4 It is a block diagram of a device according to an exemplary embodiment (general structure of a mobile terminal).

[0025] Figure 5 It is a block diagram of a device (general structure of a server) according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0027] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0028] The technical solution in the embodiments of the present disclosure can be applied to various image blur processing scenarios that require simulated rotational focus effects and / or simulated radiation effects.

[0029] In some application scenarios, the images captured by the image acquisition device do not have a blurring effect, and it is necessary to add a blurring effect to the images; for example, devices such as mobile phones have a high image processing capability and high imaging requirements; however, due to the small aperture and short focal length of the mobile phone camera, it is impossible to capture images with a blurring effect, so the captured images need to be blurred.

[0030] In other application scenarios, the images captured by the image acquisition device have a blur effect, but the blur effect may not meet the user's needs, and corresponding blur processing can also be performed. For example, for professional camera equipment, if the user's shooting skills are poor, the user may not be satisfied with the blur effect of the image captured, and the image also needs to be blurred.

[0031] It can be understood that in addition to the above two application scenarios, the technical solution of the embodiment of the present disclosure can also be used in more scenarios involving blurring processing.

[0032] In the related technologies, in these application scenarios involving blur processing, background blur technology is used to achieve image blur processing. For example, the progressive background blur technology has a blur process including: subject target segmentation, depth estimation, background layering blurring the background, and fusion of the blurred background and subject target. Among them, the background layering blurring is based on the maximum blur intensity, and several blur parameters are set in a decreasing intensity manner, with weak blurring implemented at close range and strong blurring implemented at a distance.

[0033] In this background blur technology, the blur template is a single convolution kernel template. By changing the size of the convolution kernel, only background blur can be achieved, but the rotation focus blur effect and radial blur effect cannot be simulated.

[0034] Based on this, an embodiment of the present disclosure provides an image blurring solution, which performs image blurring based on intensity parameters and direction parameters, thereby achieving image blurring processing that simulates rotational focusing effects and radiation effects.

[0035] In addition, based on the asymptotic depth blur, this image blur solution adopts a dynamically adjusted elliptical convolution kernel or a more efficient quantized elliptical convolution kernel, applies different convolution kernels at different positions, and simulates the generation of rotational focus or radial blur effects, providing users with more blur options and a better blur experience.

[0036] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the image processing method can be applied to a terminal or a server, etc., and includes the following steps.

[0037] Step S11, obtaining pixel information of a pixel point in a target image, wherein the pixel information includes a depth value of the pixel point and a relative position between the pixel point and the center of the target image.

[0038] In step S11, the target image is an image that needs to be blurred.

[0039] In some embodiments, an image includes a subject part and a non-subject part, the subject part does not need to be blurred, and the non-subject part needs to be blurred. For example, an image includes a person and an environment, where the person can be the subject part and the environment can be the non-subject part.

[0040] Then, for the target image, it can be determined based on the target subject image, in which the target subject image includes a subject area and a non-subject area, and the target image corresponds to an area in the non-subject area that needs to be blurred.

[0041] In some embodiments, the entire non-main area needs to be blurred; in other embodiments, only a portion of the non-main area needs to be blurred.

[0042] In some embodiments, the image processing method further includes: determining a target image based on the target subject image.

[0043] In some embodiments, determining the target image based on the target subject image includes: determining a subject area and a non-subject area in the target subject image; and separating the subject area from the target subject image, wherein the separated image is the target image.

[0044] The non-subject area includes a background area and a mixed area; each pixel in the background area has no relationship with the subject, while the mixed area contains pixels related to the subject.

[0045] In some embodiments, the determination of the main body area and the non-main body area is achieved based on a preset segmentation model, wherein the preset segmentation model may be a segmentation model based on a deep learning algorithm, and the preset segmentation model corresponds to a training method and an application method.

[0046] In some embodiments, the u2net network model can be used as a segmentation model under the framework of pytorch (an open source Python machine learning library); the data set in the machine learning library can be used to adopt an iterative training method to implement the training of the segmentation model, thereby determining a preset segmentation model.

[0047] In some embodiments, the preset segmentation model corresponds to an input image requirement. Therefore, before applying the preset segmentation model to process the image, the image needs to be preprocessed, and then the processed image is input into the preset segmentation model, so that the preset segmentation model outputs the corresponding processing result.

[0048] For example, assuming that the image size matched by the preset segmentation model is 512*512, then for the target subject image, its image size is first reduced or enlarged to an image size of 512*512 and then input into the preset segmentation model.

[0049] In some embodiments, the segmentation result output by the preset segmentation model is an image represented by a grayscale matrix, in which the grayscale matrix includes the grayscale value of each pixel. For example, assuming that the grayscale matrix is ​​a 512*512 matrix M, and the value range of each element of the matrix M is [0, 255], then the value corresponding to the pixel in the background area is 0, and the value corresponding to the pixel in the mixed area is a value between 0-255. The closer to 255, the closer to the subject, and the value corresponding to the pixel in the subject area is 255.

[0050] In some embodiments, based on the values ​​of each pixel point, the subject can be separated from the target subject image by pixel-by-pixel operation to obtain the target image.

[0051] As an optional subject separation method, the image separation algorithm is expressed as: bg=I*(255-M) / 255; where 255 is the pixel grayscale value of the subject part (which can be changed according to different application scenarios), I represents the target subject image, M represents the image matrix output by the preset segmentation model, and bg represents the target image.

[0052] In other embodiments, if the target subject image is not divided into a portion to be blurred and a portion not to be blurred, the target subject image can be directly used as the target image, that is, the image to be blurred.

[0053] In some embodiments, the depth values ​​of the pixels in the target image may be determined during the process of processing the target subject image, that is, before separating the subject in the target subject image, the depth values ​​of the pixels are determined first, and then the subject separation operation is performed.

[0054] As an optional depth determination method, the depth of each pixel is determined based on a monocular depth estimation method. Specifically, the depth of each pixel in the target subject image is determined based on a preset depth estimation model. The preset depth estimation model corresponds to a training method and an application method.

[0055] In some embodiments, the deep learning algorithm and the corresponding network model structure adopted by the preset depth estimation model can refer to mature technologies in the field.

[0056] In some embodiments, the target subject image is input into a preset depth estimation model. After the preset depth estimation model processes the target subject image, it outputs a depth image corresponding to the target subject image, thereby determining the depth value of each pixel in the target subject image.

[0057] It can be understood that the preset depth estimation model may also correspond to the conditions of the input image. Therefore, before the target subject image is input into the preset depth estimation model, the target subject image can also be preprocessed so that the processed image matches the preset depth estimation model, so that the preset depth estimation model can output the corresponding depth determination result.

[0058] In some embodiments, the depth value of each pixel point is a relative depth value, which is a depth value relative to a preset reference object. The larger the depth value, the farther away from the preset reference object; the smaller the depth value, the closer to the preset reference object.

[0059] For example, in the process of determining the depth value, the size of the target subject image is first adjusted to match the preset depth estimation model, and then the preset depth estimation model is used to obtain the depth image D, and the depth values ​​in the depth image D are all relative depth values. Assuming that the depth image D is a 512*512 matrix, the value range of the elements in the matrix is ​​[0, 255]; 0 ​​represents the pixel closest to the reference object, and 255 represents the pixel farthest from the reference object; the pixel corresponding to the value between 0-255, the closer to 255, indicates that it is farther from the reference object.

[0060] In the disclosed embodiment, the preset depth estimation model can be used to accurately estimate the depth value of the image. Moreover, the estimated depth value is a relative depth value, which can characterize the distance of the pixel point compared to the preset reference object, so that the blur parameter determined based on the depth value is more in line with the actual situation of the image.

[0061] It can be understood that when blurring the target image, blurring needs to be performed on each pixel in the target image, and the blurring processing method for each pixel point is the same. In implementation, a pixel-by-pixel processing method is usually adopted. Therefore, in the embodiments of the present disclosure, the pixel points described can be understood as the pixel points of the target image, and the blurring processing method corresponding to each pixel point is the same.

[0062] Step S12, determining a blur parameter of the pixel point according to the depth value and the relative position.

[0063] In step S12, the relative position includes the distance between the pixel point and the center of the target image, and the angle between the pixel point and the center of the target image.

[0064] Then, as an optional implementation, step S12 includes: determining a first blur intensity of the pixel point based on the depth value; determining a second blur intensity based on the first blur intensity, the distance between the pixel point and the center of the target image, and a preset distance; the intensity parameters include the first blur intensity and the second blur intensity; determining a direction parameter based on the angle between the pixel point and the center of the target image.

[0065] In this embodiment, the blurring parameters include: a first blurring intensity, a second blurring intensity, and a direction parameter.

[0066] In the embodiment of the present disclosure, the first blur intensity can characterize the intensity of background blur, so the first blur intensity is associated with the depth value; the second blur intensity can characterize the intensity of a richer blur effect, such as: rotation intensity, radiation intensity, so it is associated with the distance between the pixel point and the center of the image.

[0067] The first blur strength is associated with the depth value of the pixel, so the greater the depth value of the pixel, the greater the first blur strength. The second blur strength is associated with the distance between the pixel and the center of the target image, so the greater the distance, the greater the second blur strength.

[0068] In some embodiments, determining a first blurring intensity of a pixel point according to a depth value includes: determining the first blurring intensity according to the depth value, a preset blurring intensity, and a preset depth.

[0069] The preset blur intensity can be understood as a preset maximum blur intensity, which can be set according to different application scenarios.

[0070] The preset depth may be the maximum depth value among the depth values ​​of each pixel point, or may be other preset values, which may be set according to different application scenarios. For example, the preset depth may be 255.

[0071] For example, the first blur intensity of a pixel point is expressed as: K(i, j)=D(i, j) / Dmax*BH; wherein, (i, j) represents the pixel point, BH represents the preset blur intensity, Dmax represents the preset depth, and D(i, j) represents the depth value of the pixel point.

[0072] In some implementations, the first blur strength of the pixel point can be represented by an integer. Then, the first blur strength of the pixel point can also be represented as: Indicates rounding down.

[0073] In other embodiments, in combination with different application scenarios, other corresponding relationships may be used between the depth value and the first blur strength. Based on other corresponding relationships, the first blur strength may be determined in a corresponding manner, which is not limited in the embodiments of the present disclosure.

[0074] In the embodiment of the present disclosure, the center of the target image can be understood as the geometric center of the target image; for example, assuming that the width and height of the target image are w and h respectively, the center of the target image is expressed as

[0075] It can be understood that according to the effect of the blur lens, the center position of the image does not rotate, and the closer the image is to the edge, the stronger the blur. Therefore, for an image, the second blur intensity can be enhanced and diffused from the center position to the surroundings. Therefore, the direction parameter and the second blur intensity of each pixel can be calculated based on the center of the image as a reference point.

[0076] For example, the second blur strength is expressed as: in, (i, j) represents a pixel point, represents the center of the target image, w represents the width of the target image, h represents the height of the target image, and d max Represents the preset distance, l r represents the blur intensity, d (i,j) Represents the distance between the pixel and the center of the target image; the direction parameter is expressed as:

[0077] The preset distance may be the farthest distance from the center of the target image, which may be understood as the distance between the four vertices of the target image and the center of the target image.

[0078] In the embodiment of the present disclosure, the first blur intensity can be used to achieve a progressive blur effect, and the second blur intensity and direction parameter can be used to achieve a rotational focus and radial blur effect. Thus, by using the blur parameters of the pixel point, a richer blur effect can be achieved on the basis of progressive depth blur.

[0079] Step S13, blurring the target image according to the intensity parameter and the direction parameter.

[0080] In the embodiment of the present disclosure, based on the intensity parameter and the direction parameter, a corresponding blur template may be generated first, and then blur processing may be implemented based on the blur template.

[0081] Therefore, as an optional implementation, step S13 includes: determining a blur template corresponding to the pixel point according to the intensity parameter and the direction parameter; and blurring the target image according to the blur template.

[0082] In an embodiment of the present disclosure, two methods for determining a blur template are provided. In one method, a blur template is constructed in real time based on intensity parameters and direction parameters. In another method, multiple blur templates are constructed in advance, and a corresponding blur template is directly selected based on intensity parameters and direction parameters.

[0083] It can be understood that the first blur intensity is associated with the long axis of the elliptical convolution kernel, the second blur intensity is quantized and associated with the short axis of the elliptical convolution kernel, and the directional parameters are further limited; on the basis of asymptotic depth blur, a dynamically adjusted elliptical convolution kernel or a more efficient quantized elliptical convolution kernel can be used, and different convolution kernels can be applied at different positions to simulate the generation of rotational focus or radial blur effects; providing users with more blur options and a better blur experience.

[0084] In some embodiments, the determination method is selected according to the blur processing requirements in the application scenario. For example, if the processing requirements have high accuracy requirements, a real-time construction determination method can be used; if the processing requirements have high efficiency requirements, a determination method based on a preset blur template can be used.

[0085] In the embodiment of the present disclosure, the blur template can be understood as an elliptical convolution kernel. For this convolution kernel, the length of its major axis is the first blur intensity, the length of its minor axis is the second blur intensity, and the angle between the pixel point and the center of the target image is the direction parameter; thus, the blur template can be determined based on the blur template parameters and the blur intensity.

[0086] Then, the real-time blur template construction process includes: based on the intensity parameter and the direction parameter, the preset circular convolution kernel template is subjected to image transformation processing to obtain an elliptical convolution kernel. Among them, the first blur intensity is the long axis of the elliptical convolution kernel, the second blur intensity is the short axis of the elliptical convolution kernel, and the direction parameter represents the direction of the elliptical convolution kernel relative to the center of the target image.

[0087] The preset circular convolution kernel module can be understood as a preset basic circular convolution kernel model, and the specific size can be preset according to different application scenarios.

[0088] In the disclosed embodiment, the accuracy of the blur template is ensured by constructing the blur template in real time based on the intensity parameter and the direction parameter.

[0089] As another optional implementation, the process of determining the blur template includes: determining the blur template corresponding to the pixel point from a plurality of preset blur templates according to the intensity parameter and the direction parameter.

[0090] It can be understood that the preset multiple blurring templates correspond to intensity parameters and direction parameters respectively, and based on the intensity parameter and direction parameter corresponding to the current pixel point, the blurring template corresponding to it can be determined.

[0091] In some embodiments, based on multiple blur templates, intensity parameters and direction parameters corresponding to the multiple blur templates, a corresponding relationship table can be configured, and the corresponding relationship table includes intensity parameters and direction parameters corresponding to different blur templates. Then, based on the intensity parameter and direction parameter of the current pixel point, the blur template can be quickly determined by searching the corresponding relationship table.

[0092] In some embodiments, the preset process of the blur template includes: determining a preset number of blur templates according to a preset accuracy condition; presetting multiple blur templates according to the preset number, and determining intensity parameters and direction parameters corresponding to the multiple blur templates respectively.

[0093] The preset accuracy condition can be expressed in the form of an accuracy value or an accuracy value range.

[0094] It can be understood that when the pixel position does not change much, the intensity parameter and direction parameter obtained may be the same. Therefore, if the accuracy represented by the preset accuracy condition is high, the number of blur templates required to be configured is large; if the accuracy represented by the preset accuracy condition is low, the number of blur templates required to be configured may be small.

[0095] Therefore, the preset number of blur templates can be determined based on the preset accuracy condition.

[0096] For example, in order to achieve a progressive blurring effect, all convolution kernels can be constructed starting from the maximum first blurring strength; for example, if the maximum first blurring strength is BH, all convolution kernels with major axis values ​​of [1, BH] need to be constructed. Specifically, for an elliptical convolution kernel with a major axis lr=10, multiple convolution kernels with a minor axis sr (i.e., the second blurring strength) value of [1, 10] need to be constructed.

[0097] In addition, the quantization angle (used to determine the number of directional parameters) needs to be considered. Assume that the quantization angle is denoted as q angle , since the image is symmetrical up and down, quantization only needs to consider 180°, so the number of quantization is q angle You can choose an angle that is divisible by 180°. If q angle =10, there are 18 direction parameters. Combined with the above example, when the major axis lr = 10, sr has 10 cases and 18 direction parameters, so 180 elliptical convolution kernels need to be constructed in advance. If lr = 5, there are a total of 5*180=90 elliptical convolution kernels.

[0098] Understandable, q angle Determines the accuracy of quantization. If the value is smaller, more elliptical convolution kernels need to be preset, and the convolution kernel accuracy at each position is higher. If the value is larger, fewer convolution kernels are required, but it is insensitive to the position changes of image pixels, and the convolution kernel accuracy is reduced.

[0099] As an example, q angle = 5. In the case of continuously adjusting the blur intensity, when sampling this quantization accuracy, the preset convolution kernel can show good performance, eliminating the operation of generating the convolution kernel in real time each time, and improving the efficiency of image blur.

[0100] In the embodiment of the present disclosure, a plurality of blur templates are preset, and when a blur template is needed, the corresponding blur template is directly selected, thereby improving the efficiency of image blurring.

[0101] As an example, see Figure 2A and Figure 2B, shows a schematic diagram of the blur template, which is an elliptical convolution kernel. Figure 2A In the figure, the convolution kernel corresponds to the rotation and focus effect. The inner convolution kernel is quantized at 90 degrees, and the outer convolution kernel is quantized at 45 degrees. At the same time, the major axis and minor axis of the inner convolution kernel are smaller than those of the outer convolution kernel, and the blurring and rotation and focus intensities of the outer convolution kernel are greater than those of the inner convolution kernel.

[0102] exist Figure 2B In the figure, the convolution kernel situation corresponding to the radiation effect is similar to the convolution kernel situation corresponding to the rotation focus effect. The inner convolution kernel is quantized at 90 degrees, and the outer convolution kernel is quantized at 45 degrees. At the same time, the major axis and minor axis of the inner convolution kernel are smaller than those of the outer convolution kernel, and the blurring and radiation intensities of the outer convolution kernel are greater than those of the inner convolution kernel.

[0103] exist Figure 2A and Figure 2B In the above example, the number of quantized elliptical convolution kernels is the same, but the distribution of multiple elliptical convolution kernels is different. Figure 2A The distribution of multiple elliptical convolution kernels in is a rotation distribution, and Figure 2B The distribution of multiple elliptical convolution kernels in is radial distribution, so, Figure 2A Multiple elliptical convolution kernels in can be used to simulate the rotation effect. Figure 2B Multiple elliptical convolution kernels in can be used to simulate the radiation effect.

[0104] In some embodiments, two blur templates may be preconfigured, one blur template is used to achieve a simulated rotation focus effect, and the other blur template is used to achieve a simulated radiation effect. When applied, the corresponding blur template is determined from the two blur templates according to actual blur requirements.

[0105] It can be understood that the blurring template is an elliptical convolution kernel, based on which the convolution algorithm of the elliptical convolution kernel is executed, and pixel-by-pixel operation processing is performed to achieve blurring of the target image. The implementation method of the convolution algorithm based on the convolution kernel refers to the mature technology in the art.

[0106] In some embodiments, since the target image is a background image extracted from the target main image, after the target image is blurred, the blurring operation of the background image is completed. In order to present the overall image blurring effect, the background image needs to be merged with the main image to determine the final blurred image.

[0107] Therefore, as an optional implementation, the image processing method further includes: synthesizing the blurred target image with the subject image separated based on the target subject image to determine a final blurred image.

[0108] Exemplarily, the formula O = I * M + BG * (1 - M) is adopted to fuse and generate a blurred background and a main body. Here, O is the final synthesized output image. Among them, I is the target main body image, M is the main body image, and BG is the background image. When M(i, j) = 1, O(i, j) = I(i, j), indicating that the value of the pixel point in the final output is the pixel value corresponding to the protected main body; when M(i, j) = 0, O(i, j) = BG(i, j), indicating that the value of the pixel point in the final output is the pixel value corresponding to the blurred background; when 0 < M(i, j) < 1, it indicates that the value of the pixel point in the final output is the result of fusing the protected main body and the blurred background according to M(i, j); the value of M(i, j) here can be understood as a fusion weight value, and its value is determined based on the output result of a preset segmentation model.

[0109] As can be seen from the introduction of the embodiments of the present disclosure, the blurring parameter of a pixel point is determined through the pixel information of the pixel point, and then, based on the blurring parameter of the pixel point, the blurring process of the target image is realized; since the blurring parameter of the pixel point includes not only an intensity parameter but also a direction parameter, the target image blurred based on the blurring parameter can simulate and realize the blurring effects based on the intensity parameter and the direction parameter, including the defocus effect and the radial effect; thus, the blurring process that simulates the defocus effect and the radial effect is realized.

[0110] Figure 3 It is a block diagram of an image processing device shown according to an exemplary embodiment. Refer to Figure 3 , the device 300 includes an acquisition module 301, a determination module 302, and a blurring module 303.

[0111] The acquisition module 301 is configured to acquire the pixel information of the pixel points in the target image; the pixel information includes the depth value of the pixel point and the relative position between the pixel point and the center of the target image; the determination module 302 is configured to determine the blurring parameter of the pixel point according to the depth value and the relative position; the blurring parameter includes an intensity parameter and a direction parameter; the blurring module 303 is configured to perform blurring processing on the target image according to the intensity parameter and the direction parameter.

[0112] In the embodiments of the present disclosure, the relative position includes the distance between the pixel point and the center of the target image, and the angle between the pixel point and the center of the target image; the determination module 301 is further configured to: determine the first blurring intensity of the pixel point according to the depth value; determine the second blurring intensity according to the first blurring intensity, the distance between the pixel point and the center of the target image, and a preset distance; the intensity parameter includes the first blurring intensity and the second blurring intensity; determine the direction parameter according to the angle between the pixel point and the center of the target image.

[0113] In the embodiment of the present disclosure, the determination module 301 is further configured to: determine the first blur strength according to the depth value, the preset blur strength and the preset depth.

[0114] In the embodiment of the present disclosure, the first blurring intensity is expressed as: K(i, j)=D(i, j) / Dmax*BH; wherein, (i, j) represents the pixel point, BH represents the preset blurring intensity, Dmax represents the preset depth, and D(i, j) represents the depth value of the pixel point.

[0115] In the embodiment of the present disclosure, the second blurring strength is expressed as: in, (i, j) represents the pixel point, represents the center of the target image, w represents the width of the target image, h represents the height of the target image, d max represents the preset distance, l r represents the blur intensity, d (i,j) represents the distance between the pixel point and the center of the target image; the direction parameter is expressed as:

[0116] In the embodiment of the present disclosure, the blur module 303 is further configured to: determine a blur template corresponding to the pixel point according to the intensity parameter and the direction parameter; and blur the target image according to the blur template.

[0117] In the embodiment of the present disclosure, the blurring module 303 is further configured to: perform image transformation processing on a preset circular convolution kernel template based on the intensity parameter and the direction parameter to obtain an elliptical convolution kernel; wherein the first blurring intensity is the long axis of the elliptical convolution kernel, the second blurring intensity is the short axis of the elliptical convolution kernel, and the direction parameter represents the direction of the elliptical convolution kernel relative to the center of the target image.

[0118] In the embodiment of the present disclosure, the blur module 303 is further configured to: determine the blur template corresponding to the pixel point from a plurality of preset blur templates according to the intensity parameter and the direction parameter.

[0119] In the disclosed embodiment, the blur module 303 is further configured to: determine a preset number of blur templates according to a preset accuracy condition; preset multiple blur templates according to the preset number, and determine intensity parameters and direction parameters corresponding to the multiple blur templates respectively.

[0120] In the disclosed embodiment, the target image is an image determined based on a target subject image, the target subject image includes a subject area and a non-subject area, and the target image corresponds to an area in the non-subject area that needs to be blurred.

[0121] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0122] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the image processing method provided by the present disclosure are implemented.

[0123] Figure 4 4 is a block diagram of a device 400 for implementing the image processing solution of the embodiment of the present disclosure according to an exemplary embodiment. For example, the device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0124] Reference Figure 4 , the device 400 may include one or more of the following components: a first processing component 402 , a first memory 404 , a first power component 406 , a multimedia component 408 , an audio component 410 , a first input / output interface 412 , a sensor component 414 , and a communication component 416 .

[0125] The first processing component 402 generally controls the overall operation of the device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The first processing component 402 may include one or more first processors 420 to execute instructions to complete all or part of the steps of the above-mentioned image processing method. In addition, the first processing component 402 may include one or more modules to facilitate the interaction between the first processing component 402 and other components. For example, the first processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the first processing component 402.

[0126] The first memory 404 is configured to store various types of data to support operations on the device 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phone book data, messages, pictures, videos, etc. The first memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access first memory (SRAM), electrically erasable programmable read-only first memory (EEPROM), erasable programmable read-only first memory (EPROM), programmable read-only first memory (PROM), read-only first memory (ROM), magnetic first memory, flash first memory, magnetic disk or optical disk.

[0127] The first power source component 406 provides power to various components of the device 400. The first power source component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 400.

[0128] The multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0129] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the device 400 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the first memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0130] The first input / output interface 412 provides an interface between the first processing component 402 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0131] The sensor assembly 414 includes one or more sensors for providing various aspects of the status assessment of the device 400. For example, the sensor assembly 414 can detect the open / closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400, and the sensor assembly 414 can also detect the position change of the device 400 or a component of the device 400, the presence or absence of user contact with the device 400, the orientation or acceleration / deceleration of the device 400, and the temperature change of the device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0132] The communication component 416 is configured to facilitate wired or wireless communication between the device 400 and other devices. The device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0133] In an exemplary embodiment, the device 400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned image processing method.

[0134] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a first memory 404 including instructions, and the instructions can be executed by a first processor 420 of the apparatus 400 to perform the above-mentioned image processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access first memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0135] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned image processing method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, such as the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction may be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned image processing method may be implemented; alternatively, the integrated circuit or chip may receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned image processing method.

[0136] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned image processing method when executed by the programmable device.

[0137] Figure 5 is a block diagram of a device 500 for implementing the image processing solution of the embodiment of the present disclosure according to an exemplary embodiment. For example, the device 500 may be provided as a server. Figure 5 The apparatus 500 includes a second processing component 522, which further includes one or more processors, and a second memory resource represented by a second memory 532 for storing instructions executable by the second processing component 522, such as an application. The application stored in the second memory 532 may include one or more modules, each corresponding to a set of instructions. In addition, the second processing component 522 is configured to execute the instructions to perform the above-mentioned image processing method.

[0138] The device 500 may further include a second power supply component 526 configured to perform power management of the device 500, a wired or wireless network interface 550 configured to connect the device 500 to a network, and a second input / output interface 558. The device 500 may operate based on an operating system stored in the second memory 532, such as Windows Server 2003. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0139] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0140] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Get pixel information of pixel points in the target image; The pixel information includes a depth value of the pixel and a relative position of the pixel and a center of the target image; Determine a blur parameter of the pixel point according to the depth value and the relative position; the blur parameter includes an intensity parameter and a direction parameter; The target image is blurred according to the intensity parameter and the direction parameter.

2. The image processing method according to claim 1, characterized in that: The relative position includes the distance between the pixel point and the center of the target image, and the angle between the pixel point and the center of the target image; Determining the blur parameter of the pixel point according to the depth value and the relative position includes: Determine a first blur intensity of the pixel point according to the depth value; Determining a second blur intensity according to the first blur intensity, the distance between the pixel point and the center of the target image, and a preset distance; the intensity parameter includes the first blur intensity and the second blur intensity; The direction parameter is determined according to an angle between the pixel point and the center of the target image.

3. The image processing method according to claim 1 or 2, characterized in that: The blurring the target image according to the intensity parameter and the direction parameter comprises: Determining a blur template corresponding to the pixel point according to the intensity parameter and the direction parameter; The target image is blurred according to the blur template.

4. The image processing method according to claim 3, characterized in that: The blur template includes an elliptical convolution kernel, the intensity parameter includes a first blur intensity and a second blur intensity, and determining the blur template corresponding to the pixel point according to the intensity parameter and the direction parameter includes: Based on the intensity parameter and the direction parameter, image transformation processing is performed on the preset circular convolution kernel template to obtain an elliptical convolution kernel; wherein the first blurring intensity is the long axis of the elliptical convolution kernel, the second blurring intensity is the short axis of the elliptical convolution kernel, and the direction parameter represents the direction of the elliptical convolution kernel compared to the center of the target image.

5. The image processing method according to claim 3, characterized in that: The determining, according to the intensity parameter and the direction parameter, a blurring template corresponding to the pixel point comprises: According to the intensity parameter and the direction parameter, a blurring template corresponding to the pixel point is determined from a plurality of preset blurring templates.

6. The image processing method according to claim 5, characterized in that: The image processing method further comprises: According to the preset accuracy condition, determining the preset number of blur templates; According to the preset number, a plurality of blurring templates are preset, and intensity parameters and direction parameters respectively corresponding to the plurality of blurring templates are determined.

7. The image processing method according to claim 2, characterized in that: The determining a first blurring intensity of the pixel point according to the depth value includes: The first blur strength is determined according to the depth value, a preset blur strength and a preset depth.

8. The image processing method according to claim 7, characterized in that: The first blur intensity is expressed as: K(i, j)=D(i, j) / Dmax*BH; wherein, (i, j) represents the pixel point, BH represents the preset blur intensity, Dmax represents the preset depth, and D(i, j) represents the depth value.

9. The image processing method according to claim 2, characterized in that: The second blur strength is expressed as: in, (i, j) represents the pixel point, represents the center of the target image, w represents the width of the target image, h represents the height of the target image, d max represents the preset distance, l r represents the blur intensity, d (i,j) represents the distance between the pixel and the center of the target image; The direction parameter is expressed as:

10. The image processing method according to claim 1, characterized in that: The target image is an image determined based on a target subject image, the target subject image includes a subject area and a non-subject area, and the target image corresponds to an area in the non-subject area that needs to be blurred.

11. An image processing device, characterized in that: include: An acquisition module is configured to acquire pixel information of a pixel point in a target image; The pixel information includes a depth value of the pixel and a relative position of the pixel and a center of the target image; A determination module, configured to determine a blur parameter of the pixel point according to the depth value and the relative position; the blur parameter includes an intensity parameter and a direction parameter; The blur module is configured to blur the target image according to the intensity parameter and the direction parameter.

12. An image processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the steps of the method described in any one of claims 1-10.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.

14. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 10.

Images