Photographing method and device, electronic equipment and storage medium

Abstract

The invention relates to a photographing method and device, electronic equipment and a storage medium. The photographing method comprises the following steps: acquiring each color channel image in three-primary color channels of an image acquired by a camera of the electronic equipment during photographing, and respectively acquiring a blurring kernel for deblurring processing for each color channel image; and performing deblurring processing on each color channel image based on the corresponding blurring kernel to obtain a photographed image. According to the invention, deblurring processing can be carried out on each color channel image in the three-primary color channel by using the corresponding blurring kernel, and the photographed image is obtained based on each processed color channel image, so that deblurring and color edge removal processing can be carried out on the image at the same time, the image quality is enhanced, and the photographing experience of a user is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image processing, and in particular, to a photographing method and apparatus, an electronic device, and a storage medium. BACKGROUND

[0002] With the rapid development of mobile electronic devices, the number of pixels of a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor of an electronic device is increasing, and the spatial sampling frequency thereof is also increasing, which requires a higher Modulation Transfer Function (MTF) of a high-frequency modulation of a lens. Monochromatic aberration of an optical lens can cause blurring of an image, and chromatic aberration can introduce color fringes of other colors. The closer to the edge of a field of view, the more serious the blurring and color fringes caused by aberration. However, the related art has poor processing capability for chromatic aberration and blurring problems existing in an image, and is prone to produce negative effects in the process of processing. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides a photographing method and apparatus, an electronic device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a photographing method is provided, including: acquiring each color channel image in three primary color channels of an image collected when a camera of an electronic device takes a photograph, and acquiring a blur kernel for deblurring processing for each color channel image; and performing deblurring processing on each color channel image based on the corresponding blur kernel to obtain a photographed image.

[0005] In an implementation, a blur kernel corresponding to a target channel is determined in the following manner, the target channel including each color channel in three primary color channels: a wavelength weight coefficient of a corresponding target channel is set according to a lens attribute of the camera, and a field of view is set based on a number of image blocks, the number of image blocks being a preset positive integer; and a blur kernel corresponding to the number of image blocks of the target channel is obtained based on the wavelength weight coefficient, the field of view, and a model for generating a blur kernel.

[0006] In an implementation, the obtaining of the blur kernel corresponding to the number of image blocks of the target channel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel includes: obtaining a first blur kernel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel; mirroring the first blur kernel to obtain a second blur kernel corresponding to the number of image blocks of the target channel; and determining the second blur kernel as the blur kernel corresponding to the number of image blocks of the target channel.

[0007] In an embodiment, the blur kernel used for the deblurring of each color channel image is obtained based on the current lens attribute and the current field of view of the camera.

[0008] In an embodiment, the deblurring of each color channel image based on the corresponding blur kernel comprises: segmenting each color channel image to obtain a first sub-image corresponding to the number of blur kernels of the color channel image; performing edge blur expansion on each first sub-image, and deblurring the first sub-image after edge blur expansion based on the blur kernel corresponding to the first sub-image to obtain a deblurred first sub-image; and splicing the deblurred first sub-images of each color channel image to obtain a deblurred image of each color channel image.

[0009] In an embodiment, the edge blur expansion of each first sub-image comprises: expanding m rows and n columns of pixels at the rightmost edge of the first sub-image to obtain a second sub-image, wherein m is the number of rows of pixels in the first sub-image, and n is a predetermined positive integer; and expanding n rows and m+n columns of pixels at the lowermost edge of the second sub-image to obtain a first sub-image after edge blur expansion.

[0010] In an embodiment, the expansion of m rows and n columns of pixels at the rightmost edge of the first sub-image to obtain a second sub-image comprises: expanding m rows and n columns of pixels with a value of 0 at the rightmost edge of the first sub-image; assigning a pixel value of the first column of the n columns based on the pixel values of the first column and the last column of the first sub-image; assigning a pixel value of the n-th column of the n columns based on the pixel values of the first column and the n-th column of the first sub-image; assigning a pixel value of the i-th column of the n columns based on the pixel values of the i-1-th column and the n-i+1-th column of the n columns, and assigning a pixel value of the n-i-th column of the n columns based on the pixel values of the i-th column and the n-i+1-th column, wherein i is a positive integer greater than or equal to 2 and less than or equal to n / 2; and repeating the above operations until the pixel values of all pixels in the n columns are determined to obtain the second sub-image.

[0011] In one implementation, the extending n rows and m+n columns at the lowermost edge of the second sub-image to obtain an edge-extended image includes: extending n rows and m+n columns of pixels with a value of 0 at the lowermost edge of the first sub-image; assigning a pixel value of the first row of the n rows based on pixel values of the first row and the last row of the first sub-image; assigning a pixel value of the n-th row of the n rows based on pixel values of the first row and the first row of the first sub-image; assigning a pixel value of the i-th row of the n rows based on pixel values of the i-1-th row and the n-i+1-th row of the n rows, and assigning a pixel value of the n-i-th row of the n rows based on the pixel value of the i-th row and the pixel value of the n-i+1-th row, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; and repeating the above operations until all pixel values of the n rows are determined to obtain the edge-extended image.

[0012] According to a second aspect of the embodiments of the present disclosure, a photographing device is provided, which comprises:

[0013] The acquisition unit is configured to acquire each color channel image in three primary color channels of an image collected when the camera of the electronic device takes a photograph, and acquire a blur kernel for deblurring processing for each color channel image; and the processing unit is configured to perform deblurring processing on each color channel image based on the corresponding blur kernel to obtain a photographed image.

[0014] In one implementation, the acquisition unit determines the blur kernel corresponding to a target channel in the following manner: setting a wavelength weight coefficient of the target channel according to a lens attribute of the camera, setting a field of view based on a number of image blocks, and obtaining the blur kernel corresponding to the number of image blocks for the target channel based on the wavelength weight coefficient, the field of view, and a model for generating a blur kernel.

[0015] In one implementation, the acquisition unit obtains the blur kernel corresponding to the number of image blocks for the target channel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel in the following manner: obtaining a first blur kernel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel; mirroring the first blur kernel to obtain a second blur kernel corresponding to the number of image blocks for the target channel; and determining the second blur kernel as the blur kernel corresponding to the number of image blocks for the target channel.

[0016] In one implementation, the obtaining unit obtains the blur kernel for deblurring for each color channel image in the following manner: obtaining the current lens attribute and the current field of view used by the camera; and obtaining the blur kernel matching the current lens attribute and the current field of view for each color channel image.

[0017] In one implementation, the processing unit performs deblurring for each color channel image based on the corresponding blur kernel in the following manner: performing segmentation for each color channel image to obtain a first sub-image corresponding to the number of blur kernels for the color channel image; performing blur edge extension for each first sub-image, and performing deblurring on the first sub-image after blur edge extension based on the blur kernel corresponding to the first sub-image to obtain a deblurred first sub-image; and splicing the deblurred first sub-images for each color channel image to obtain a deblurred image for each color channel image.

[0018] In one implementation, the processing unit performs blur edge extension for each first sub-image in the following manner: extending m rows and n columns of pixels at the rightmost edge of the first sub-image to obtain a second sub-image, where m is the number of rows of pixels in the first sub-image, and n is a preset positive integer; and extending n rows and m+n columns of pixels at the lowermost edge of the second sub-image to obtain the first sub-image after edge extension.

[0019] In one implementation, the processing unit extends m rows and n columns of pixels at the rightmost edge of the first sub-image to obtain a second sub-image in the following manner: extending m rows and n columns of pixels with a value of 0 at the rightmost edge of the first sub-image; assigning a pixel value of the first column in the n columns based on pixel values of the first column and the last column of the first sub-image; assigning a pixel value of the n-th column in the n columns based on a pixel value of the first column in the n columns and a pixel value of the first column in the first sub-image; assigning a pixel value of the i-th column in the n columns based on a pixel value of the i-1-th column in the n columns and a pixel value of the n-i+1-th column in the n columns, and assigning a pixel value of the n-i-th column in the n columns based on the pixel value of the i-th column and the pixel value of the n-i+1-th column, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; and repeating the above operations until pixel values of all pixels in the n columns are determined to obtain the second sub-image.

[0020] In an implementation, the n rows and m+n columns are extended at the lowermost edge of the second sub-image to obtain an edge-extended image, including: extending the pixels with a value of 0 at the lowermost edge of the first sub-image by n rows and m+n columns; assigning the pixel value of the first row of the n rows based on the pixel values of the first row and the last row of the first sub-image; assigning the pixel value of the n-th row of the n rows based on the pixel value of the first row of the n rows and the pixel value of the first row of the first sub-image; assigning the pixel value of the i-th row of the n rows based on the pixel value of the i-1-th row and the pixel value of the n-i+1-th row, and assigning the pixel value of the n-i-th row of the n rows based on the pixel value of the i-th row and the pixel value of the n-i+1-th row, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; and repeating the above operations until the pixel values of all the pixels in the n rows are determined to obtain the edge-extended image.

[0021] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, including:

[0022] a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the photographing method in the first aspect or any one of the implementation manners of the first aspect.

[0023] According to a fourth aspect of embodiments of the present disclosure, a storage medium is provided, and the storage medium stores instructions, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can perform the method in the first aspect or any one of the implementation manners of the first aspect.

[0024] The technical solution provided by the embodiments of the present disclosure can have the following beneficial effects: by using the corresponding blur kernel to perform deblurring operation on each color channel image in the image collected by the electronic device, fully considering the factor that the refractive index of the optical lens in the electronic device is different for different wavelengths contained in different channels, by using the corresponding blur kernel information of each channel image to perform deblurring processing on each channel image, and then obtaining a photographed image based on the processed channel images, the problem of blur caused by one-way chromatic aberration on each channel image and the problem that the blur effects caused by axial chromatic aberration on each channel image are different can be simultaneously eliminated or weakened, and then the color edge problem caused by the difference can be eliminated.

[0025] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are incorporated into the specification and constitute part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0027] Figure 1 is a flowchart of a photographing method according to an example embodiment.

[0028] Figure 2 is a flowchart of a blur kernel determination method according to an example embodiment.

[0029] Figure 3 is a flowchart of a blur kernel determination method according to an example embodiment.

[0030] Figure 4 is a flowchart of a blur kernel acquisition method according to an example embodiment.

[0031] Figure 5 is a flowchart of a deblurring processing method according to an example embodiment.

[0032] Figure 6 is a flowchart of a blur edge extension method according to an example embodiment.

[0033] Figure 7 is a flowchart of a right-side blur edge extension method according to an example embodiment.

[0034] Figure 8 is a flowchart of a lower-side blur edge extension method according to an example embodiment.

[0035] Figure 9 is a schematic diagram of a photographing method flow according to an example embodiment.

[0036] Figure 10 is a schematic diagram of an existing photographing method effect according to an example embodiment.

[0037] Figure 11 is a schematic diagram of a photographing method effect according to an example embodiment.

[0038] Figure 12 is a block diagram of a photographing apparatus according to an example embodiment.

[0039] Figure 13 is a block diagram of an apparatus for photographing according to an example embodiment. DETAILED DESCRIPTION

[0040] The example embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings, in which like reference numerals represent like elements or similar elements, unless otherwise indicated. The following description of example embodiments is not representative of all embodiments consistent with the present disclosure.

[0041] In the drawings, identical or similar reference signs refer to identical or similar elements or elements having identical or similar functions throughout. The embodiments described are part of the embodiments of the present disclosure, not all of the embodiments. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present disclosure. The embodiments of the present disclosure are described in detail below with reference to the drawings.

[0042] The photographing method provided by the embodiments of the present disclosure is applied to the field of image processing. The photographing method provided by the embodiments of the present disclosure is mainly used for performing deblurring processing on three primary color channel images of a collected image respectively after a photographing operation is triggered, wherein each color channel image is deblurred using a corresponding blur kernel.

[0043] With the rapid development of electronic devices, users' requirements for photographing by electronic devices are continuously increasing. In order to improve the light intake, the aperture of the lens of the electronic device is also getting larger. The large-aperture electronic device lens is limited by physical laws and production processes, and the image aberration is more serious than that of the large-aperture professional camera lens. Moreover, the number of pixels of the Complementary Metal-Oxide-Semiconductor (CMOS) of the electronic device is increasing, and the spatial sampling frequency is also increasing, and the requirement for the high-frequency Modulation Transfer Function (MTF) of the lens is also increasing. The monochromatic aberration of the optical lens will cause the blurring of the image, and the chromatic aberration will introduce color edges of other colors. The closer to the edge of the field of view, the more serious the blurring and color edges caused by aberration. This problem seriously affects the quality of color photos and reduces the user's photographing experience. Since it is difficult and costly to further improve the imaging quality of the lens, image processing algorithms are needed to improve the image quality.

[0044] In the related art, the deblurring operation and the decoloring operation performed by the electronic device on the collected image are independent of each other, and cannot effectively process the blurring caused by monochromatic aberration and the color edge problem caused by chromatic aberration. Moreover, the image deblurring algorithm in the related art is highly dependent on the accurate estimation of the blur kernel, i.e., the Point Spread Function (PSF), and the PSF has the characteristics of changing with wavelength, field of view, etc. How to efficiently obtain the blur kernel suitable for electronic device photo processing is currently a difficulty. The current de-coloring algorithm mainly performs size scaling on images of different channels, which can only process the magnification chromatic aberration, i.e., the focus plane landing positions of different colors are different, and cannot process the axial chromatic aberration, i.e., the focal planes of different colors are inconsistent, and the PSF form is inconsistent.

[0045] Therefore, the embodiment of the present disclosure provides a photographing method, each color channel image in three primary color channels of a collected image is respectively deblurred using a blur kernel corresponding to each color channel image, so as to realize deblurring and color edge removal of the collected image at the same time.

[0046] Figure 1 FIG. 1 is a flowchart of a photographing method according to an example embodiment, as shown in the figure, comprising the following steps. Figure 1

[0047] In step S11, each color channel image in three primary color channels of an image collected when a camera of an electronic device takes a photograph is obtained, and a blur kernel for deblurring each color channel image is obtained.

[0048] In the embodiment of the present disclosure, the image collected when the camera of the electronic device takes a photograph can be a preview image displayed by the electronic device, or can be an initial image obtained by the camera of the electronic device after a shutter button is pressed.

[0049] In the embodiment of the present disclosure, the three primary color channels are red, green and blue channels in RGB color mode, and the three primary color channel images obtained in the embodiment of the present disclosure are R channel images, G channel images and B channel images corresponding to the collected image.

[0050] In the embodiment of the present disclosure, the blur kernel corresponding to each color channel image is data pre-stored in the local of the electronic device, and it should be understood that the blur kernels corresponding to each color channel image can be the same or different.

[0051] In step S12, each color channel image is deblurred based on the corresponding blur kernel, and a photographing image is obtained.

[0052] In the embodiment of the present disclosure, after the R, G and B channel images of the collected image and the blur kernels corresponding to each channel image are obtained, each channel image is deblurred based on the blur kernel corresponding to each channel image, and the processed channel images are combined to obtain a final photographing image. The deblurring of each channel image based on the blur kernel corresponding to each channel image can be deconvolution of each channel image using the blur kernel corresponding to each channel image to obtain the deblurred channel image.

[0053] In the embodiment of the present disclosure, the blur kernel corresponding to each channel image can be obtained based on lens data of the electronic device using simulation software or a model.

[0054] ​In the embodiments of the present disclosure, by using the corresponding blur kernel to deblur each color channel image collected by the electronic device, it is fully considered that the refractive index of the optical lens in the electronic device for different channels containing wavelengths is different, and therefore the deblurring processing is performed on each channel image using the corresponding blur kernel information of each channel image, so as to eliminate or weaken the blur problem caused by the single color difference of each channel image and the different blur effects caused by the axial color difference of each channel image, and further to cause the color edge problem.

[0055] In the embodiments of the present disclosure, the blur kernel can be an accurate estimation of a point spread function (PSF). The deblurring processing based on the corresponding blur kernel can be a processing operation including a deconvolution operation using the blur kernel.

[0056] In the embodiments of the present disclosure, the imaging process of the camera of the electronic device on an object (Object) can be described as the convolution of the object and the PSF of the lens plus the noise of the CMOS sensor. Since the sensor of the electronic device records the image in the format of RGB three channels, the imaging process can be described as

[0057] Image R (x,y)=Object*PSF R (x,y)+noise

[0058] Image G (x,y)=Object*PSF G (x,y)+noise

[0059] Image B (x,y)=Object*PSF B (x,y)+noise

[0060] In the above formula, "*" represents the convolution operation, the subscripts R, G, and B represent the RGB three channels respectively, (x, y) represents the coordinates of the pixel points of the image, and also corresponds to the field position of the lens, the (0, 0) point is the center of the field of view, and the center point of the image. In the formula, for a single channel image, the blur caused by the lens aberration is represented by PSF, and PSF changes with the field position. For a color photo, the PSF of different channels at the same field position is different, which causes the color edge in the image. A general rule is that the closer (x, y) is to the edge, the greater the blur degree of PSF, and the greater the difference between the PSFs of different channels.

[0061] Figure 2 is a flow chart of a blur kernel determination method according to an example embodiment, as shown in Figure 2As shown, comprising the following steps.

[0062] In step S21, the wavelength weight coefficient of the corresponding target channel is set according to the lens attribute of the camera, and the field of view is set based on the number of picture blocks.

[0063] In the embodiments of the present disclosure, the target channel includes each color channel in the three primary color channels. The lens attribute of the camera has an important influence on the formation of the blur kernel data. The lens attribute can include factors such as material, design, manufacturing precision, and optical characteristics. Different lens attributes will affect the refraction and focusing effect of light passing through the lens, and thus affect the blur kernel data. Therefore, by modeling and analyzing the response of the lens to different wavelengths of light, the contribution weight of each wavelength in the formation process of the blur kernel corresponding to each target channel can be obtained, and the wavelength weight coefficient corresponding to each target channel can be determined.

[0064] In the embodiments of the present disclosure, the number of picture blocks is a preset positive integer. It should be understood that the number of picture blocks can include the number of horizontal blocks e and the number of vertical blocks f, for example, the number of picture blocks is set to e x f, that is, the picture is divided into e blocks in the horizontal direction and f blocks in the vertical direction. It should be understood that the above setting of the number of picture blocks and the manner of determining the number of blocks are only used for illustrative purposes, and the number of picture blocks is not limited in the embodiments of the present disclosure.

[0065] In the embodiments of the present disclosure, the size of the field of view corresponding to each blur kernel can be set based on the number of picture blocks. For example, for the same lens attribute, the size of the field of view corresponding to the blur kernel when the picture block is e x f is different from the size of the field of view corresponding to the blur kernel when the picture block is (e+10) x (f+10). It should be understood that the above setting of the number of picture blocks and the manner of determining the number of blocks are only used for illustrative purposes, and the number of picture blocks is not limited in the embodiments of the present disclosure.

[0066] In step S22, based on the wavelength weight coefficient, the field of view, and the model for generating the blur kernel, the blur kernel corresponding to the number of picture blocks of the target channel is obtained.

[0067] In the embodiments of the present disclosure, the model for generating the blur kernel can be a pre-determined simulation software or a trained model. The wavelength weight coefficient corresponding to the target channel and the field of view are input into the model for generating the blur kernel, and the blur kernel data output by the model is obtained, so as to obtain the blur kernel corresponding to the target channel and improve the accuracy of the obtained blur kernel.

[0068] In the embodiments of the present disclosure, since the blur kernel is a PSF, the PSF is continuously and nonlinearly changed with the field of view, but within a certain range, the PSF in the region can be considered to be basically consistent, and therefore the PSF is output according to a certain field of view interval.

[0069] In the embodiments of the present disclosure, since the optical lens is rotationally symmetric at the center and the CMOS sensor is rectangular, the blur kernel data of part of the blocks can be obtained, and the blur kernel data of all blocks can be obtained by using the mirror symmetry, so as to reduce the calculation amount of the blur kernel determination and improve the calculation efficiency of the blur kernel determination.

[0070] Figure 3 A flowchart of a blur kernel determination method according to an example embodiment is shown in FIG. 1. Figure 3 As shown in FIG. 1, the method comprises the following steps.

[0071] In step S31, a first blur kernel is obtained based on a wavelength weight coefficient, a field of view, and a model used to generate the blur kernel.

[0072] In the embodiments of the present disclosure, the first blur kernel is a continuous and nonlinear block corresponding blur kernel information. All picture blocks can be divided to establish quadrants in the left-right direction and the up-down direction with the picture block at the center region as the origin, to obtain four quadrant regions, a horizontal coordinate axis region, and a vertical coordinate axis region.

[0073] In the embodiments of the present disclosure, the first blur kernel can include the blur kernel corresponding to one of the four quadrant regions, the horizontal coordinate axis region, and the vertical coordinate axis region.

[0074] In step S32, the first blur kernel is mirrored to obtain a second blur kernel corresponding to the number of picture blocks of the target channel.

[0075] In the embodiments of the present disclosure, since the blur kernel data at the horizontal and vertical coordinate axes is not mirror symmetric, the blur kernel of one of the four quadrant regions in the first blur kernel is subjected to left-right mirroring and up-down mirroring to obtain the blur kernel data corresponding to the remaining three quadrants, the obtained blur kernel data corresponding to the three quadrants is combined with the first blur kernel data, and the combined blur kernel is set as the second blur kernel.

[0076] In step S33, the second blur kernel is determined as the blur kernel corresponding to the number of picture blocks of the target channel.

[0077] In the embodiments of the present disclosure, the picture is divided into four quadrants, the blur kernel of the entire field of view can be obtained by calculating one of the quadrants and using the quadrant to perform mirror symmetry, which can significantly reduce the workload. However, due to the rotational symmetry at the center, the blur kernel at the coordinate axis should be output separately.

[0078] In an example embodiment, the model used to generate the blur kernel can be optical design software. In the optical design software, open the design file of a specified lens, set the wavelength weight coefficient according to the lens, the transmittance of the infrared filter, and the spectral response of the CMOS sensor, set the (x, y) field of view at a certain interval according to the number of image blocks, then set the sampling interval of the PSF intensity distribution according to the Nyquist criterion, determine the number of sampling pixels of a single PSF image when the ratio of the value at the edge of the single PSF image to the value at the center is greater than the dynamic range of the CMOS, obtain 1 channel, p*q high-sampling PSF intensity distribution data, then perform four-quadrant symmetry to obtain (2p-1)*(2q-1) PSF data, the PSF at (0, y) and (x, 0) is not symmetric to the left and right and up and down, respectively, and the field of view distribution number is odd. Set the wavelength weight coefficient respectively, repeat 3 times, and obtain the results of 3 channels. Finally, according to the pixel size of the CMOS, down-sample and normalize the PSF, and if the default output is four-in-one, it is necessary to down-sample to the size of 2 times the pixel size, that is, the physical size of one pixel of the down-sampled PSF data is equal to the physical size of one pixel of the final image.

[0079] In the embodiments of the present disclosure, the blur kernel data of the target channel corresponding to the image block number is determined first, and then the blur kernel information of the target channel corresponding to the image block number is obtained through mirroring processing, thereby reducing the calculation cost required for obtaining the blur kernel information of the target channel corresponding to the image block number, and the blur kernel information obtained through mirroring is more accurate compared with the traditional scheme for determining the blur kernel information of the target channel corresponding to the image block number.

[0080] In the embodiments of the present disclosure, the blur kernel information obtained by the above method can be saved locally in the electronic device, so that when the electronic device takes a picture with a specific lens attribute and a field of view, the blur kernel corresponding to the specific lens attribute and the field of view can be retrieved and deblurred locally.

[0081] Figure 4 is a flow chart of a blur kernel acquisition method according to an example embodiment, as shown in Figure 4 , comprising the following steps.

[0082] In step S41, the current lens attribute used by the camera and the current field of view are obtained.

[0083] In the embodiments of the present disclosure, if the image captured when the camera of the electronic device takes a picture is the preview image displayed by the electronic device, the current lens attribute used by the camera of the electronic device and the current field of view are obtained; if the image captured when the camera of the electronic device takes a picture is the initial image obtained after the camera of the electronic device presses the shooting button, the lens attribute and the field of view when the initial image is taken by the electronic device are obtained.

[0084] In step S42, a blur kernel matching the current lens attribute and the current field of view is obtained for each color channel image.

[0085] In the embodiments of the present disclosure, the electronic device determines the blur kernel corresponding to the R, G and B color channel images corresponding to the lens attribute and the field of view in the local based on the obtained lens attribute and field of view. The electronic device realizes the acquisition of each color channel image corresponding to the lens attribute and the field of view used when photographing.

[0086] In the embodiments of the present disclosure, the electronic device searches the blur kernel pre-stored in the local through the lens attribute and the field of view when photographing, and realizes the acquisition of the blur kernel of each color channel corresponding to the current lens attribute and the field of view.

[0087] In the embodiments of the present disclosure, when the blur kernel is used to deblur the image, the deblurring processing inevitably performs Fast Fourier Transform (FFT) on the image in the deconvolution process. In the calculation process of FTT, the image data is tiled and extended, the left edge of the image is connected to the right edge, causing a fault, and introducing additional spatial high frequency. After deconvolution, the additional high frequency is enhanced, so that the image appears additional stripes. This edge anomaly is called ringing effect. By blurring the edge of the image, the occurrence of edge anomaly can be reduced, but too many blurs will increase the amount of calculation.

[0088] Figure 5 is a flowchart of a deblurring processing method according to an example embodiment, as shown in Figure 5 includes the following steps.

[0089] In step S51, each color channel image is segmented to obtain a first sub-image corresponding to the number of blur kernels of the channel image.

[0090] In the embodiments of the present disclosure, by segmenting each color channel image, the same number of first sub-images as the blur kernel of the channel image is obtained, wherein the sub-image obtained after segmentation corresponds to the determined blur kernel of the channel image. For example, the number of blur kernels is e x f, and the number of first sub-images obtained after segmentation of each channel image is also e x f. The first sub-image in the a-th row and the b-th column corresponds to the blur kernel in the a-th row and the b-th column. It should be understood that the above setting of the number of picture blocks, the manner of determining the number of blocks, and the row and column information of the first sub-image are only used for example illustration, and the number of picture blocks is not limited in the embodiments of the present disclosure.

[0091] In step S52, the blurred edge is expanded for each first sub-image, and the first sub-image after the blurred edge expansion is deblurred based on the blur kernel corresponding to the first sub-image, to obtain a first sub-image after deblurring.

[0092] In the embodiments of the present disclosure, the blurred edge expansion is an image processing technology, which adds additional and relevant information to the edge of the image, so as to supplement the edge information of each first sub-image and prevent the occurrence of ringing phenomenon.

[0093] In the embodiments of the present disclosure, the first sub-image after the blurred edge expansion is deblurred using the blur kernel corresponding to the first sub-image, to obtain a first sub-image after deblurring.

[0094] In step S53, for each channel image in each color channel image, the first sub-image after deblurring is spliced respectively, to obtain an image after deblurring of each color channel image.

[0095] In the embodiments of the present disclosure, the first sub-image after deblurring is placed back to the position before segmentation, and the edges of adjacent self sub-images are spliced, to obtain each color channel image after deblurring.

[0096] It should be understood that the blurred edge expansion increases the number of pixels contained in the first sub-image, and the deblurring processing does not change the number of pixels of the first sub-image, so that the content of the first sub-image after deblurring can be extracted, for example, each first sub-image contains k×k pixels, and each first sub-image after the edge blurred expansion and deblurring contains l×l pixels, wherein l is a positive integer greater than k. The first sub-image after deblurring can be cut or extracted to obtain a first sub-image after deblurring with k×k pixels, to avoid the problem of increasing the image size of each color channel image after the first sub-image is subjected to blurred edge expansion and deblurring. It should be understood that the number of pixels contained in the first sub-image and the number of pixels contained in the first sub-image after the edge blurred expansion are only used for exemplary illustration, and the number of pixels contained in the first sub-image and the number of pixels contained in the first sub-image after the edge blurred expansion are not limited in the embodiments of the present disclosure.

[0097] In the embodiments of the present disclosure, the right lower side of the first sub-image can be expanded to reduce the amount of calculation required for edge blurred expansion.

[0098] Figure 6 A flowchart of a blurred edge expansion method according to an exemplary embodiment is shown in FIG. 1, which includes the following steps. Figure 6 As shown in FIG. 1, the method includes the following steps.

[0099] In step S61, m rows and n columns of pixels are expanded at the rightmost edge of the first sub-image, to obtain a second sub-image.

[0100] In the embodiments of the present disclosure, m is the number of pixel rows in the first sub-image, and n is a preset positive integer. If the first sub-image contains m rows and m columns of pixels, the second sub-image contains m rows and m+n columns of pixels, wherein the left m columns in the second sub-image are composed of the pixels of the first sub-image, and the right n columns in the second sub-image are composed of the pixels of the edge extension.

[0101] In the embodiments of the present disclosure, the method of performing edge blur extension on the right side of the first sub-image can be implemented in the following manner:

[0102] In step S62, m rows and n columns of pixels are extended on the lowermost edge of the second sub-image to obtain the first image after edge extension.

[0103] In the embodiments of the present disclosure, the second sub-image contains m rows and m+n columns of pixels, and thus after m rows and n columns of pixels are extended on the lowermost edge of the second sub-image, the first image after edge extension contains m+n rows and m+n columns of pixels. Among them, the upper m rows of pixels in the first image after edge extension are composed of the m rows of pixels of the second sub-image, and the lower n rows of pixels in the first image after edge extension are composed of the pixels of the edge extension.

[0104] In the embodiments of the present disclosure, by performing edge blur extension on the lower right side of each first sub-image, the ringing effect generated when performing deconvolution processing on the first sub-image is effectively avoided, and compared with the traditional scheme of performing edge blur extension on all four sides of the image, the embodiments of the present disclosure effectively reduce the calculation amount of edge blur extension and reduce the requirement for the processing capacity of the electronic device.

[0105] Figure 7 is a flowchart of a right-side edge blur extension method according to an exemplary embodiment, as shown in Figure 7 , which includes the following steps.

[0106] In step S71, m rows and n columns of pixels with a value of 0 are extended on the rightmost edge of the first sub-image.

[0107] In the embodiments of the present disclosure, m rows and n columns of pixels with a value of 0 are extended on the rightmost edge of the first sub-image, which allows the algorithm or processing flow to dynamically determine the value of the pixel according to the local characteristics or global requirements of the image in the subsequent steps, thereby increasing the flexibility and controllability of the edge blur extension.

[0108] In step S72, based on the pixel values of the first column and the last column of the first sub-image, the pixel value of the first column in the n columns is assigned.

[0109] In the embodiments of the present disclosure, the pixel value of the to-be-determined column pixel can be determined by using the Gaussian blur kernel generated based on the preset parameter and the adjacent non-zero column pixel of the to-be-determined column pixel through a convolution operation. For the first column counted from left to right in the n columns to be expanded, the left adjacent non-zero column of the first column is the last column of the first sub-image.

[0110] In the embodiments of the present disclosure, due to the continuation characteristic of the fast Fourier transform, when processing a discrete image signal of a limited length, the FFT processes it as a periodically continued signal, so that the spectral characteristics of the image signal can be more comprehensively analyzed. Therefore, the right adjacent non-zero column of the pixel value of the first column in the n columns to be expanded is the first column of the first sub-image. The pixel value of the first column and the pixel value of the last column of the first sub-image are assigned to the pixel value of the first column in the n columns, so that the pixel value of the first column of the determined edge expansion is related to the content of the first sub-image.

[0111] In step S73, the pixel value of the n-th column in the n columns is assigned based on the pixel value of the first column in the n columns and the pixel value of the first column in the first sub-image.

[0112] In the embodiments of the present disclosure, the pixel value of the n-th column in the n columns to be expanded is determined based on the adjacent non-zero column and the Gaussian blur kernel generated based on the preset parameter, that is, the pixel value of the n-th column in the n columns to be expanded needs to be determined based on the pixel value of the first column in the n columns determined in the previous step and the pixel value of the first column in the first sub-image.

[0113] In step S74, the pixel value of the i-th column in the n columns is assigned based on the pixel value of the i-1-th column in the n columns and the pixel value of the n-i+1-th column, and the pixel value of the n-i-th column in the n columns is assigned based on the pixel value of the i-th column and the pixel value of the n-i+1-th column.

[0114] In the embodiments of the present disclosure, i is a positive integer greater than or equal to 2 and less than or equal to n / 2. After the pixel value of the n-th column is determined, the pixel values of the 2nd, n-1th, 3rd, n-2th, … columns are determined. The pixel value of each column is determined based on the pixel value of the adjacent non-zero column.

[0115] In step S75, the above operation is repeated until the determination of all pixel values in the n columns is completed, and the second sub-image is obtained.

[0116] In the embodiments of the present disclosure, the pixel value of the n columns to be expanded is determined in a left-right alternating manner, and the second sub-image is finally determined, which can ensure that the continuity of the edge is maintained during the expansion process, and help to balance the processing effect, avoid uneven phenomenon of one side during the expansion process, and avoid error accumulation problem caused by single direction processing.

[0117] In the embodiments of the present disclosure, the lower fuzzy edge expansion method can be based on the same principle as the right fuzzy edge expansion method to expand the second image.

[0118] Figure 8 FIG. 6 is a flowchart of a lower fuzzy edge expansion method according to an exemplary embodiment, as shown in the figure, including the following steps. Figure 8

[0119] In step S81, n rows of m+n columns of pixels with a value of 0 are expanded at the lowermost edge of the first sub-image.

[0120] In the embodiments of the present disclosure, the n rows of m+n columns of pixels with a value of 0 are expanded at the rightmost edge of the first sub-image, allowing the algorithm or processing flow to dynamically determine the value of the pixel according to the local characteristics or global requirements of the image in the subsequent steps, increasing the flexibility and controllability of the fuzzy edge expansion.

[0121] In step S82, the pixel value of the first row of the n rows is assigned based on the pixel values of the first row and the last row of the first sub-image.

[0122] In the embodiments of the present disclosure, the pixel value of the first row of the n rows is determined by convolution operation using the Gaussian blur kernel generated based on the preset parameters and the adjacent non-zero row pixels of the to-be-determined row pixels. For the first row of the n rows to be expanded, the left adjacent non-zero row is the last row of the first sub-image. The right adjacent non-zero row of the pixel value of the first row of the n rows to be expanded is the first row of the first sub-image. The pixel value of the first row of the n rows is assigned based on the pixel values of the first row and the last row of the first sub-image, so that the determined pixel value of the first row of the edge expansion is related to the content of the first sub-image.

[0123] In step S83, the pixel value of the n-th row of the n rows is assigned based on the pixel value of the first row of the n rows and the pixel value of the first row of the first sub-image.

[0124] In the embodiments of the present disclosure, the pixel value of the n-th row of the n rows to be expanded is determined based on the adjacent non-zero row and the Gaussian blur kernel generated based on the preset parameters, i.e. the pixel value of the n-th row of the n rows to be expanded is determined based on the pixel value of the first row of the n rows determined in the previous step and the pixel value of the first row of the first sub-image.

[0125] In step S84, the pixel value of the i-th row of the n rows is assigned based on the pixel value of the i-1-th row of the n rows and the pixel value of the n-i+1-th row of the n rows, and the pixel value of the n-i-th row of the n rows is assigned based on the pixel value of the i-th row and the pixel value of the n-i+1-th row, i is a positive integer greater than or equal to 2 and less than or equal to n / 2.

[0126] ​In the embodiments of the present disclosure, i is a positive integer greater than or equal to 2 and less than or equal to n / 2. After determining the pixel value of the nth row, the pixel values of the 2, n-1, 3, n-2, … rows are determined in sequence. The pixel value of each row is determined based on the pixel value of the adjacent non-zero row.

[0127] In step S85, the above operation is repeated until the determination of all pixel values in the n rows is completed, and the edge expanded image is obtained.

[0128] In the embodiments of the present disclosure, the left and right alternating confirmation of the pixel value is used to determine the pixel value of the n rows that need to be expanded, and finally determine the second sub-image, which can ensure the continuity of the edge during the expansion process, and help to balance the processing effect, avoid the uneven phenomenon of one-sided deviation in the expansion process, and avoid the error accumulation problem caused by single direction processing.

[0129] In an exemplary embodiment, according to the imaging principle and the division of PSF data, the image processing needs to process the picture by channel and region, that is, the RGB three channels are processed respectively, and then the picture of a single channel is divided into (2c-1)*(2d-1) or more times to be substituted into the algorithm, as shown in the following formula: Figure 9 Figure 9 is a schematic diagram of a photographing method flow according to an exemplary embodiment. Since the calculation of each region and each channel is independent of each other, the processing process can be parallel processed by a central processing unit (CPU) + a graphics processing unit (GPU) + a chip capable of realizing digital signal processing technology (Digital Signal Processing, DSP), thereby improving the running speed.

[0130] In Figure 9 ​In the embodiment, the electronic device reads the R, G and B color channel data of the photographed picture and reads the PSF data corresponding to each channel, divides each single channel image obtained into (2c-1)*(2d-1) small blocks, and performs blurred edge expansion on each small block. It should be understood that, in order to avoid the ringing effect of the deconvolution algorithm, a method of expanding the blurred edge picture with small calculation amount can be used. For example, taking the expansion of a 25*25 picture to 32*32 as an example, due to the continuation characteristic of FFT, only the lower right needs to be expanded. In this case, the right edge expansion is implemented as follows: first, 25*7 pixels of the right edge of the picture are expanded with a value of 0, and then a convolution operation is performed on the right edge of the original picture, i.e., the 25th column, with the Gaussian blur kernel as the weight to obtain 25*1 pixels of the value as the 1st column of the expanded edge, i.e., the 25th column of the expanded image. A convolution operation is performed on the 1st column of the picture to obtain 25*1 pixels of the value as the 7th column of the expanded edge, i.e., the 32nd column of the expanded image. The 2nd column of the expanded edge is obtained by convolution of the Gaussian blur kernel and the 1st column of the expanded edge, the 6th column of the expanded edge is obtained by convolution of the Gaussian blur kernel and the 7th column of the expanded edge, and so on. The values are filled into the expanded edge to obtain the blurred edge. This method has fast calculation speed and good effect.

[0131] After the edge blur expansion is implemented, in order to avoid that the synthesized channel images are too large, a preset algorithm can be used to solve each small block, and then the upper left or the center effective part is taken out. Then the image blocks are merged into a complete image to obtain a single color channel image after the deblurring processing. The above operation is repeated until the images of the three color channels R, G and B are completed, and the images of the three color channels are synthesized to obtain the final photographed image.

[0132] In the embodiment of the present disclosure, the deconvolution algorithm used is to solve the following objective function by using the alternating multiplier method, and the clear image is obtained by the optimization method:

[0133]

[0134] wherein x is the clear image to be solved, D is the difference operator, K is the PSF blur operator, f is the original image collected, and μ is a preset weight coefficient. The photograph of the electronic device is divided into R, G and B channels and regions with the corresponding PSF data as the prior data, and the edge blur expansion is performed. The deconvolution algorithm processing can realize the deblurring and de-color edge at one time, improve the image clarity of the color photograph, and does not have negative effects such as ringing artifacts.

[0135] In an example embodiment, the electronic device is taken as an example of a terminal. The terminal uses a long-focus lens to perform conventional image acquisition, as shown in FIG. Figure 10 Figure 10 ​is a schematic diagram of the effect of the existing photographing method according to an example embodiment. In Figure 10 It can be seen that there is blur and color fringe at the edge of the black and white image in the image, which seriously affects the imaging quality of the terminal and reduces the user experience of using the terminal to take a photo.

[0136] In an example embodiment, the same terminal as in Figure 10 is used to take an image using the same long-focus lens at the same position and using the same lens parameters as the terminal in Figure 10 Figure 11 is shown. Figure 11 is a schematic diagram of the effect of the photographing method according to an example embodiment.

[0137] In Figure 11 , the photographing image obtained by the method in the embodiment of the present disclosure is improved in blur and color fringe at the edge of the black and white image in the image compared to Figure 10 , improving the clarity of the image obtained by the electronic device and improving the user experience of using the electronic device to take a photo.

[0138] In the embodiment of the present disclosure, the clarity of the photographed photo is improved by improving each color channel image in the three primary color channels of the collected image, especially in places with large color light and dark contrast, and a certain color fringe is corrected, making the photo clearer and sharper. In addition, compared to the sharpening algorithm in the image signal processor (Image Signal Processor, ISP) of the electronic device, since the blur kernel PSF is accurately estimated by the present application, the edge reversal defect will not occur after the algorithm processing, avoiding the over-modification of the algorithm, making the photo more natural and real.

[0139] Based on the same concept, the embodiment of the present disclosure also provides a photographing device.

[0140] It can be understood that the photographing device provided by the embodiment of the present disclosure contains the corresponding hardware structure and / or software module for executing each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solution of the present disclosure.

[0141] Figure 12 is a block diagram of a photographing device 100 according to an example embodiment. Refer to Figure 12 ​The device comprises an acquisition unit 101 and a processing unit 102.

[0142] The acquisition unit 101 is configured to acquire each color channel image in three primary color channels of an image collected when a camera of an electronic device takes a picture, and acquire a blur kernel for deblurring processing for each color channel image.

[0143] The processing unit 102 is configured to perform deblurring processing on each color channel image based on the corresponding blur kernel, to obtain a picture image.

[0144] In an embodiment, the acquisition unit 101 determines the blur kernel corresponding to a target channel in the following manner: setting a wavelength weight coefficient of the corresponding target channel according to a lens attribute of the camera, setting a field of view based on a number of image blocks, the number of image blocks being a preset positive integer; and obtaining the blur kernel corresponding to the number of image blocks of the target channel based on the wavelength weight coefficient, the field of view, and a model for generating a blur kernel.

[0145] In an embodiment, the acquisition unit 101 obtains the blur kernel corresponding to the number of image blocks of the target channel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel in the following manner: obtaining a first blur kernel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel; mirroring the first blur kernel to obtain a second blur kernel corresponding to the number of image blocks of the target channel; and determining the second blur kernel as the blur kernel corresponding to the number of image blocks of the target channel.

[0146] In an embodiment, the acquisition unit 101 acquires the blur kernel for deblurring processing for each color channel image in the following manner: acquiring a current lens attribute and a current field of view used by the camera; and acquiring a blur kernel matching the current lens attribute and the current field of view for each color channel image.

[0147] In an embodiment, the processing unit 102 performs deblurring processing on each color channel image based on the corresponding blur kernel in the following manner: performing segmentation on each channel image in the color channel images to obtain a first sub-image corresponding to a number of blur kernels of the channel image; performing blur edge expansion on each first sub-image, and performing deblurring processing on the first sub-image after blur edge expansion based on the blur kernel corresponding to the first sub-image, to obtain a deblurring processed first sub-image; and splicing the deblurring processed first sub-images for each channel image in the color channel images, to obtain a deblurring processed image of each color channel image.

[0148] In an embodiment, the processing unit 102 performs edge extension on each first sub-image in the following manner: m rows and n columns of pixels are extended on the rightmost edge of the first sub-image to obtain a second sub-image, where m is the number of rows of pixels in the first sub-image, and n is a predetermined positive integer; and m+n columns of pixels are extended on the lowermost edge of the second sub-image to obtain the first sub-image after edge extension.

[0149] In an embodiment, the processing unit 102 performs edge extension on the rightmost edge of the first sub-image in the following manner to obtain a second sub-image: m rows and n columns of pixels with value 0 are extended on the rightmost edge of the first sub-image; the value of the pixel in the first column of the n columns is determined based on the values of the pixels in the first column and the last column of the first sub-image; the value of the pixel in the n-th column of the n columns is determined based on the values of the pixels in the first column and the n-th column of the first sub-image; the value of the pixel in the i-th column of the n columns is determined based on the values of the pixels in the i-1-th column and the n-i+1-th column of the n columns, and the value of the pixel in the n-i-th column of the n columns is determined based on the values of the pixels in the i-th column and the n-i+1-th column, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; and the above operations are repeated until the values of all pixels in the n columns are determined to obtain the second sub-image.

[0150] In an embodiment, m+n columns of pixels are extended on the lowermost edge of the second sub-image to obtain the image after edge extension, including: m+n columns of pixels with value 0 are extended on the lowermost edge of the first sub-image; the value of the pixel in the first row of the n rows is determined based on the values of the pixels in the first row and the last row of the first sub-image; the value of the pixel in the n-th row of the n rows is determined based on the values of the pixels in the first row and the n-th row of the first sub-image; the value of the pixel in the i-th row of the n rows is determined based on the values of the pixels in the i-1-th row and the n-i+1-th row of the n rows, and the value of the pixel in the n-i-th row of the n rows is determined based on the values of the pixels in the i-th row and the n-i+1-th row, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; and the above operations are repeated until the values of all pixels in the n rows are determined to obtain the image after edge extension.

[0151] As to the apparatus in the above embodiments, the specific manners in which the respective modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0152] Figure 13is a block diagram of a device 200 for photographing according to an exemplary embodiment. The device 200 can be, for example, a mobile phone, a computer, a digital broadcasting terminal, a message communicator, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0153] Referring to Figure 13 The device 200 can include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, an input / output (I / O) interface 212, a sensor component 214, and a communication component 216.

[0154] The processing component 202 generally controls the overall operations of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 can include one or more processors 220 to execute instructions to complete all or part of steps of the above-described methods. In addition, the processing component 202 can include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 can include a multimedia module to facilitate the interaction between the multimedia component 208 and the processing component 202.

[0155] The memory 204 is configured to store various types of data to support operations of the device 200. Examples of these data include instructions for any application or method operating on the device 200, contact data, phonebook data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.

[0156] The power component 206 provides power to the various components of the device 200. The power component 206 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 200.

[0157] The multimedia component 208 includes a screen providing an output interface between the device 200 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0158] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC) to receive an external audio signal when the device 200 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 also includes a speaker to output audio signals.

[0159] The I / O interface 212 provides an interface between the processing component 202 and peripheral interface modules, such as a keypad, a click wheel, buttons, and so on. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0160] The sensor component 214 includes one or more sensors to provide various state assessments for the device 200. For example, the sensor component 214 can detect an open / closed position of the device 200, relative positioning of components, such as a display and a keypad of the device 200, a change in position of the device 200 or a component of the device 200, presence or absence of user contact with the device 200, a change in orientation of the device 200 or acceleration / deceleration of the device 200, and temperature changes of the device 200. The sensor component 214 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 214 can also include a light sensor, such as a CMOS or CCD image sensor, for use in an imaging application. In some embodiments, the sensor component 214 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0161] The communication component 216 is configured to facilitate wired or wireless communication between the apparatus 200 and other devices. The apparatus 200 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0162] In an exemplary embodiment, the apparatus 200 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components, for performing the above-described methods.

[0163] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 204 including instructions, is also provided, which can be executed by the processor 220 of the apparatus 200 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0164] It can be understood that, in the present disclosure, "multiple" refers to two or more, and other quantifiers are similar thereto. The association relationship of "and / or" describing the associated objects means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. The singular form "a", "said" and "the" are also intended to include the plural form, unless the context clearly indicates otherwise.

[0165] It can be further understood that the terms "first", "second", and the like are used to describe various information, but the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or importance. In fact, the expressions "first", "second", and the like can be completely interchangeable. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information without departing from the scope of the present disclosure.

[0166] It will be further understood that "connected" can include direct connection between two members or indirect connection between two members through other members.

[0167] It will be further understood that, unless otherwise specified, "connected" includes direct connection or indirect connection through others.

[0168] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure including those variations that can be incorporated into other forms, methods and apparatuses of the present technology without departing from the spirit or essential characteristics thereof. It is intended that the scope of the present disclosure should include all changes coming within the meaning and equivalency of the claims.

[0169] It is to be understood that the present disclosure is not limited to the precise construction described and as shown in the accompanying drawings, and that changes can be made in various details without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims appended hereto.

Claims

1. A photographing method, characterized by, The method comprises the following steps: obtaining each color channel image in three primary color channels of an image collected by a camera of an electronic device when the camera takes a picture, and obtaining a blur kernel for deblurring processing for each color channel image; performing deblurring processing on each color channel image based on the corresponding blur kernel to obtain a picture taken by the camera.

2. The photographing method of claim 1, wherein, The blur kernel corresponding to the target channel is determined in the following manner, and the target channel comprises each color channel in three primary color channels: setting a wavelength weight coefficient of the corresponding target channel according to the lens attribute of the camera, and setting a field of view based on the number of picture blocks, which is a preset positive integer; obtaining the blur kernel corresponding to the target channel and the number of picture blocks based on the wavelength weight coefficient, the field of view, and a model for generating a blur kernel.

3. The photographing method of claim 2, wherein, The method for obtaining the blur kernel corresponding to the target channel and the number of picture blocks based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel comprises: obtaining a first blur kernel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel; mirroring the first blur kernel to obtain a second blur kernel corresponding to the target channel and the number of picture blocks; determining the second blur kernel as the blur kernel corresponding to the target channel and the number of picture blocks.

4. The photographing method of claim 2 or 3, characterized in that, The method for obtaining the blur kernel for deblurring processing for each color channel image comprises: obtaining the current lens attribute and the current field of view used by the camera; obtaining a blur kernel matching the current lens attribute and the current field of view for each color channel image.

5. The photographing method of claim 2 or 3, wherein, The method for performing deblurring processing on each color channel image based on the corresponding blur kernel comprises: segmenting each channel image in the color channel images to obtain a first sub-image corresponding to the number of blur kernels of the channel image; performing blur edge expansion on each first sub-image, and performing deblurring processing on the first sub-image after blur edge expansion based on the blur kernel corresponding to the first sub-image to obtain a first sub-image after deblurring processing; splicing the first sub-image after deblurring processing for each channel image in the color channel images to obtain an image after deblurring processing of each color channel image.

6. The photographing method of claim 5, wherein, The method for performing blur edge expansion on each first sub-image comprises: extending m rows and n columns of pixels on the rightmost edge of the first sub-image to obtain a second sub-image, wherein m is the number of rows of pixels in the first sub-image, and n is a preset positive integer; extending n rows and m+n columns of pixels on the lowermost edge of the second sub-image to obtain the first sub-image after edge expansion.

7. The photographing method of claim 6, wherein, The method for extending m rows and n columns of pixels on the rightmost edge of the first sub-image to obtain a second sub-image comprises: extending pixels with a value of 0 on the rightmost edge of the first sub-image; assigning the pixel value of the first column in the n columns based on the pixel values of the first column and the last column of the first sub-image; assigning the pixel value of the n-th column in the n columns based on the pixel value of the first column in the n columns and the pixel value of the first column in the first sub-image; assigning a pixel value of an i-th column in the n columns based on pixel values of an (i-1)-th column and an (n-i+1)-th column in the n columns, and assigning a pixel value of an (n-i)-th column in the n columns based on the pixel value of the i-th column and the pixel value of the (n-i+1)-th column, wherein i is a positive integer greater than or equal to 2 and less than or equal to n / 2; repeating the above operations until all pixel values in the n columns are determined, to obtain the second sub-image.

8. The photographing method of claim 6, wherein, The second sub-image is extended by n rows and m+n columns at a lowermost edge of the second sub-image to obtain an edge-extended image, including: extending n rows and m+n columns of pixels with a value of 0 at a lowermost edge of the first sub-image; assigning a pixel value of a first row in the n rows based on pixel values of a first row and a last row of the first sub-image; assigning a pixel value of an n-th row in the n rows based on pixel values of a first row and an n-th row in the first sub-image; assigning a pixel value of an i-th row in the n rows based on pixel values of an (i-1)-th row and an (n-i+1)-th row in the n rows, and assigning a pixel value of an (n-i)-th row in the n rows based on the pixel value of the i-th row and the pixel value of the (n-i+1)-th row, wherein i is a positive integer greater than or equal to 2 and less than or equal to n / 2; repeating the above operations until all pixel values in the n rows are determined, to obtain the edge-extended image.

9. A photographing apparatus, characterized by comprising: including: an acquisition unit, configured to acquire each color channel image in three primary color channels of a collected image when a camera of an electronic device takes a picture, and acquire a blur kernel for deblurring processing for each color channel image; a processing unit, configured to perform deblurring processing on each color channel image based on a corresponding blur kernel, to obtain a photographed image.

10. The apparatus of claim 9, wherein, The acquisition unit determines a blur kernel corresponding to a target channel in the following manner, wherein the target channel includes each color channel in the three primary color channels: setting a wavelength weight coefficient of the corresponding target channel according to a lens attribute of the camera, and setting a field of view based on a number of image blocks, wherein the number of image blocks is a preset positive integer; obtaining a blur kernel corresponding to the number of image blocks of the target channel based on the wavelength weight coefficient, the field of view, and a model for generating a blur kernel.

11. The apparatus of claim 10, wherein, The acquisition unit obtains a blur kernel corresponding to the number of image blocks of the target channel based on the wavelength weight coefficient, the field of view, and a model for generating a blur kernel in the following manner: obtaining a first blur kernel based on the wavelength weight coefficient, the field of view, and the model for generating a blur kernel; mirroring the first blur kernel to obtain a second blur kernel corresponding to the number of image blocks of the target channel; determining the second blur kernel as the blur kernel corresponding to the number of image blocks of the target channel.

12. The apparatus of claim 10 or 11, wherein, The acquisition unit acquires a blur kernel for deblurring processing for each color channel image in the following manner: acquiring a current lens attribute and a current field of view used by the camera; acquiring a blur kernel matching the current lens attribute and the current field of view for each color channel image.

13. The apparatus of claim 10 or 11, wherein, The processing unit adopts the following manner to respectively perform deblurring processing on each color channel image based on the corresponding blur kernel: respectively performing segmentation on each of the color channel images to obtain a first sub-image corresponding to the number of blur kernels of the channel image; respectively performing blur edge extension on each first sub-image, and performing deblurring processing on the first sub-image after blur edge extension based on the blur kernel corresponding to the first sub-image to obtain a first sub-image after deblurring processing; respectively splicing the first sub-image after deblurring processing for each color channel image to obtain an image after deblurring processing of each color channel image.

14. The photographing apparatus according to claim 13, wherein The processing unit adopts the following manner to perform blur edge extension on each first sub-image: extending m rows and n columns of pixels on the rightmost edge of the first sub-image to obtain a second sub-image, where m is the number of rows of pixels in the first sub-image, and n is a preset positive integer; extending n rows and m+n columns of pixels on the lowermost edge of the second sub-image to obtain the first sub-image after edge extension.

15. The photographing apparatus according to claim 14, wherein The processing unit adopts the following manner to extend m rows and n columns of pixels on the rightmost edge of the first sub-image to obtain a second sub-image: extending m rows and n columns of pixels with a value of 0 on the rightmost edge of the first sub-image; assigning the pixel value of the first column of the n columns based on the pixel values of the first column and the last column of the first sub-image; assigning the pixel value of the n-th column of the n columns based on the pixel values of the first column and the last column of the first sub-image; assigning the pixel value of the i-th column of the n columns based on the pixel values of the i-1-th column and the n-i+1-th column, and assigning the pixel value of the n-i-th column of the n columns based on the pixel value of the i-th column and the pixel value of the n-i+1-th column, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; repeating the above operations until the pixel values of all pixels in the n columns are determined to obtain the second sub-image.

16. The photographing apparatus according to claim 14, wherein extending n rows and m+n columns of pixels on the lowermost edge of the second sub-image to obtain the first sub-image after edge extension, including: extending n rows and m+n columns of pixels with a value of 0 on the lowermost edge of the first sub-image; assigning the pixel value of the first row of the n rows based on the pixel values of the first row and the last row of the first sub-image; assigning the pixel value of the n-th row of the n rows based on the pixel values of the first row and the last row of the first sub-image; assigning the pixel value of the i-th row of the n rows based on the pixel values of the i-1-th row and the n-i+1-th row, and assigning the pixel value of the n-i-th row of the n rows based on the pixel value of the i-th row and the pixel value of the n-i+1-th row, where i is a positive integer greater than or equal to 2 and less than or equal to n / 2; repeating the above operations until the pixel values of all pixels in the n rows are determined to obtain the first sub-image after edge extension.

17. An electronic device, comprising: including: a processor; a memory for storing processor-executable instructions; The processor is configured to execute the method in any one of claims 1 to 8.

18. A storage medium, characterized by The storage medium has instructions stored therein, and when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the method in any one of claims 1 to 8.

Images