Shooting method, electronic device and storage medium

Abstract

The embodiments of the present application provide a shooting method, an electronic device, and a storage medium. In the method, the electronic device determines a first position and at least one second position corresponding to a camera motor in a moiré removal scenario of a target application, and in response to a target operation, collects a first frame corresponding to the first position and a second frame corresponding to the second position, thereby performing moiré detection on the first frame and the second frame. Furthermore, the electronic device determines a target frame when moiré exists in the first frame, and performs aberration resolution on the target frame to obtain a shooting frame corresponding to the target operation. In this way, the electronic device collects multiple image frames corresponding to different camera motor positions, and obtains a captured image without moiré based on the moiré detection results of each image frame, thereby avoiding the appearance of moiré in the captured image and improving the image quality of the captured image.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent terminals, and in particular to a photographing method, an electronic device and a storage medium. Background Art

[0002] With the development of terminal technology and the improvement of electronic device functions, people may need to use the camera of the electronic device to capture the content displayed on the screen of other devices, or capture objects with dense textures such as clothing, buildings, and roofs when using electronic devices.

[0003] If the pixel spatial frequency of the camera of the electronic device is close to the spatial frequency of the stripes in the captured image, moiré will be generated in the image captured by the camera of the electronic device. The moiré in the image will affect the display quality of the image, so it is necessary to eliminate the moiré in the image. Summary of the invention

[0004] In order to solve the above technical problems, the embodiments of the present application provide a shooting method, an electronic device and a storage medium. In the method, the electronic device acquires a plurality of image frames corresponding to different camera motor positions, and obtains a captured image without moiré according to the moiré detection results of each image frame, thereby avoiding the occurrence of moiré in the captured image and improving the image quality of the captured image.

[0005] In a first aspect, an embodiment of the present application provides a shooting method. The method includes: the electronic device determines the first position and the second position corresponding to the camera motor in a moiré removal scenario of a target application; in response to a target operation, the electronic device collects a first frame corresponding to the first position and a second frame corresponding to the second position, and performs moiré detection on the first frame and the second frame; when moiré exists in the first frame, the electronic device determines the target frame according to the first frame and the second frame, or determines the target frame according to the second frame; the electronic device performs aberration resolution on the target frame to obtain a shooting frame corresponding to the target operation.

[0006] The number of the second positions is at least one; and the number of the second frames is the same as the number of the second positions.

[0007] Exemplarily, the target application is a system camera application, or a third-party application with a camera function.

[0008] Exemplarily, the target operation may be an operation of clicking a shooting control of a target application.

[0009] The first position may be the position of the camera motor when the camera is in focus. The second position may be the position of the camera motor when the camera is out of focus. Specifically, the second position may include the position of the camera motor when the camera is out of focus for the first time and the position of the camera motor when the camera is out of focus for the second time.

[0010] The first frame may be a focused frame, and the second frame may be a defocused frame. Specifically, the second frame may include a first defocused frame and a second defocused frame.

[0011] Specifically, the second position corresponding to the camera motor corresponds to the second frame collected. That is, when the camera motor is at the second position, the electronic device collects the second frame.

[0012] Moire detection is performed on the first frame and the second frame, that is, whether moire exists in the first frame and the second frame is detected.

[0013] Specifically, if the first frame does not have moiré patterns, the electronic device may determine the first frame as a captured frame corresponding to the target operation.

[0014] In this way, the electronic device collects multiple image frames corresponding to different camera motor positions and obtains a captured image without moiré based on the moiré detection results of each image frame, thereby avoiding the appearance of moiré in the captured image and improving the image quality of the captured image.

[0015] According to the first aspect, the electronic device determines the second position corresponding to the camera motor, which may include: after determining the first position, the electronic device determines the third position and calculates the depth of field data corresponding to the first position; the electronic device determines the fourth position based on the depth of field data and the third position; the electronic device determines the second position corresponding to the camera motor based on the fourth position.

[0016] Among them, the third position is the focal position when the camera is focused; the fourth position is the focal position when the camera is out of focus.

[0017] The distance between the second position and the fourth position is the same as the distance between the third position and the first position.

[0018] For example, refer to Figure 7 As shown in (1) or (2), the first position may be the position of the camera lens, the third position may be the position of the photographed object, the depth of field data may be the value range of DOF, the fourth position may be the defocus position, and the second position may be the position of the camera lens after the movement.

[0019] Specifically, after determining the first position, the electronic device may determine the third position according to the Gaussian imaging formula 1 / f=1 / u+1 / v, where f is the focal length, u is the object distance when focusing, and v is the image distance when focusing.

[0020] In this way, the electronic device can accurately determine the second position corresponding to the camera motor to reduce the possibility of moiré patterns appearing in the second frame corresponding to the second position, while making the second frame corresponding to the second position have a lighter degree of blur.

[0021] According to the first aspect, or any implementation of the first aspect above, the electronic device calculates the depth of field data corresponding to the first position, which may include: the electronic device calculates the depth of field data corresponding to the first position according to the formula , calculate the depth of field data corresponding to the first position.

[0022] Among them, DOF is the depth of field data; u is the distance between the third position and the first position; F# is the aperture number of the camera; coc is the diameter of the confusion circle when the camera motor is in the first position; f is the focal length of the camera.

[0023] The distance between the third position and the first position is the object distance during focusing.

[0024] According to the first aspect, or any implementation of the first aspect above, the electronic device determines the fourth position based on the depth of field data and the third position, which may include: the electronic device determines the first depth of field position based on the depth of field data; the electronic device determines a first intermediate position between the first depth of field position and the third position based on the first depth of field position and the third position; the electronic device determines the fourth position within the range of the first intermediate position and the first depth of field position.

[0025] For example, refer to Figure 7 As shown in (1), the first depth of field position may be a position of the foreground depth, and the first intermediate position may be a position of 1 / 2 the foreground depth.

[0026] In this way, the electronic device can determine the focal position of the camera when it is out of focus at a position close to the foreground depth of field within the depth of field, thereby further reducing the possibility of moiré patterns appearing in the second frame corresponding to the second position, and at the same time making the second frame corresponding to the second position have a lighter degree of blur.

[0027] According to the first aspect, or any implementation of the first aspect above, the electronic device determines the fourth position based on the depth of field data and the third position, which may include: the electronic device determines the second depth of field position based on the depth of field data; the electronic device determines the second intermediate position between the second depth of field position and the third position based on the second depth of field position and the third position; the electronic device determines the fourth position within the range of the second intermediate position and the second depth of field position.

[0028] For example, refer to Figure 7 As shown in (2), the second depth of field position may be a position of the rear depth of field, and the second intermediate position may be a position of 1 / 2 the rear depth of field.

[0029] In this way, the electronic device can determine the focal position of the camera when it is out of focus within the depth of field range and at a position closer to the rear depth of field, thereby further reducing the possibility of moiré patterns appearing in the second frame corresponding to the second position, and at the same time making the second frame corresponding to the second position have a lighter degree of blur.

[0030] According to the first aspect, or any implementation of the first aspect above, the electronic device determines the second position corresponding to the camera motor, which may include: the electronic device determines the range of the second position based on the first position and the first condition; the electronic device determines the second position within the range of the second position.

[0031] Among them, the first condition, that is, the preset distance threshold condition, can be a preset threshold condition of the distance between the camera motor position when focusing and the camera motor position when defocusing.

[0032] In this way, the electronic device can avoid the situation where the position of the camera motor when defocused is too far away from the position of the camera motor when focused, causing the second frame corresponding to the second position to be too blurry by presetting the distance threshold condition.

[0033] According to the first aspect, or any implementation method of the first aspect above, the electronic device performs moiré detection on the first frame and the second frame, which may include: the electronic device performs a first processing on the first frame to obtain a third frame corresponding to the second frame; the electronic device performs a second processing on the third frame to obtain a fourth frame; the electronic device performs a third processing on the fourth frame to obtain the moiré detection results of the first frame and the second frame.

[0034] The number of the third frames is the same as the number of the second frames.

[0035] Specifically, the first processing may be image blur processing, the second processing may include image registration processing and image difference calculation, and the third processing may include color space conversion, threshold segmentation and pixel point analysis.

[0036] The third frame may be a blurred frame, and the fourth frame may be a difference frame.

[0037] For example, refer to Figure 8 As shown, when the second frame includes the first defocused frame and the second defocused frame, the third frame may include the first blurred frame and the second blurred frame, and the fourth frame may include the first difference frame, the second difference frame and the third difference frame.

[0038] In this way, the electronic device obtains the moiré detection results of the first frame and the second frame by various image processing of the first frame and the second frame, and can accurately determine whether there are moirés in the first frame and the second frame.

[0039] According to the first aspect, or any implementation of the first aspect above, the electronic device performs a first processing on the first frame to obtain a third frame corresponding to the second frame, which may include: the electronic device determines a first difference between the object distance corresponding to the first frame and the object distance corresponding to the second frame; the electronic device determines a first function corresponding to the first difference; the electronic device performs convolution processing on the first function and the first frame to obtain a third frame corresponding to the second frame.

[0040] Among them, the first function, that is, the point spread function corresponding to the first difference.

[0041] For example, refer to Figure 7 As shown in (1) or (2), the object distance corresponding to the first frame is u, and the object distance corresponding to the second frame is u'.

[0042] In this way, the electronic device obtains a blurred frame corresponding to the second frame by convolving the point spread function corresponding to the first difference with the first frame, and can blur the first frame toward the blur degree of the second frame, so that the blur degree of the blurred frame corresponding to the second frame is close to the blur degree of the second frame, thereby avoiding the influence of moiré patterns on the registration effect during the image registration process.

[0043] According to the first aspect, or any implementation of the first aspect above, the electronic device determines the first function corresponding to the first difference, which may include: the electronic device determines the second function corresponding to the first frame; the electronic device determines the target function and the target distance corresponding to the target function based on the object distance corresponding to the first frame and the second function; the electronic device searches for the first target distance in the target distance; the electronic device determines the first target function corresponding to the first target distance as the first function corresponding to the first difference.

[0044] The difference between the first target distance and the first difference is less than the difference threshold.

[0045] Specifically, the difference threshold may be a pre-set value close to 0. For example, the difference threshold may be 0.5 or 0.3, etc., which is not limited in the present application.

[0046] The target distance is an offset distance based on the object distance corresponding to the first frame; and the number of the target function and the target distance is at least one.

[0047] The second function, that is, the point spread function corresponding to the first frame, and the target function, may be a point spread function calibrated based on the point spread function corresponding to the first frame.

[0048] Specifically, there is a one-to-one correspondence between the objective function and the target distance.

[0049] For example, refer to Figure 9aAs shown, when the point spread function corresponding to the first frame is K1 and the object distance corresponding to the first frame is d1, the target function can be a calibrated point spread function K2, a calibrated point spread function K3, or a calibrated point spread function K4, and the target distance can be an offset distance o2 based on the object distance d1, an offset distance o3 based on the object distance d1, or an offset distance o4 based on the object distance d1.

[0050] In this way, the electronic device can accurately determine the point spread function corresponding to the blur degree when blurring the first frame.

[0051] According to the first aspect, or any implementation of the first aspect above, the electronic device performs a second processing on the third frame to obtain a fourth frame, which may include: the electronic device performs a registration processing on the third frame and the second frame to obtain a fifth frame corresponding to the third frame; the electronic device performs a difference calculation on the fifth frame, the third frame and / or the second frame to obtain a fourth frame.

[0052] Among them, the number of the fifth frame is the same as the number of the third frame.

[0053] The fifth frame may be a registration frame.

[0054] For example, refer to Figure 8 As shown, when the second frame includes a first defocused frame and a second defocused frame, the fifth frame may include a first registered frame and a second registered frame.

[0055] In this way, the electronic device obtains difference image frames between different image frames by performing registration processing and difference calculation on each image frame, and can clearly show the moiré pattern in the image.

[0056] According to the first aspect, or any implementation of the first aspect above, when the number of second frames is multiple, the electronic device aligns the third frame with the second frame, which may include: the electronic device aligns the second frame with the third frame corresponding to the second frame.

[0057] Specifically, when there are multiple second frames, each second frame has a corresponding third frame, and each second frame is registered with the corresponding third frame.

[0058] According to the first aspect, or any implementation manner of the first aspect above, when the number of second frames is multiple, the electronic device performs difference calculation on the fifth frame, the third frame and / or the second frame, which may include: the electronic device performs difference calculation on each second frame, or performs difference calculation on each third frame; the electronic device performs difference calculation on the third frame and the fifth frame corresponding to the third frame.

[0059] For example, refer to Fig.11As shown in (1) or (2), the electronic device can calculate the difference between the blurred frame and the registration frame corresponding to the blurred frame.

[0060] For example, refer to Fig.11 As shown in (3), when the defocused frame includes the first defocused frame and the second defocused frame, and the blurred frame includes the first blurred frame and the second blurred frame, the electronic device can perform a difference calculation on the first defocused frame and the second defocused frame. Fig.11 As shown in (4), the electronic device can perform difference calculation on the first blurred frame and the second blurred frame.

[0061] In this way, the electronic device can more clearly display the moiré patterns in the image by performing difference calculations on each image frame.

[0062] According to the first aspect, or any implementation method of the first aspect above, the electronic device performs a third processing on the fourth frame to obtain the moiré detection results of the first frame and the second frame, which may include: the electronic device performs a color space conversion on the fourth frame to obtain a sixth frame; the electronic device performs threshold segmentation and pixel analysis on the sixth frame to obtain the moiré detection results of the first frame and the second frame.

[0063] The sixth frame is an image frame in the LCH color space, and the number of the sixth frame is the same as the number of the fourth frame.

[0064] Specifically, the fourth frame is an image frame in the sRGB color space.

[0065] Specifically, performing color space conversion on the fourth frame may include converting the image frame in the sRGB color space into an image frame in the RGB color space, then converting it into an image frame in the LAB color space, and finally converting it into an image frame in the LCH color space.

[0066] In this way, the electronic device can further improve the accuracy of the moiré detection results of the first frame and the second frame by performing color space conversion, threshold analysis, and pixel point analysis on the difference frame.

[0067] According to the first aspect, or any implementation method of the first aspect above, the electronic device performs threshold segmentation and pixel analysis on the sixth frame, which may include: the electronic device determines the first value and the second value of each pixel in the sixth frame; the electronic device segments the pixels whose first value and / or second value meet the second condition; the electronic device determines, in the image frame after the pixel segmentation, an area whose number of pixels meets the third condition, and when the number of areas meets the fourth condition, determines that the second frame and the first frame corresponding to the fourth frame have moiré patterns; when the number of areas does not meet the fourth condition, the electronic device determines that the first frame and / or the second frame corresponding to the fourth frame do not have moiré patterns.

[0068] The first value may be a value of the pixel in the C color space, and the second value may be a value of the pixel in the H color space.

[0069] The second condition may be a pre-set condition for segmenting pixels.

[0070] Specifically, the second condition includes whether the first value or the second value of the pixel point is greater than the average value of the first value or the second value of each pixel point in the sixth frame, or whether the first value or the second value of the pixel point is greater than the median value of the first value or the second value of each pixel point in the sixth frame.

[0071] The third condition may be whether the number of pixels in the moiré pattern and the area in the image frame after pixel segmentation is greater than a preset number of pixels.

[0072] Specifically, the region refers to the moiré pattern together with the region.

[0073] The fourth condition may be whether the number of moiré patterns and regions in the image frame after pixel segmentation is greater than a preset number of moiré patterns and regions.

[0074] According to the first aspect, or any implementation of the first aspect above, when the number of second frames is one, the electronic device determines the target frame based on the first frame and the second frame, or determines the target frame based on the second frame, which may include: when there is moiré in the second frame, the electronic device performs image fusion on the first frame and the second frame to obtain the target frame; when there is no moiré in the second frame, the electronic device determines the second frame as the target frame.

[0075] Exemplarily, image fusion may be, for example, median filtering among multiple frames.

[0076] In this way, when the number of the second frames is one, the electronic device can determine the target frame according to whether the second frame has moiré patterns, so as to remove the moiré patterns of the image frame, or obtain an image frame without moiré patterns.

[0077] According to the first aspect, or any implementation of the first aspect above, when the number of second frames is multiple, the electronic device determines the target frame according to the first frame and the second frame, or determines the target frame according to the second frame, which may include: when moiré exists in each second frame, the electronic device fuses the first frame and the second frame to obtain the target frame; when the electronic device includes a second frame without moiré in each second frame, the second frame without moiré is determined as the target frame; when the electronic device includes multiple second frames without moiré in each second frame, one of the second frames without moiré is determined as the target frame, or the second frames without moiré are fused to obtain the target frame.

[0078] In this way, when the number of second frames is multiple, the electronic device can determine the target frame according to whether the second frame has moiré patterns, so as to remove the moiré patterns of the image frame, or obtain an image frame without moiré patterns.

[0079] According to the first aspect, or any implementation method of the first aspect above, the electronic device performs aberration resolution on the target frame to obtain a captured frame corresponding to the target operation, which may include: the electronic device determines a third function corresponding to the target frame; the electronic device determines the target frame in the Fourier domain and the third function in the Fourier domain based on the target frame and the third function corresponding to the target frame; the electronic device determines the captured frame in the Fourier domain based on the target frame in the Fourier domain and the third function in the Fourier domain; the electronic device performs an inverse Fourier transform on the captured frame in the Fourier domain to obtain a captured frame corresponding to the target operation.

[0080] The third function may be a point spread function corresponding to the target frame.

[0081] Specifically, the target frame in the Fourier domain and the third function in the Fourier domain are subjected to deconvolution processing to obtain a captured frame in the Fourier domain.

[0082] In this way, the electronic device can restore the high-frequency details in the out-of-focus frame or restore the detail loss in the moiré area in the fused image frame through aberration resolution.

[0083] According to the first aspect, or any implementation of the first aspect above, the electronic device determines a third function corresponding to a target frame, including: the electronic device determines a second function corresponding to a first frame, and a fourth function corresponding to a second frame; when the target frame is determined by a first frame and a second frame, the electronic device determines an average function of the second function and the fourth function as the third function; when the target frame is determined by a second frame, the electronic device determines the fourth function as the third function; when the target frame is determined by multiple second frames, the electronic device determines an average function of each fourth function as the third function.

[0084] The third function may be a point spread function corresponding to the second frame.

[0085] In a second aspect, an embodiment of the present application provides an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the shooting method of the first aspect and any one of the first aspects.

[0086] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0087] In a third aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes the shooting method of the first aspect and any one of the first aspects.

[0088] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0089] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed, enables a computer to execute a shooting method as described in the first aspect or any one of the items in the first aspect.

[0090] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0091] In a fifth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the shooting method of the first aspect or any one of the first aspects to control the receiving pin to receive a signal, and to control the sending pin to send a signal.

[0092] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 A plurality of captured images are shown as examples;

[0094] Figure 2 is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0095] Figure 3 A schematic diagram of the software structure of an electronic device is shown as an example;

[0096] Figure 4 An application scenario is shown as an example;

[0097] Figure 5a is a flowchart of an exemplary shooting method;

[0098] Figure 5b A schematic diagram of a motor position is shown as an example;

[0099] Figure 6 The figure is a schematic diagram of module interaction of an exemplary shooting method;

[0100] Figure 7 is a schematic diagram showing an exemplary method of determining a motor position when defocusing;

[0101] Figure 8 The figure is a schematic diagram of a process of moiré detection shown as an example;

[0102] Figure 9a-9c A schematic diagram of a fuzzy process is shown as an example;

[0103] Fig.10 is a schematic diagram of an exemplary registration process;

[0104] Fig.11 A schematic diagram of a difference calculation is shown as an example;

[0105] Fig.12 is a schematic diagram of an exemplary color space conversion;

[0106] Fig.13 is a schematic diagram of an exemplary moiré mask;

[0107] Figure 14a-Figure 14c A schematic diagram showing an exemplary aberration solution. DETAILED DESCRIPTION

[0108] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0109] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0110] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0111] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0112] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0113] Some terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0114] Focusing, also known as focusing, means that the focus is on the object being photographed when shooting an image, that is, the object being photographed can be clearly imaged. In other words, the image shot by the electronic device when focusing can ensure that the object being photographed can be clearly imaged.

[0115] Defocus means that the focus is not on the object when shooting an image, that is, the object cannot be clearly imaged and the image will appear blurred. In other words, the image taken by the electronic device when it is out of focus will cause the image of the object to be photographed to be blurred because the light is not accurately focused.

[0116] The motor is the motor corresponding to the camera of the electronic device. The motor is surrounded by a lens, and the position of the motor is also the position of the lens. The motor can push the lens to move to achieve the focusing or defocusing function.

[0117] The focus frame can be when the motor pushes the lens to the motor position when the focus is achieved, and the camera captures the image frame, that is, a clear image frame.

[0118] The defocused frame may be an image frame captured by the camera when the motor pushes the lens to the motor position when the lens is defocused, that is, a blurred image frame.

[0119] Depth of field DOF refers to the imaging range in which a clear image can be obtained in a camera lens or other imager, or it can be understood as the clear range formed before and after the focus. That is, after focusing is completed, the distance of the clear image presented in the range before and after the focus is called the depth of field.

[0120] The depth of field may include the front depth of field and the back depth of field. The front depth of field may include the clear range before the focus, and the back depth of field may include the clear range after the focus.

[0121] The circle of confusion refers to a diffuse circular projection formed on the image plane when the imaging light beam cannot converge at one point due to aberration when the object is imaged. Generally, the closer the object distance is, the larger the circle of confusion is.

[0122] Point Spread Function (PSF) is a function that describes the imaging characteristics of a point light source or a point target in an imaging system. It reflects the response of the imaging system to the point light source. The blur of the image is closely related to the shape and size of the point spread function. The wider the point spread function, the more blurred the response of the imaging system to the point light source, that is, the higher the blur of the image. Therefore, by determining the point spread function, the blur of the image can be determined, and the image can be blurred.

[0123] LCH image frame refers to the representation of image frame in LCH color space. L represents the brightness component, that is, the lightness or brightness level of the color. C represents the chroma component, that is, the saturation component, that is, the purity and vividness of the color. H represents the hue component, that is, the position of the color on the color wheel.

[0124] RGB image frame refers to the representation of image frame in RGB color space. R represents red, G represents green, and B represents blue.

[0125] LAB image frame refers to the representation of image frames in LAB color space. L represents brightness, ranging from 0 (black) to 100 (white). A represents color components from green to red. B represents color components from blue to yellow.

[0126] sRGB image frame refers to the way image frames are represented in the standard RGB color space.

[0127] A mask refers to a matrix or image frame having the same size as an original image frame, which is used to indicate or identify pixels or regions in a specific area in the original image frame.

[0128] When electronic devices use cameras to capture objects with dense textures, such as clothing, buildings, or displays, moiré patterns are often produced. Figure 1Several captured images (or photos) are shown as examples, wherein the captured images are shown in the form of grayscale images. Figure 1 (1) is the moiré pattern produced when photographing clothing. Figure 1 (2) in the figure is the moiré pattern produced when photographing the display screen.

[0129] In such Figure 1 In the captured image shown in (1), since the physical size spatial frequency of the clothing is close to the pixel spatial frequency of the camera, moiré patterns will be generated in the image when the camera captures the clothing image, for example Figure 1 The area 101 shown in (1) in FIG. Figure 1 In the captured image shown in (2), since the physical size spatial frequency of the captured display screen is close to the pixel spatial frequency of the camera, moiré patterns will be generated in the image when the camera captures the display screen, for example Figure 1 The area 102 shown in (1) in FIG.

[0130] The moiré patterns in the captured images will affect the display quality of the images, and therefore, the moiré patterns in the images need to be eliminated.

[0131] In order to solve the above technical problems, an embodiment of the present application provides a shooting method. In the shooting method, the electronic device responds to the user's operation of triggering the moiré removal control to determine the motor position when defocusing. In this way, the electronic device can collect in-focus frames and out-of-focus frames when the user triggers the photo control, and perform moiré detection on the in-focus frames and out-of-focus frames. Among them, the number of defocus frames can be one or more. Furthermore, when there is moiré in the in-focus frame, the electronic device can select a defocus frame with a lighter degree of moiré or fuse multiple frames to perform aberration resolution, thereby restoring the high-frequency information in the image.

[0132] In this way, the electronic device can improve the image quality of the captured image and avoid the occurrence of moiré patterns in the captured image.

[0133] The shooting method provided in the embodiment of the present application can be applied to electronic devices. The electronic device can be any device with a camera function. Optionally, the electronic device in the embodiment of the present application can be a mobile phone with a camera function, a sports camera (GoPro), a digital camera, a tablet computer, a wearable device, a desktop, a laptop, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmentedreality, AR) / virtual reality (virtual reality, VR) device, etc. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0134] In order to better understand the embodiments of the present application, the structure of the electronic device in the embodiments of the present application is introduced below:

[0135] like Figure 2 The figure shows a schematic diagram of the structure of the electronic device 100. Optionally, the electronic device 100 may be a terminal, which may also be referred to as a terminal device, and the terminal may be a device with a camera function such as a cellular phone, a tablet computer (pad), a wearable device or an Internet of Things device, which is not limited in this application.

[0136] It should be understood that Figure 2 The electronic device 100 shown is only one example of an electronic device, and the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 2 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0137] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an acceleration sensor, a temperature sensor, a motion sensor, an air pressure sensor, a magnetic sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0138] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0139] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0140] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0141] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0142] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0143] The mobile communication module 150 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0144] The wireless communication module 160 can provide wireless communication solutions applied to the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0145] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0146] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0147] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0148] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor. ISP is used to process the data fed back by camera 193. Camera 193 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to ISP for conversion into a digital image signal. ISP outputs the digital image signal to DSP for processing. DSP converts the digital image signal into an image signal in standard RGB, YUV and other formats.

[0149] In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

[0150] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function.

[0151] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121, for example, enabling the electronic device 100 to implement the shooting method in the embodiment of the present application. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc.

[0152] In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0153] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0154] The pressure sensor is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor can be arranged on the display screen 194. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor.

[0155] The touch sensor is also called a "touch panel". The touch sensor can be set on the display screen 194, and the touch sensor and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. In some embodiments, the touch screen can be a folding screen.

[0156] The battery may include one or more cells and may be used to power a load.

[0157] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0158] The charging management module 140 may include a plurality of charging chips, wherein the plurality of charging chips may be connected to one battery, or each chip may be connected to one battery, so that the charging management module 140 may charge the battery through the charging chips.

[0159] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the memory 121, the display screen 194, the camera 193 and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0160] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0161] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0162] The layered architecture of the electronic device 100 divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android Runtime) and the system library, the hardware abstraction layer (HAL) and the kernel layer.

[0163] The application layer can include a series of application packages.

[0164] like Figure 3 As shown, the application package may include a camera, a gallery, and a third-party application with a camera function, etc. Exemplarily, the application package may include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, and a short message.

[0165] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer, including various components and services to support developers' Android development. The application framework layer includes some predefined functions.

[0166] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, a camera service, a moiré removal module, and the like.

[0167] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0168] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0169] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0170] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0171] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0172] The camera service is used to call the camera (including the front camera and / or the rear camera) in response to the application's request. Specifically in this embodiment, the camera service can respond to the image capture request sent by the application, and then perform image acquisition according to the image capture request to obtain an image.

[0173] The moiré removal module can be used to determine the motor position when the image is out of focus, and to detect moiré in the in-focus and out-of-focus frames. When there is no moiré in the in-focus frame, the moiré removal module can also be used to select an out-of-focus frame with less moiré or fuse multiple frames for aberration resolution.

[0174] Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system.

[0175] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0176] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0177] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0178] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0179] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0180] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0181] A 2D graphics engine is a drawing engine for 2D drawings.

[0182] The HAL layer is an interface layer between the operating system kernel and the hardware circuit. The HAL layer includes but is not limited to: a camera HAL module and an audio HAL module. The camera HAL module is used to process the image stream, and the audio HAL module is used to process the audio stream (for example, noise reduction, directional enhancement, etc.).

[0183] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, audio driver, image sensor driver module, image signal processing module, etc. The hardware includes at least processor, display screen, camera, ISP, etc. Among them, the image signal processing module is used to process image preview request and photo request according to the instruction of camera HAL module, so that the image sensor driver module can collect images according to the corresponding request. The image signal processing module can also be used to process the original image collected by the camera, such as image denoising, image optimization, etc.

[0184] Understandably, Figure 3 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0185] It is understandable that in order to implement the shooting method in the embodiment of the present application, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0186] A shooting scenario provided by an embodiment of the present application is described below. In this scenario, an electronic device is taken as an example of a mobile phone for explanation.

[0187] It is understandable that the term "user interface" in the specification, claims and drawings of this application refers to the medium interface for interaction and information exchange between the application or operating system and the user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to the user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. Visual interface elements.

[0188] Figure 4 (1) in the figure shows an exemplary user interface 200 for displaying applications on a mobile phone. The user interface 200 displays a page with application icons, which may include multiple application icons (for example, a clock application icon, a calendar application icon, a gallery application icon 220, a memo application icon, an email application icon, an application store application icon, a settings application icon, etc.). A page indicator may also be displayed below the multiple application icons to indicate the positional relationship between the currently displayed page and other pages. There are multiple tray icons below the page indicator (for example, a camera application icon 210, a contact application icon, a phone application icon, and a message application icon). The tray icon remains displayed when the page is switched. The embodiment of the present application does not limit the content displayed on the user interface 200.

[0189] It is understandable that the mobile phone can detect the user operation (such as touch / click operation) of the user on the camera application icon 210, and in response to the operation, the mobile phone can display Figure 4 The shooting interface 300 shown in (2) in the figure. The shooting interface 300 can be the user interface of the default shooting mode of the camera application, and the user can complete the shooting on this interface. Furthermore, the image obtained by the shooting can be stored in the gallery application (the user can view the image obtained by touching or clicking the gallery application icon 220). The camera application is an image shooting application on electronic devices such as smartphones and tablets. This application does not limit the name of this application. The gallery application is an image storage application on electronic devices such as smartphones and tablets. This application does not limit the name of this application. In other words, the user can open the shooting interface 300 of the camera application by clicking the camera application icon 210. It can be understood that the default camera in the default shooting mode is not limited to the rear camera, and the mobile phone can also set the front camera as the default camera.

[0190] Figure 4(2) in FIG. 1 shows an exemplary user interface 300 of a camera application in a mobile phone. Figure 4 As shown in (2) in FIG. 3 , the shooting interface 300 may include a parameter adjustment area 310, a preview area 320, a camera mode option area 330, a gallery shortcut control 340, a shutter control 350, and a camera flip control 360. The various controls in the parameter adjustment area 310 are used to adjust corresponding shooting parameters, including but not limited to: a flash setting control, an AI recognition switch setting control, a color standard setting control, and a more detailed camera setting control.

[0191] The preview area 320 can be used to display a preview image, which is an image collected by the mobile phone through the camera in real time. The mobile phone can refresh the display content in the preview area 320 in real time to facilitate the user to preview the image currently collected by the camera.

[0192] One or more shooting mode options may be displayed in the camera mode option 330. The one or more shooting mode options may include: a moiré removal mode option 331, a night scene mode option, a portrait mode option, a photo mode option, a video mode option, and more options. It is understandable that the one or more shooting mode options may be displayed as text information on the interface, such as "moiré removal", "night scene", "portrait", "photo", "video", "more", and may also be displayed as icons or other forms of interactive elements (IE), which are not limited in this application.

[0193] When a user operation on a moiré removal mode option is detected, the mobile phone can start the moiré removal mode selected by the user. In particular, when a user operation on more options is detected, the mobile phone can further display more other shooting mode options, such as a time-lapse photography mode option, a continuous shooting mode option, etc., to show the user more abundant camera functions. It is understandable that the camera mode option area 330 can also include more or fewer shooting mode options. Figure 4 The camera mode option shown in (2) is only one implementation of the present application and should not be regarded as a limitation of the present application.

[0194] The moiré removal mode control 331 can be used to monitor the user operation that triggers the moiré removal. The mobile phone can detect the user operation acting on the moiré removal mode control 331, and in response to the operation, the mobile phone can turn on the moiré removal shooting mode. In the moiré removal shooting mode, the mobile phone can remove the moiré in the image when shooting the image. That is, the user can enter the moiré removal mode by operating the moiré removal mode control 331.

[0195] The gallery quick control 320 can be used to open the gallery application. In response to a user operation, such as a touch operation, acting on the gallery quick control 320, the mobile phone can open the gallery application.

[0196] The shutter control 340 can be used to monitor the user operation that triggers the photo taking. The mobile phone can detect the user operation acting on the shutter control 340. In response to the operation, the mobile phone obtains the corresponding image frame and saves the final output image as a picture in the gallery application. In addition, the mobile phone can also display thumbnails of the saved images in the gallery shortcut control 320. In other words, the user can trigger the photo taking by operating the shutter control 340. It can be understood that the shutter control 340 can be a button or other forms of controls, and this application does not limit this.

[0197] The camera flip control 360 can be used to monitor the user operation that triggers the flipping of the camera. The mobile phone can detect the user operation acting on the camera flip control 360, such as a touch operation, and in response to the operation, the mobile phone can switch the camera used for shooting, such as switching the rear camera to the front camera, or switching the front camera to the rear camera.

[0198] The shooting interface 300 may also include more or fewer controls, which is not limited in the embodiment of the present application.

[0199] Continue to refer to Figure 4 In (2), after the user clicks the moiré removal control 330 to enter the moiré removal mode, the user can click the shutter control 340, such as Figure 4 As shown in (2) in the figure. The user clicks the shutter control 340. In response to the user operation, the camera application sends a photo request to the kernel layer. After receiving the photo request, the kernel layer reports the continuous multiple frames of image data subsequently output by the image sensor to the camera HAL module. The camera HAL module will call back one or more frames to the moiré removal module. After removing the moiré, the moiré removal module sends a photo frame and a thumbnail to the camera application. The camera application can confirm that the photo is completed based on the received photo frame or thumbnail, and then restore the shutter control 340 to the enabled state, so that the user can take the next photo.

[0200] Figure 5a The flowchart of a shooting method is shown as an example. Figure 5a As shown, the process of the shooting method specifically includes:

[0201] In response to the user operation on the camera application icon 210, the mobile phone displays Figure 4In the shooting interface 300 shown in (2), the user can trigger the moiré removal option 331, and the mobile phone responds to the operation and enters the moiré removal mode. At the same time, the camera of the mobile phone performs a continuous focusing operation.

[0202] When the camera of the mobile phone is focusing, the mobile phone can determine the motor position when focusing and the motor position when defocusing. Specifically, the motor position when defocusing can include the first motor position when defocusing and the second motor position when defocusing.

[0203] The motor position when out of focus can be on either side of the motor position when in focus (e.g. Figure 5b ), or on the same side of the motor position during focusing (as shown in (1) in the figure). Figure 5b (2) and (3) in the figure).

[0204] In this case, if the user clicks the shutter control 350, the mobile phone responds to the user operation, and the camera application sends a photo request. In this way, the mobile phone can collect the focus frame and the defocus frame according to the motor position when focusing and the motor position when defocusing. Among them, the defocus frame can include the first defocus frame and the second defocus frame.

[0205] Therefore, the mobile phone can perform moiré detection on the collected in-focus frames and out-focus frames to determine whether there are moiré patterns in the in-focus frames and out-focus frames.

[0206] If there is no moiré in the in-focus frame, the in-focus frame can be determined as the captured frame and reported to the camera application. If there is moiré in the in-focus frame and no moiré in the defocused frame, the defocused frame can be subjected to aberration analysis to obtain the captured frame and reported to the camera application. If there is moiré in both the in-focus frame and the defocused frame, the in-focus frame and the defocused frame can be fused, and the aberration analysis can be performed on the fused image frame to obtain the captured frame and report to the camera application.

[0207] The following is a specific example to explain a shooting method provided in an embodiment of the present application. Figure 6 FIG. 1 is a schematic diagram showing the interaction of various modules of an electronic device, as shown in FIG. Figure 6 As shown, the process of the shooting method may specifically include:

[0208] S101: In response to a user clicking a moiré removal control, the camera application sends a focus request to the camera service. After receiving the focus request, the camera service sends the focus request to the camera driver module via the camera HAL module.

[0209] After the user clicks the moiré removal control, the camera application enters the moiré removal mode. After entering the moiré removal mode, the camera application can send a focus request to the camera service.

[0210] S102, the camera driving module obtains focus data through the camera, and returns the focus data to the camera HAL. After receiving the focus data, the camera HAL module sends the focus data to the moiré removal module.

[0211] The focus data may include motor position data during focusing, etc.

[0212] Specifically, after receiving the focus data, the camera HAL module can also send the focus data to the camera application through the camera service ( Figure 6 (not shown). The camera application receives the focus data sent by the camera service and determines that the focus is successful.

[0213] It should be noted that after the camera application enters the moiré removal mode, it can continuously send focus requests to the camera service until the focus is successful.

[0214] It is understandable that if the camera driver module fails to obtain the focus data through the camera, it means that the camera has not focused successfully, and it can be determined that the focus has failed. Therefore, the camera driver module cannot report the focus data to the camera application, and the camera application can continue to send a focus request to the camera service without receiving the focus data.

[0215] S103: After receiving the focus data, the moiré removal module may determine the first motor position and the second motor position corresponding to the defocus according to the focus data, and send the first motor position and the second motor position to the camera HAL module.

[0216] The first motor position may be a motor position when the image is out of focus. The second motor position may be another motor position when the image is out of focus. Optionally, the first motor position and the second motor position may be located on either side of the motor position when the image is in focus.

[0217] Reference Figure 7 As shown in (1) in the figure, when focusing, the position of the object being photographed is also the focal position when in focus. The position of the camera lens when focusing is also the motor position when in focus. The position of the photosensitive element is also the position of the image sensor in the mobile phone camera. The distance between the camera lens and the object being photographed when focusing is also the object distance when in focus, as shown in Figure 7 u shown in (1) in the figure. The distance between the camera lens and the photosensitive element when focusing, that is, the image distance when in focus, is as follows: Figure 7 v shown in (1) in .

[0218] After successful focusing, the moiré removal module can calculate the depth of field DOF corresponding to the photographed object based on the object distance, focal length, aperture number and confusion circle diameter when in focus.

[0219] Specifically, the moiré removal module can calculate the depth of field according to the following formula:

[0220]

[0221] Among them, u represents the object distance, F# represents the aperture number, coc represents the diameter of the circle of confusion, and f represents the focal length.

[0222] It can be understood that the diameter of the circle of confusion can be determined based on the modulation transfer function of the lens.

[0223] Continue to refer to Figure 7 In (1), after determining the depth of field, the moiré removal module can determine a defocus position in the range of 1 / 2 of the foreground depth of field to the foreground depth of field. The distance between the defocus position and the moved camera lens can be the object distance u when the focus is achieved.

[0224] Specifically, when the camera is out of focus, the camera motor can push the camera lens to the position of the camera lens after the movement, so that the focus position moves to the defocused position when the camera is out of focus. Therefore, a motor position when the camera is out of focus can be determined according to the position of the camera lens after the movement.

[0225] Continue to refer to Figure 7 (1) in the figure is the distance between the camera lens and the object after the camera moves, that is, the object distance when the camera is out of focus, as Figure 7 u' shown in (1) in the figure. The distance between the camera lens and the photosensitive element after the movement, that is, the image distance when out of focus, is as follows: Figure 7 v' shown in (1) in .

[0226] Specifically, after the camera lens moves, the image distance when defocused can be determined according to the focal length and the object distance when defocused, and the motor position when defocused can be determined according to the image distance when defocused. It can be understood that the focal length of the camera is fixed and it can be determined according to the Gaussian imaging formula that when the object distance changes, the image distance also changes accordingly.

[0227] Reference Figure 7 As shown in (2), after determining the depth of field, the moiré removal module can determine another defocus position within the range of 1 / 2 of the back depth of field to the back depth of field. The distance between the defocus position and the camera lens after the movement can be the object distance u when the focus is in focus.

[0228] Specifically, when the camera is defocused, the camera motor can push the camera lens to the position of the camera lens after the movement, so that the focus position moves to the defocused position when the camera is defocused. Thus, another motor position when the camera is defocused can be determined according to the position of the camera lens after the movement.

[0229] Continue to refer to Figure 7(2) in the figure is the distance between the camera lens and the object after the camera moves, that is, the object distance when the camera is out of focus, as Figure 7 u' shown in (2) in the figure. The distance between the camera lens and the photosensitive element after the movement, that is, the image distance when out of focus, is as follows: Figure 7 v' shown in (2) in .

[0230] Specifically, after the camera lens moves, the image distance when defocused can be determined according to the focal length and the object distance when defocused, and thus the motor position when defocused can be determined according to the image distance when defocused.

[0231] It is understandable that when the camera motor position is moved, the position of the photosensitive element and the position of the object being photographed remain fixed, and the focus position is changed by moving the camera lens position. When the focus position is at the position of the object being photographed, it is in focus. When the focus position is at the defocus position, it is defocused. When in focus, the image captured is a clear image of the object. When defocused, the image captured is a blurred image of the object.

[0232] In the embodiment of the present application, the defocus position is determined within the range of 1 / 2 foreground depth of field to foreground depth of field, and within the range of 1 / 2 back depth of field to back depth of field, which can ensure that the captured blurred image is not too blurred.

[0233] In an optional embodiment, the focus data may include motor position data when in focus and at least one motor position data when out of focus. Accordingly, the moiré removal module may determine the first motor position and the second motor position corresponding to the defocus according to the motor position data when in focus and at least one motor position data when out of focus in the focus data.

[0234] Specifically, when the distance between the motor position when in focus and the motor position when out of focus meets a preset distance threshold, the motor position when out of focus is determined as the first motor position or the second motor position. The preset distance threshold can be set according to actual needs, and the embodiment of the present application does not limit this.

[0235] It should be noted that, under normal circumstances, the pixel size of a mobile phone sensor is 0.8um-2.0um, and the pixel size of the monitor being photographed is 0.2mm-0.3mm. The pixel size of the monitor being photographed is generally 100 times that of the mobile phone sensor. At this time, the pixel size of the monitor being photographed when it is on the sensor image plane is close to the sensor pixel size. As a result, when taking an image of the monitor being photographed, more obvious moiré patterns will appear.

[0236] In an embodiment of the present application, by moving the camera motor position to a defocused position, the pixel size of the displayed image being photographed is made to differ greatly from the pixel size of the sensor when on the sensor image plane, thereby achieving a defocused frame shot at the defocused position without obvious moiré patterns.

[0237] S104: The camera application responds to the user clicking the photo control and sends a shooting request to the camera HAL module via the camera service.

[0238] The camera service receives the image capture request, performs relevant processing corresponding to the capture request, such as creating a corresponding service instance, and sends an image frame acquisition request corresponding to the capture request to the camera HAL module. For the relevant processing flow of the camera service, please refer to the existing technology, which will not be repeated here.

[0239] S105: After receiving the shooting request, the camera HAL module sends the first motor position, the second motor position and the shooting request to the camera driving module. The camera driving module obtains the in-focus frame, the first out-of-focus frame and the second out-of-focus frame through the camera, and sends the in-focus frame, the first out-of-focus frame and the second out-of-focus frame to the moiré removal module through the camera HAL module.

[0240] The first defocused frame and the second defocused frame may be image frames captured by the camera motor at a first motor position and a second motor position, respectively.

[0241] S106 , a moiré removal module performs moiré detection on the in-focus frame, the first out-of-focus frame, and the second out-of-focus frame.

[0242] Specifically, the process of the moiré removal module performing moiré detection on the in-focus frame, the first out-of-focus frame, and the second out-of-focus frame may include:

[0243] S1. Blur the focused frame to obtain a first blurred frame and a second blurred frame.

[0244] The in-focus frame is blurred by blurring the in-focus frame according to blurring degrees of the first out-of-focus frame and the second out-of-focus frame.

[0245] Specifically, blurring the in-focus frame to obtain the blurred frame may include: determining the blur degree of the blurred frame according to the blur degree of the out-of-focus frame, and convolving the blur degree of the blurred frame with the in-focus frame to obtain the blurred frame. It is understandable that the blur degree can be characterized by a point spread function corresponding to the frame.

[0246] Specifically, determining the blur degree of the blurred frame according to the blur degree of the defocused frame may include: determining the point spread function of the blurred frame, that is, the blur degree of the blurred frame, according to the difference between the object distance of the in-focus frame and the object distance of the defocused frame, and the point spread function corresponding to the in-focus frame.

[0247] Specifically, according to the difference between the object distance of the in-focus frame and the object distance of the out-of-focus frame, the offset distance based on the object distance of the in-focus frame can be determined, and the point spread function calibrated based on the point spread function of the in-focus frame can be determined according to the offset distance, so that the point spread function of the blurred frame can be determined according to the calibrated point spread function.

[0248] It should be noted that each frame has a corresponding point spread function. The point spread function corresponding to the in-focus frame may correspond to a plurality of calibrated point spread functions. The calibrated point spread function may be obtained by offsetting the point spread function corresponding to the in-focus frame.

[0249] Reference Figure 9a As shown, the point spread function K1 corresponding to the focus frame can correspond to multiple calibrated point spread functions, such as calibrated point spread function K2, calibrated point spread function K3, and calibrated point spread function K4. Each calibrated point spread function can correspond to a different degree of blur. That is, each calibrated point spread function can be obtained by diffusion blurring based on the point spread function K1 of the focus frame.

[0250] Specifically, when the object is in focus, a certain degree of defocus is performed, that is, an offset is performed based on the object distance d to obtain an offset distance o, and the point spread function when defocused is determined based on the point spread function when in focus, that is, the calibrated point spread function corresponding to the offset distance o is determined based on the point spread function corresponding to the in-focus frame.

[0251] Continue to refer to Figure 9a Based on the point spread function K1 and the object distance d1, the calibration point spread function K2 and the offset distance o2 based on the object distance d1 corresponding to the calibration point spread function K2 can be determined. The calibration point spread function K3 and the offset distance o3 based on the object distance d1 corresponding to the calibration point spread function K3 can also be determined. The calibration point spread function K4 and the offset distance o4 based on the object distance d1 corresponding to the calibration point spread function K4 can also be determined.

[0252] Reference Figure 9bAs shown, the object distance of the first defocused frame is D1, and the object distance of the focused frame is d1, and the difference between the object distance D1 of the first defocused frame and the object distance d1 of the focused frame is calculated. According to the difference between D1 and d1, the offset distance on based on the object distance d1 can be found, where the difference between D1 and d1 is close to on. According to the offset distance on based on the object distance d1, the calibration point spread function Kn corresponding to the offset distance on based on the object distance d1 can be determined, and the calibration point spread function Kn can be determined as the point spread function of the first blurred frame. In this way, the calibration point spread function Kn is convolved with the focused frame to obtain the first blurred frame.

[0253] Reference Fig.9c As shown, the object distance of the second defocused frame is D2, and the object distance of the focused frame is d1, and the difference between the object distance D2 of the second defocused frame and the object distance d1 of the focused frame is calculated. According to the difference between D2 and d1, the offset distance om based on the object distance d1 can be found, where the difference between D2 and d1 is close to om. According to the offset distance om based on the object distance d1, the calibration point spread function Km corresponding to the offset distance om based on the object distance d1 can be determined, and the calibration point spread function Km can be determined as the point spread function of the second blurred frame. In this way, the calibration point spread function Km is convolved with the focused frame to obtain the second blurred frame.

[0254] S2. Perform registration processing on the first blurred frame, the second blurred frame, the first defocused frame, and the second defocused frame to obtain a first registered frame and a second registered frame.

[0255] The registration process can be performed based on image feature points for matching, and image alignment can be performed based on the matching results. For the related processing flow of the registration process and image alignment, reference can be made to the prior art, which will not be described in detail here.

[0256] It is understandable that when the camera captures a defocused frame, the position of the motor changes, and the photographed object will undergo a displacement change, thus requiring the image to be aligned.

[0257] Specifically, refer to Fig.10 As shown in (1) in FIG. 1 , the first blurred frame and the first defocused frame are registered to obtain a first registered frame. Fig.10 As shown in (2), the second blurred frame and the second defocused frame are registered to obtain a second registered frame.

[0258] S3, performing difference calculation on two frames among the first blurred frame, the second blurred frame, the first defocused frame, the second defocused frame, the first registered frame, and the second registered frame to obtain a first difference frame, a second difference frame, and a third difference frame.

[0259] The difference calculation may be to calculate the difference of each pixel in the image frame. For the relevant processing flow of the difference calculation, reference may be made to the prior art, which will not be described in detail here.

[0260] Specifically, refer to Fig.11 As shown in (1) in FIG. 1 , the first blurred frame and the first registered frame are subjected to difference calculation to obtain a first difference frame. Fig.11 As shown in (2) in FIG. 1 , the second blurred frame and the second registered frame are subjected to difference calculation to obtain a second difference frame. Fig.11 As shown in (3), the first defocused frame and the second defocused frame are difference calculated to obtain a third difference frame.

[0261] Optionally, the third difference frame can be calculated from the blurred frame. Fig.11 As shown in (4), the first blurred frame and the second blurred frame are difference calculated to obtain a third difference frame.

[0262] S4. Perform color space conversion on the first difference frame, the second difference frame and the third difference frame respectively to obtain a first LCH frame, a second LCH frame and a third LCH frame.

[0263] Specifically, the difference frame may be an image frame in the sRGB color space. Fig.12 As shown, the difference frame of the sRGB color space is converted into an RGB frame, and then the RGB frame is converted into a LAB frame, and then the LAB frame is converted into a LCH frame.

[0264] For the relevant processing flow of color space conversion, please refer to the existing technology and will not be repeated here.

[0265] S5. Perform parameter threshold processing on the first LCH frame, the second LCH frame, and the third LCH frame respectively to obtain a first moiré mask, a second moiré mask, and a third moiré mask.

[0266] Parameter threshold processing may include: counting the C color space value and the H color space value of each pixel in the LCH frame, that is, counting the C value and the H value of each pixel in the LCH frame; calculating the average C value or the median C value of the entire image frame in the LCH frame, and calculating the average H value or the median H value of the entire image frame in the LCH frame; and segmenting the pixels in the LCH frame whose C values ​​exceed the average C value or the median C value, and the pixels whose H values ​​exceed the average H value or the median H value.

[0267] Exemplarily, the pixel points whose C value exceeds the average C value or the median C value, and the pixel points whose H value exceeds the average H value or the median H value in the LCH frame are segmented, and the pixel points whose C value exceeds the average C value or the median C value, and the pixel points whose H value exceeds the average H value or the median H value are marked as 1, and the other pixel points are marked as 0. It can be understood that the segmented area composed of the pixel points marked as 1 is the moiré area.

[0268] Specifically, the image frame obtained after pixel segmentation is a moiré mask. It is understandable that in the moiré mask, there may be obvious moiré areas or there may be no obvious moiré areas.

[0269] For example, Fig.13 (1), (2), and (3) are schematic diagrams of moiré masks. Fig.13 In the moiré mask shown in (1) of FIG. 1 , region 401 is an obvious moiré region. Fig.13 In the moiré mask shown in (2) of FIG. 1 , region 402 is an obvious moiré region. Fig.13 In the moiré mask shown in (3), there is no obvious moiré area.

[0270] Optionally, parameter threshold processing may further include: performing morphological processing on the image frames obtained after segmentation.

[0271] Exemplarily, the image frame obtained after segmentation is first corroded and then expanded, or the image frame obtained after segmentation is first expanded and then corroded, or the image frame obtained after segmentation is corroded and expanded multiple times, etc., which is not limited in the present application.

[0272] S6. Analyze the pixels of the first moiré mask, the second moiré mask, and the third moiré mask respectively to obtain moiré detection results.

[0273] The pixel point analysis on the moiré mask may be performed on the segmented pixel points in the moiré mask.

[0274] Segmentation pixels are pixels whose C values ​​in the moiré mask exceed the average C value or the median C value, or whose H values ​​exceed the average H value or the median H value. In the moiré mask, the area where multiple segmentation pixels are connected is the moiré joint area.

[0275] Each moiré mask may include multiple moiré-related regions. In the moiré mask, when the number of pixels included in each moiré-related region is greater than a preset number of pixels, it can be determined that the moiré mask includes moiré.

[0276] Optionally, in a moiré mask, when the number of pixels of a moiré pattern that is greater than a preset number of pixels and the area of ​​the moiré pattern that exceeds a preset number of areas, it can be determined that the moiré pattern mask includes the moiré pattern.

[0277] For example, it is assumed that the moiré mask includes 5000 pixels and 10 moiré patterns with regions, the preset number of pixels is 100, and the preset number of regions is 6. If the number of pixels in the 10 moiré patterns with regions is greater than 100, then it can be determined that the moiré mask includes moiré patterns. Alternatively, if there are 8 moiré patterns with a number of pixels greater than 100 with regions, then it can also be determined that the moiré mask includes moiré patterns.

[0278] The moiré detection result may be the moiré detection result of the in-focus frame, the first defocused frame, and the second defocused frame, that is, the detection result of whether there is moiré in the in-focus frame, the detection result of whether there is moiré in the first defocused frame, and the detection result of whether there is moiré in the second defocused frame.

[0279] Since the moiré mask is obtained by aligning the in-focus frame and the out-focus frame, calculating the difference, converting the color space, etc., after determining whether there is moiré in the moiré mask, it can be determined whether there is moiré in the in-focus frame, the first out-focus frame, and the second out-focus frame.

[0280] Specifically, if there is no moiré in the first moiré mask, it can be determined that there is no moiré in the in-focus frame and the first defocused frame. If there is no moiré in the second moiré mask, it can be determined that there is no moiré in the in-focus frame and the second defocused frame. If there is no moiré in the third moiré mask, it can be determined that there is no moiré in the first defocused frame and the second defocused frame.

[0281] Specifically, if the first moiré mask includes moiré and the third moiré mask does not include moiré, it can be determined that there is moiré in the in-focus frame, and there is no moiré in the first defocus frame and the second defocus frame. Similarly, if the second moiré mask includes moiré and the third moiré mask does not include moiré, it can be determined that there is moiré in the in-focus frame, and there is no moiré in the first defocus frame and the second defocus frame.

[0282] S107: When the moiré removal module determines that the in-focus frame has no moiré, the in-focus frame is sent to the camera application via the camera service.

[0283] S108. When the moiré removal module determines that the focused frame has moiré and at least one defocused frame has no moiré, the module determines a target frame according to the defocused frame without moiré.

[0284] Specifically, if only one defocused frame has no moiré, the defocused frame without moiré can be determined as the target frame. If both defocused frames have no moiré, the two defocused frames can be image fused and the fused frame can be determined as the target frame.

[0285] Exemplarily, assuming that there is no moiré in the first defocused frame and there is moiré in the focused frame, the first defocused frame can be determined as the target frame. Assuming that there is moiré in the focused frame and there is no moiré in the first defocused frame and the second defocused frame, the first defocused frame and the second defocused frame can be image fused to obtain the target frame.

[0286] S109 . When the moiré removal module determines that both the in-focus frame and the two out-of-focus frames have moiré, the two out-of-focus frames are image-fused with the in-focus frame to obtain a target frame.

[0287] Specifically, if the in-focus frame, the first defocused frame, and the second defocused frame all have moiré patterns, the in-focus frame, the first defocused frame, and the second defocused frame may be image fused to obtain a target frame.

[0288] Specifically, the image fusion may be, for example, median filtering between multiple frames to remove moiré patterns in the image frames.

[0289] S110 , the moiré removal module performs aberration calculation on the target frame to obtain a captured frame, and sends the captured frame to the camera application through the camera service.

[0290] Specifically, performing aberration calculation on the target frame may include: determining the target frame and the point spread function corresponding to the target frame, and the point spread function corresponding to the target frame in the Fourier domain and the target frame in the Fourier domain, and determining the shooting frame in the Fourier domain according to the point spread function corresponding to the target frame in the Fourier domain and the target frame in the Fourier domain. In this way, the shooting frame in the Fourier domain may be inversely transformed to obtain the shooting frame.

[0291] If the target frame is a defocused frame without moiré, the point spread function corresponding to the target frame is also the point spread function corresponding to the defocused frame without moiré.

[0292] If the target frame is obtained by image fusion of two defocused frames, the point spread function corresponding to the target frame can be determined according to the point spread functions corresponding to the two defocused frames. Specifically, the point spread function corresponding to the target frame = (point spread function corresponding to the first defocused frame + point spread function corresponding to the second defocused frame) / 2.

[0293] If the target frame is obtained by image fusion of the in-focus frame, the first defocused frame, and the second defocused frame, the point spread function corresponding to the target frame can be determined according to the point spread functions corresponding to the in-focus frame, the first defocused frame, and the second defocused frame. Specifically, the point spread function corresponding to the target frame = (point spread function corresponding to the in-focus frame + point spread function corresponding to the first defocused frame + point spread function corresponding to the second defocused frame) / 3.

[0294] For example, refer to Fig.14a As shown, when there is moiré in the in-focus frame and there is no moiré in one of the defocused frames, that is, when the target frame is a defocused frame without moiré, the point spread function corresponding to the defocused frame without moiré can be determined. Then, based on the defocused frame without moiré, the Fourier domain defocused frame corresponding to the defocused frame without moiré is determined, and based on the point spread function corresponding to the defocused frame without moiré, the Fourier domain point spread function corresponding to the point spread function corresponding to the defocused frame without moiré is determined. Thus, based on the Fourier domain defocused frame and the Fourier domain point spread function, the Fourier domain captured frame can be determined. In this way, the Fourier domain captured frame is subjected to an inverse Fourier transform to obtain the captured frame.

[0295] Reference Fig.14b As shown, when the in-focus frame has moiré and the two defocus frames have no moiré, that is, when the target frame is obtained by fusing the two defocus frames, the first defocus frame and the second defocus frame can be fused to obtain a fused frame (that is, the target frame), and then the Fourier domain fused frame corresponding to the fused frame is determined. According to the point spread function Kx of the first defocus frame and the point spread function Ky of the second defocus frame, the point spread function K=(Kx+Ky) / 2 is determined, and then the Fourier domain point spread function corresponding to the point spread function is determined. Therefore, according to the Fourier domain fused frame and the Fourier domain point spread function, the Fourier domain captured frame can be determined. In this way, the Fourier domain captured frame is subjected to an inverse Fourier transform to obtain the captured frame.

[0296] Reference Fig.14c As shown, when the in-focus frame and the two defocus frames have moiré patterns, that is, when the target frame is obtained by fusing the in-focus frame and the defocus frame, the in-focus frame, the first defocus frame, and the second defocus frame can be image fused to obtain a fused frame (that is, the target frame), and then the Fourier domain fused frame corresponding to the fused frame is determined. According to the point spread function K1 of the in-focus frame, the point spread function Kx of the first defocus frame, and the point spread function Ky of the second defocus frame, the point spread function K=(K1+Kx+Ky) / 3 is determined, and then the Fourier domain point spread function corresponding to the point spread function is determined. Therefore, according to the Fourier domain fused frame and the Fourier domain point spread function, the Fourier domain captured frame can be determined. In this way, the Fourier domain captured frame is subjected to an inverse Fourier transform to obtain the captured frame.

[0297] It should be pointed out that in the above embodiments, the camera application is used as an example for explanation, and the same is true for other third-party applications with camera functions, which will not be described in detail here.

[0298] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the shooting method in the above-mentioned embodiment.

[0299] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the shooting method in the above-mentioned embodiment.

[0300] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the shooting method in the above-mentioned method embodiments.

[0301] Among them, the electronic device (such as a mobile phone, etc.), computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0302] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0303] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0304] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A shooting method, characterized in that: The method comprises: In a moiré removal scenario of a target application, determining a first position and a second position corresponding to a camera motor; wherein the number of the second position is at least one; In response to the target operation, collecting a first frame corresponding to the first position and a second frame corresponding to the second position; wherein the number of the second frames is the same as the number of the second positions; Performing moiré detection on the first frame and the second frame; When moiré patterns exist in the first frame, determining a target frame according to the first frame and the second frame, or determining the target frame according to the second frame; Performing aberration calculation on the target frame to obtain a shooting frame corresponding to the target operation; Wherein, determining the second position corresponding to the camera motor includes: After determining the first position, determining a third position, and calculating the depth of field data corresponding to the first position; wherein the third position is the focal position when the camera is focusing; Determine a fourth position according to the depth of field data and the third position; wherein the fourth position is a focus position when the camera is out of focus; According to the fourth position, the second position corresponding to the camera motor is determined; wherein the distance between the second position and the fourth position is the same as the distance between the third position and the first position.

2. The method according to claim 1, characterized in that The calculating the depth of field data corresponding to the first position includes: According to the formula , calculating the depth of field data corresponding to the first position; Among them, DOF is the depth of field data; u is the distance between the third position and the first position; F# is the aperture number of the camera; coc is the diameter of the circle of confusion when the camera motor is in the first position; f is the focal length of the camera.

3. The method according to claim 1, characterized in that The step of determining the fourth position according to the depth of field data and the third position includes: Determine a first depth of field position according to the depth of field data; wherein the first depth of field position is a position of the foreground depth of field; Determining a first intermediate position between the first depth of field position and the third position according to the first depth of field position and the third position; The fourth position is determined within a range between the first intermediate position and the first depth of field position.

4. The method according to claim 1, characterized in that: The step of determining the fourth position according to the depth of field data and the third position includes: Determine a second depth of field position according to the depth of field data; wherein the second depth of field position is a position of a rear depth of field; Determine a second intermediate position between the second depth of field position and the third position according to the second depth of field position and the third position; The fourth position is determined within the range between the second intermediate position and the second depth of field position.

5. The method according to claim 1, characterized in that The performing moiré detection on the first frame and the second frame includes: Performing a first process on the first frame to obtain a third frame corresponding to the second frame; wherein the number of the third frames is the same as the number of the second frames; Performing a second process on the third frame to obtain a fourth frame; The fourth frame is subjected to a third processing to obtain moiré detection results of the first frame and the second frame.

6. The method according to claim 5, characterized in that The performing a first process on the first frame to obtain a third frame corresponding to the second frame includes: determining a first difference between an object distance corresponding to the first frame and an object distance corresponding to the second frame; determining a first function corresponding to the first difference; The first function and the first frame are convolved to obtain a third frame corresponding to the second frame.

7. The method according to claim 6, characterized in that The determining a first function corresponding to the first difference includes: determining a second function corresponding to the first frame; Determine a target function and a target distance corresponding to the target function according to the object distance corresponding to the first frame and the second function; wherein the target distance is an offset distance based on the object distance corresponding to the first frame; and the number of the target function and the target distance is at least one; In the target distances, searching for a first target distance; wherein the difference between the first target distance and the first difference is less than a difference threshold; A first objective function corresponding to the first target distance is determined as a first function corresponding to the first difference.

8. The method according to claim 5, characterized in that The performing a second process on the third frame to obtain a fourth frame comprises: Performing registration processing on the third frame and the second frame to obtain a fifth frame corresponding to the third frame; wherein the number of the fifth frame is the same as the number of the third frame; Performing difference calculation on the fifth frame, the third frame and / or the second frame to obtain the fourth frame; Wherein, when the number of the second frames is multiple, performing difference calculation on the fifth frame, the third frame and / or the second frame includes: performing difference calculation on each of the second frames, or performing difference calculation on each of the third frames; A difference calculation is performed between the third frame and a fifth frame corresponding to the third frame.

9. The method according to claim 8, characterized in that When the number of the second frames is plural, the registering the third frame with the second frame comprises: The second frame and a third frame corresponding to the second frame are registered.

10. The method according to claim 5, characterized in that The performing the third processing on the fourth frame to obtain the moiré detection results of the first frame and the second frame includes: Performing color space conversion on the fourth frame to obtain a sixth frame; wherein the sixth frame is an image frame in the LCH color space; Threshold segmentation and pixel analysis are performed on the sixth frame to obtain moiré detection results of the first frame and the second frame.

11. The method according to claim 10, characterized in that The performing threshold segmentation and pixel point analysis on the sixth frame includes: Determine a first value and a second value of each pixel in the sixth frame; Segmenting the pixel points whose first value and / or second value satisfy a second condition; In the image frame after pixel segmentation, determining an area where the number of pixels meets the third condition; When the number of the regions satisfies a fourth condition, determining that a second frame corresponding to the fourth frame and the first frame have moiré patterns; When the number of the regions does not satisfy a fourth condition, determining that the first frame and / or the second frame corresponding to the fourth frame does not have moiré patterns; The second condition includes whether the first value or the second value of the pixel is greater than the average value of the first value or the second value of each pixel in the sixth frame, or whether the first value or the second value of the pixel is greater than the median value of the first value or the second value of each pixel in the sixth frame; The third condition is whether the number of pixels in the moiré pattern and the area in the image frame after pixel segmentation is greater than a preset number of pixels; The fourth condition is whether the number of moiré patterns and regions in the image frame after pixel segmentation is greater than a preset number of moiré patterns and regions.

12. The method according to claim 1, characterized in that When the number of the second frames is one, determining a target frame according to the first frame and the second frame, or determining the target frame according to the second frame, comprises: When moiré patterns exist in the second frame, the first frame and the second frame are image-fused to obtain the target frame; When the second frame does not have moiré patterns, the second frame is determined as the target frame.

13. The method according to claim 1, characterized in that When the number of the second frames is plural, determining the target frame according to the first frame and the second frame, or determining the target frame according to the second frame, comprises: When moiré patterns exist in each of the second frames, performing image fusion on the first frame and the second frame to obtain the target frame; When each of the second frames includes a second frame without moiré patterns, determining the second frame without moiré patterns as the target frame; When each of the second frames includes a plurality of second frames without moiré patterns, one of the second frames without moiré patterns is determined as the target frame, or the target frame is obtained by performing image fusion on the second frames without moiré patterns.

14. The method according to claim 1, characterized in that The performing aberration calculation on the target frame to obtain a shooting frame corresponding to the target operation includes: Determine a third function corresponding to the target frame; Determine the target frame in Fourier domain and the third function in Fourier domain according to the target frame and the third function corresponding to the target frame; Determining a captured frame in the Fourier domain according to the target frame in the Fourier domain and the third function in the Fourier domain; Performing an inverse Fourier transform on the captured frame in the Fourier domain to obtain a captured frame corresponding to the target operation.

15. The method according to claim 14, characterized in that The determining of the third function corresponding to the target frame comprises: determining a second function corresponding to the first frame and a fourth function corresponding to the second frame; When the target frame is determined by the first frame and the second frame, an average function of the second function and the fourth function is determined as the third function; When the target frame is determined by one of the second frames, determining the fourth function as the third function; When the target frame is determined by a plurality of the second frames, an average function of the fourth functions is determined as the third function.

16. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the shooting method as described in any one of claims 1-15.

17. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is enabled to execute the shooting method according to any one of claims 1 to 15.

Images